Method and device for indicating fusion mode candidate number
By obtaining and adjusting indicator values in the code stream in video encoding, optimizing the fusion motion vector prediction and geometric division motion compensation of video sequences, the problem of low video encoding efficiency in the prior art is solved, and a higher code stream is achieved Utilization and decoding efficiency.
Patent Information
- Application Number
- CN202410391282.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2020-01-14
- Filing Date
- 2021-01-13
- Publication Date
- 2024-11-12
AI Technical Summary
When the existing video encoding technology transmits and stores video data in a communication network with limited bandwidth, it is difficult to achieve compression ratio improvement without affecting the image quality, and it is difficult to efficiently indicate the number of candidates for the fusion mode, affecting the utilization rate of the code stream. and decoding efficiency.
By obtaining the indicator value in the code stream of the video sequence, we judge the number of fusion motion vector prediction candidates and whether geometric division motion compensation is enabled, and adjust the maximum number of geometric division fusion mode candidates according to the threshold and preset values, thereby optimizing the code stream and decoding efficiency.
It improves the utilization rate of code streams and decoding efficiency, enhances the performance of video encoding and decoding, and adapts to the challenges of limited network resources and high video quality requirements.
Smart Images

Figure CN118945378A_ABST
Abstract
Description
[0001] This application is a divisional application. The application number of the original application is 202180007546.8, and the original application date is January 13, 2021. The entire contents of the original application are incorporated into this application by reference.
[0002] Cross-reference to related applications
[0003] This patent application claims priority to US62 / 961,159 filed on January 14, 2020. The entire disclosure of the above patent application is incorporated by reference into this application. Technical Field
[0004] Embodiments of the present application generally relate to the field of dynamic image coding, and more specifically to indicating the number of candidate fusion modes. Background Art
[0005] Video coding (video encoding and decoding) is widely used in digital video applications such as broadcast digital television, video transmission over the Internet and mobile networks, real-time conversation applications such as video chat and video conferencing, DVD and Blu-ray Discs, video content acquisition and editing systems, and camcorders for security applications.
[0006] Even if a video is relatively short, a large amount of video data is required to describe it, which can cause difficulties when the data is to be streamed or otherwise transmitted over a communication network with limited bandwidth capacity. Therefore, video data is often compressed before being transmitted over modern telecommunications networks. Since memory resources may be limited, the size of the video may also be an issue when storing the video in a storage device. Video compression devices typically encode the video data using software and / or hardware on the source side and then transmit or store the video data, thereby reducing the amount of data required to represent the digital video image. A video decompression device that decodes the video data then receives the compressed data on the destination side. With limited network resources and a growing demand for higher video quality, there is a need for improved compression and decompression techniques that can increase the compression ratio with little impact on image quality. Summary of the invention
[0007] Embodiments of the present application provide an apparatus and method for encoding and decoding as described in the independent claims.
[0008] The above and other objects are achieved by the subject matter claimed in the independent claims. Other implementations are apparent from the dependent claims, the description and the drawings.
[0009] Particular embodiments are outlined in the accompanying independent claims, further embodiments are outlined in the dependent claims.
[0010] A first aspect of the present invention provides a method for obtaining a maximum number of geometric partition fusion mode candidates for video decoding. The method comprises:
[0011] Acquire a bitstream of a video sequence; acquire a value of a first indicator according to the bitstream, wherein the first indicator indicates a maximum number of fused motion vector prediction (MVP) candidates; acquire a value of a second indicator according to the bitstream, wherein the second indicator indicates whether geometric partitioning-based motion compensation is enabled for the video sequence; when the value of the first indicator is greater than a threshold and the value of the second indicator is equal to a preset value, parse a value of a third indicator from the bitstream, wherein the third indicator indicates subtracting the maximum number of geometric partitioning fusion mode candidates from the value of the first indicator.
[0012] According to an embodiment of the present invention, a scheme for an indicator indicating the number of fusion mode candidates is disclosed. The maximum number of geometric partition fusion mode candidates is indicated under certain conditions. Therefore, code stream utilization and decoding efficiency are improved.
[0013] In one implementation, the method further includes: when the value of the first indicator is equal to the threshold and the value of the second indicator is equal to the preset value, setting the value of the maximum number of the geometric partitioning fusion mode candidates to 2.
[0014] In one implementation, the method further includes: when the value of the first indicator is less than the threshold or the value of the second indicator is not equal to the preset value, setting the value of the maximum number of geometric partitioning fusion mode candidates to 0.
[0015] In one implementation, the threshold is 2.
[0016] In one implementation, the threshold is 1.
[0017] In one implementation, the step of obtaining the value of the second indicator is performed after the step of obtaining the value of the first indicator.
[0018] In an implementation manner, the first indicator is obtained according to a syntax element decoded in the code stream.
[0019] In one implementation, when the value of the first indicator is greater than or equal to the threshold, the value of the second indicator is parsed from a sequence parameter set (SPS) of the code stream. For example, a syntax element in a sequence parameter set (SPS) of the code stream is parsed to obtain the value of the second indicator.
[0020] In one implementation, the value of the second indicator is obtained from a sequence parameter set (SPS) of the code stream, for example, by parsing syntax elements in a sequence parameter set (SPS) of the code stream to obtain the value of the second indicator.
[0021] In one implementation, the value of the third indicator is obtained from a sequence parameter set (SPS) of the code stream. For example, the syntax elements in the sequence parameter set (SPS) of the code stream are parsed to obtain the value of the second indicator.
[0022] The second aspect of the present invention provides a video decoding device. The video decoding device includes: a receiving module, which is used to obtain a code stream of a video sequence; an acquisition module, which is used to obtain the value of a first indicator according to the code stream, wherein the first indicator indicates the maximum number of fused motion vector prediction (motion vector prediction, MVP) candidates; the acquisition module is used to obtain the value of a second indicator according to the code stream, wherein the second indicator indicates whether geometric partition-based motion compensation is enabled for the video sequence; a parsing module, which is used to: when the value of the first indicator is greater than a threshold value and the value of the second indicator is equal to a preset value, parse the value of a third indicator from the code stream, wherein the third indicator indicates the maximum number of geometric partition fusion mode candidates subtracted from the value of the first indicator.
[0023] The method provided in the first aspect of the present invention may be performed by the device provided in the second aspect of the present invention. Other features and implementations of the method provided in the first aspect of the present invention correspond to the features and implementations of the device provided in the second aspect of the present invention.
[0024] In one implementation, the acquisition module is used to: when the value of the first indicator is equal to the threshold and the value of the second indicator is equal to the preset value, set the value of the maximum number of the geometric partitioning fusion mode candidates to 2.
[0025] In one implementation, the acquisition module is used to set the value of the maximum number of geometric partitioning fusion mode candidates to 0 when the value of the first indicator is less than the threshold or the value of the second indicator is not equal to the preset value.
[0026] In one implementation, the threshold is 2.
[0027] In one implementation, the threshold is 1.
[0028] In one implementation, the step of obtaining the value of the second indicator is performed after the step of obtaining the value of the first indicator.
[0029] In one implementation, when the value of the first indicator is greater than or equal to the threshold, the value of the second indicator is parsed from a sequence parameter set (SPS) of the code stream.
[0030] In one implementation, the value of the second indicator is obtained from a sequence parameter set (SPS) of the code stream.
[0031] In one implementation, the value of the third indicator is obtained from a sequence parameter set (SPS) of the code stream.
[0032] In one implementation, a method for obtaining a maximum number of geometric partition fusion mode candidates for video decoding is disclosed. The method includes:
[0033] Obtain a bitstream of a video sequence; obtain a value of a first indicator according to the bitstream, wherein the first indicator indicates a maximum number of fused motion vector prediction (MVP) candidates; obtain a value of a second indicator according to the bitstream as long as the value obtained for the first indicator is equal to or greater than a threshold, wherein the second indicator indicates whether geometric partitioning-based motion compensation is enabled for the video sequence; and parse a value of a third indicator from the bitstream only when the value of the first indicator is greater than the threshold and the value of the second indicator is equal to a preset value, wherein the third indicator indicates the maximum number of geometric partitioning fusion mode candidates subtracted from the value of the first indicator.
[0034] A third aspect of the present invention provides a method for encoding the maximum number of geometric partition fusion mode candidates. The method comprises:
[0035] Determine a value of a first indicator, wherein the first indicator represents a maximum number of fused motion vector prediction (MVP) candidates; determine a value of a second indicator, wherein the second indicator represents whether geometric partitioning-based motion compensation is enabled for a video sequence; when the value of the first indicator is greater than a threshold and the value of the second indicator is equal to a preset value, encode a value of a third indicator into a bitstream, wherein the third indicator represents the maximum number of geometric partitioning fusion mode candidates subtracted from the value of the first indicator.
[0036] According to an embodiment of the present invention, a scheme for an indicator indicating the number of fusion mode candidates is disclosed. The maximum number of geometric partition fusion mode candidates is indicated under certain conditions. Therefore, code stream utilization and decoding efficiency are improved.
[0037] In one implementation, the method further includes: when the value of the first indicator is equal to the threshold and the value of the second indicator is equal to the preset value, setting the value of the maximum number of the geometric partitioning fusion mode candidates to 2.
[0038] In one implementation, the method further includes: when the value of the first indicator is less than the threshold or the value of the second indicator is not equal to the preset value, setting the value of the maximum number of geometric partitioning fusion mode candidates to 0.
[0039] In one implementation, the threshold is 2.
[0040] In one implementation, the threshold is 1.
[0041] In one implementation, the step of determining the value of the second indicator is performed after the step of determining the value of the first indicator.
[0042] In one implementation, when the value of the first indicator is greater than or equal to the threshold, the value of the second indicator is encoded into a sequence parameter set (SPS) of the code stream.
[0043] In one implementation, the value of the second indicator is encoded in a sequence parameter set (SPS) of the code stream.
[0044] In one implementation, the value of the third indicator is encoded in a sequence parameter set (SPS) of the code stream.
[0045] The fourth aspect of the present invention provides a video encoding device. The video encoding device includes: a determination module, which is used to determine the value of a first indicator, wherein the first indicator represents the maximum number of fused motion vector prediction (MVP) candidates; the determination module is used to determine the value of a second indicator, wherein the second indicator represents whether geometric partition-based motion compensation is enabled for a video sequence; and an encoding module, which is used to: when the value of the first indicator is greater than a threshold value and the value of the second indicator is equal to a preset value, encode the value of a third indicator into a bitstream, wherein the third indicator represents the maximum number of geometric partition fusion mode candidates subtracted from the value of the first indicator.
[0046] The method provided in the third aspect of the present invention may be performed by the device provided in the fourth aspect of the present invention. Other features and implementations of the method provided in the third aspect of the present invention correspond to the features and implementations of the device provided in the fourth aspect of the present invention.
[0047] In one implementation, the determination module is used to: when the value of the first indicator is equal to the threshold and the value of the second indicator is equal to the preset value, set the value of the maximum number of the geometric partitioning fusion mode candidates to 2.
[0048] In one implementation, the determination module is used to: when the value of the first indicator is less than the threshold or the value of the second indicator is not equal to the preset value, set the value of the maximum number of geometric partitioning fusion mode candidates to 0.
[0049] In one implementation, the threshold is 2.
[0050] In one implementation, the threshold is 1.
[0051] In one implementation, the step of determining the value of the second indicator is performed after the step of determining the value of the first indicator.
[0052] In one implementation, when the value of the first indicator is greater than or equal to the threshold, the value of the second indicator is encoded into a sequence parameter set (SPS) of the code stream.
[0053] In one implementation, the value of the second indicator is encoded in a sequence parameter set (SPS) of the code stream.
[0054] In one implementation, the value of the third indicator is encoded in a sequence parameter set (SPS) of the code stream.
[0055] A fifth aspect of the present invention provides a decoder, wherein the decoder comprises a processing circuit for executing the method provided by the first aspect and any one of the implementation modes of the first aspect.
[0056] A sixth aspect of the present invention provides an encoder, wherein the encoder comprises a processing circuit for executing the method provided by the third aspect and any one of the implementation modes of the third aspect.
[0057] A seventh aspect of the present invention provides a computer program. The computer program includes program code. When the program code is executed on a computer or a processor, it is used to execute the method provided by any one of the first aspect, the third aspect, and the first aspect and the third aspect.
[0058] The eighth aspect of the present invention provides a decoder. The decoder comprises: one or more processors; a non-transitory computer-readable storage medium, coupled to the one or more processors and storing a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the decoder is used to execute the method provided in any one of the first aspect, the third aspect, and any one of the implementations of the first aspect and the third aspect.
[0059] A ninth aspect of the present invention provides a non-transitory computer-readable storage medium including program code. When the program code is executed by a computer device, the computer device executes the method provided by any one of the first aspect, the third aspect, and any one of the implementations of the first aspect and the third aspect.
[0060] A tenth aspect of the present invention provides an encoder, wherein the encoder comprises a processing circuit for executing the method provided by the third aspect and any one of the implementation modes of the third aspect.
[0061] The eleventh aspect of the present invention provides an encoder, comprising: one or more processors; a non-transitory computer-readable storage medium, coupled to the one or more processors and storing a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the decoder is used to execute the method provided in any one of the third aspect and any one of the implementations of the third aspect.
[0062] A twelfth aspect of the present invention provides a non-transitory storage medium, comprising a code stream encoded / decoded by the method of any of the above embodiments.
[0063] A thirteenth aspect of the present invention provides an encoded bitstream of a video signal, comprising a plurality of syntax elements, wherein the plurality of syntax elements include a second indicator (eg, sps_geo_enabled_flag), and a third indicator sps_max_num_merge_cand_minus_max_num_geo_cand is indicated at least according to the value of sps_geo_enabled_flag under certain conditions.
[0064] A fourteenth aspect of the present invention provides a non-transitory storage medium, comprising an encoded code stream decoded by an image decoding device, wherein the code stream is generated by dividing a frame in a video signal or an image signal into multiple blocks, and the code stream includes multiple syntax elements, and the multiple syntax elements include a third indicator (e.g., sps_max_num_merge_cand_minus_max_num_geo_cand).
[0065] A fifteenth aspect of the present invention provides a method for video decoding. The method comprises:
[0066] Acquire a bitstream of a video sequence; acquire a value of a first indicator according to the bitstream, wherein the first indicator indicates a maximum number of fused motion vector prediction (motion vector prediction, MVP) candidates; acquire a value of a second indicator according to the bitstream, wherein the second indicator indicates whether geometric partition-based motion compensation is enabled for the video sequence; when the value of the first indicator is greater than a threshold and the value of the second indicator is equal to a preset value, parse a value of a third indicator from the bitstream, wherein the third indicator indicates subtracting the maximum number of geometric partition fusion mode candidates from the value of the first indicator;
[0067] Constructing a fusion candidate list of the current coding block according to motion vectors of adjacent blocks of the current coding block;
[0068] Obtaining a fusion index according to a value of the third indicator;
[0069] Acquire a motion vector of the current coding block according to the fusion index and the fusion candidate list;
[0070] The current coding block is reconstructed according to the motion vector of the current coding block.
[0071] The sixteenth aspect of the present invention provides a video decoding device. The video decoding device includes: a receiving module, which is used to obtain a code stream of a video sequence; an acquisition module, which is used to obtain the value of a first indicator according to the code stream, wherein the first indicator indicates the maximum number of fused motion vector prediction (motion vector prediction, MVP) candidates; the acquisition module is used to obtain the value of a second indicator according to the code stream, wherein the second indicator indicates whether geometric partition-based motion compensation is enabled for the video sequence; a parsing module, which is used to: when the value of the first indicator is greater than a threshold value and the value of the second indicator is equal to a preset value, parse the value of a third indicator from the code stream, wherein the third indicator indicates the maximum number of geometric partition fusion mode candidates subtracted from the value of the first indicator;
[0072] A fusion candidate list construction module, used to construct a fusion candidate list of the current coding block according to the motion vectors of the adjacent blocks of the current coding block;
[0073] The acquisition module is used to acquire a fusion index according to the value of the third indicator;
[0074] A motion vector acquisition module, configured to acquire a motion vector of the current coding block according to the fusion index and the fusion candidate list;
[0075] A pixel reconstruction module is used to reconstruct the current coding block according to the motion vector of the current coding block.
[0076] For details or examples of the fifteenth aspect of the present invention and the sixteenth aspect of the present invention, reference may be made to the examples disclosed in the first to fourteenth aspects of the present invention.
[0077] The above and other objects are achieved by the subject matter claimed in the independent claims. Other implementations are apparent from the dependent claims, the description and the drawings.
[0078] The following drawings and description set forth one or more embodiments in detail. Other features, objects, and advantages are apparent in the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0079] Embodiments of the present invention are described in more detail below with reference to the accompanying drawings and schematic diagrams.
[0080] Figure 1A The block diagram is an example of a video decoding system for implementing an embodiment of the present invention.
[0081] Figure 1B FIG. 4 is a block diagram of another example of a video decoding system for implementing an embodiment of the present invention.
[0082] Figure 2 A block diagram of an example of a video encoder for implementing an embodiment of the present invention.
[0083] Figure 3 The block diagram is an exemplary structure of a video decoder for implementing an embodiment of the present invention.
[0084] Figure 4 A block diagram of an example of an encoding device or a decoding device.
[0085] Figure 5 A block diagram of another example of an encoding device or a decoding device.
[0086] Figure 6 Flowchart for weighted predictive encoder decision making and parameter estimation.
[0087] Figure 7 An example of triangular prediction mode is shown.
[0088] Figure 8 An example of geometric prediction mode is shown.
[0089] Fig. 9 Another example of geometric prediction mode is shown.
[0090] Fig.10 A block diagram of an exemplary structure of a content provision system 3100 for implementing content distribution services.
[0091] Fig.11 A block diagram of an exemplary structure of a terminal device.
[0092] Fig.12 A block diagram of an example of the inter-frame prediction method provided in the present application.
[0093] Fig.13 The present invention is a block diagram of an example of an apparatus for inter-frame prediction provided by the present application.
[0094] Fig.14 The present invention is a block diagram of another example of an apparatus for inter-frame prediction provided by the present application.
[0095] Fig.15 The present invention provides a flowchart of a method embodiment.
[0096] Fig.16 A block diagram of an embodiment of the device provided by the present invention.
[0097] In the following, identical reference numerals refer to identical features or at least functionally equivalent features, unless explicitly stated otherwise. DETAILED DESCRIPTION
[0098] In the following description, reference is made to the accompanying drawings that form part of the present invention and show, by way of illustration, specific aspects of embodiments of the present invention or specific aspects in which embodiments of the present invention may be used. It should be understood that embodiments of the present invention may be used in other aspects and may include structural changes or logical changes not depicted in the accompanying drawings. Therefore, the following detailed description should not be understood in a restrictive sense, and the scope of the present invention is defined by the appended claims.
[0099] For example, it should be understood that the disclosure related to describing a method may also be applicable to a corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units such as a functional unit to perform the one or more method steps described (e.g., one unit performs one or more steps, or multiple units perform one or more of the multiple steps respectively), even if such one or more units are not explicitly described or illustrated in the drawings. In addition, if a specific device is described according to one or more units such as a functional unit, the corresponding method may include a step to perform the function of one or more units (e.g., one step performs the function of one or more units, or multiple steps perform the function of one or more of the multiple units respectively), even if such one or more units are not explicitly described or illustrated in the drawings. Further, it is understood that, unless otherwise explicitly stated, the features of the exemplary embodiments and / or aspects described herein may be combined with each other.
[0100] Video decoding generally refers to processing a sequence of images that form a video or video sequence. In the field of video coding, the terms "frame" and "picture / image" can be used as synonyms. Video decoding (or commonly referred to as decoding) includes two parts: video encoding and video decoding. Video encoding is performed on the source side and generally includes processing (for example, by compression) the original video image to reduce the amount of data required to represent the video image (thereby storing and / or transmitting more efficiently). Video decoding is performed on the destination side and generally includes inverse processing relative to the encoder to reconstruct the video image. The "decoding" of the video image (or commonly referred to as the image) involved in the embodiment should be understood as involving the "encoding" or "decoding" of the video image or the corresponding video sequence. The encoding part and the decoding part are also collectively referred to as codec (CODEC) (encoding and decoding).
[0101] In the case of lossless video coding, the original video image can be reconstructed, that is, the reconstructed video image has the same quality as the original video image (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed by quantization, etc. to reduce the amount of data required to represent the video image, but the decoder side cannot fully reconstruct the video image, that is, the quality of the reconstructed video image is lower or worse than the quality of the original video image.
[0102] Several video coding standards belong to the group of "lossy hybrid video codecs" (i.e., combining spatial prediction and temporal prediction in the sample domain with two-dimensional transform coding in the transform domain for applying quantization). Each picture in a video sequence is usually divided into a set of non-overlapping blocks, which are usually decoded at the block level. In other words, the encoder side usually processes, i.e., encodes, the video at the block (video block) level, for example, by generating a prediction block through spatial (intra-frame) prediction and / or temporal (inter-frame) prediction, subtracting the prediction block from the current block (currently processed / to be processed block) to obtain a residual block, transforming the residual block in the transform domain and quantizing the residual block to reduce the amount of data to be transmitted (compressed), while the decoder side applies the inverse processing part relative to the encoder to the coded or compressed block to reconstruct the current block for representation. In addition, the encoder replicates the decoder processing cycle, so that the encoder and the decoder generate the same prediction (e.g., intra-frame prediction and inter-frame prediction) and / or reconstruction for processing, i.e., decoding subsequent blocks.
[0103] In the following embodiment of the video decoding system 10, the video encoder 20 and the video decoder 30 are based on Figure 1A , Figure 1B , Figure 2 and Figure 3 Give a description.
[0104] Figure 1A 1 is a schematic block diagram of an example decoding system 10, for example, a video decoding system 10 (or simply a decoding system 10) that can utilize the techniques in the present application. The video encoder 20 (or simply an encoder 20) and the video decoder 30 (or simply a decoder 30) in the video decoding system 10 are two examples, i.e., devices that can be used to perform various techniques according to various examples described in the present application.
[0105] like Figure 1A As shown, the decoding system 10 includes a source device 12 for providing encoded image data 21 to a destination device 14 or the like for decoding the encoded image data 21 .
[0106] The source device 12 includes an encoder 20 and may additionally (ie, optionally) include an image source 16 , a pre-processor (or pre-processing unit) 18 (eg, an image pre-processor 18 ), and a communication interface or communication unit 22 .
[0107] The image source 16 may include or may be any type of image capture device for capturing real-world images, etc.; and / or any type of image generation device (e.g., a computer graphics processor for generating computer-animated images); or any type of device for acquiring and / or providing real-world images, computer-animated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory / storage for storing any of the above images.
[0108] In order to distinguish the pre-processor 18 and the processing performed by the pre-processing unit 18 , the image or image data 17 may also be referred to as a raw image or raw image data 17 .
[0109] The preprocessor 18 is used to receive (raw) image data 17 and perform preprocessing on the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. The preprocessing performed by the preprocessor 18 may include trimming, color format conversion (e.g., from RGB to YCbCr), color adjustment or denoising, etc. It can be understood that the preprocessing unit 18 may be an optional component.
[0110] The video encoder 20 is used to receive the pre-processed image data 19 and provide the encoded image data 21 (combined with Figure 2 etc. for more details).
[0111] The communication interface 22 in the source device 12 can be used to receive the encoded image data 21 and send the encoded image data 21 (or data obtained after further processing of the encoded image data 21) to another device (such as the destination device 14) or any other device through the communication channel 13 for storage or direct reconstruction.
[0112] The destination device 14 includes a decoder 30 (eg, a video decoder 30 ), and may additionally (ie, optionally) include a communication interface or communication unit 28 , a post-processor 32 (or a post-processing unit 32 ), and a display device 34 .
[0113] The communication interface 28 in the destination device 14 is used to receive the encoded image data 21 (or data obtained after further processing of the encoded image data 21) directly from the source device 12 or from any other source such as a storage device (such as an encoded image data storage device), and provide the encoded image data 21 to the decoder 30.
[0114] The communication interface 22 and the communication interface 28 can be used to send or receive the encoded image data 21 or the encoded data 21 via a direct communication link (e.g., a direct wired or wireless connection) between the source device 12 and the destination device 14 or via any type of network (e.g., a wired network, a wireless network, or any combination thereof, or any type of private and public network, or any type of combination thereof).
[0115] For example, the communication interface 22 may be used to encapsulate the encoded image data 21 into a suitable format (eg, data packets), and / or process the encoded image data by any type of transmission coding or processing for transmission over a communication link or communication network.
[0116] For example, the communication interface 28 corresponding to the communication interface 22 may be used to receive the transmission data and process the transmission data by any type of corresponding transmission decoding or processing and / or decapsulation to obtain the encoded image data 21 .
[0117] Both the communication interface 22 and the communication interface 28 can be configured as Figure 1A The unidirectional communication interface indicated by the arrow of the communication channel 13 pointing from the source device 12 to the destination device 14, or is configured as a bidirectional communication interface, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission (such as encoded image data transmission), etc.
[0118] The decoder 30 is used to receive the encoded image data 21 and provide decoded image data 31 or decoded image 31 (hereinafter referred to as Figure 3 or Figure 5 etc. for more details).
[0119] The post-processor 32 in the destination device 14 is used to post-process the decoded image data 31 (also referred to as reconstructed image data) (e.g., decoded image 31) to obtain post-processed image data 33 (e.g., post-processed image 33). The post-processing performed by the post-processing unit 32 may include color format conversion (e.g., from YCbCr to RGB), color adjustment, trimming or resampling, or any other processing to provide the decoded image data 31 for display by the display device 34, etc.
[0120] The display device 34 in the destination device 14 is used to receive the post-processed image data 33 so as to display the image to a user or viewer, etc. The display device 34 can be or include any type of display for representing the reconstructed image, such as an integrated or external display or display screen. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display.
[0121] Although Figure 1A 1 and 14 are shown as separate devices, but the device embodiment may also include both devices or the functions of both devices, that is, the source device 12 or the corresponding functions and the destination device 14 or the corresponding functions. In these embodiments, the source device 12 or the corresponding functions and the destination device 14 or the corresponding functions may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.
[0122] It is obvious to those skilled in the art based on the description that the functions or Figure 1A The presence and (exact) division of functionality of the source device 12 and / or destination device 14 shown may vary depending on the actual devices and applications.
[0123] The encoder 20 (eg, video encoder 20) or the decoder 30 (eg, video decoder 30) or both the encoder 20 and the decoder 30 may be configured to: Figure 1B The encoder 20 may be implemented by the processing circuit 46, such as one or more microprocessors, one or more digital signal processors (DSP), one or more application-specific integrated circuits (ASIC), one or more field-programmable gate arrays (FPGA), one or more discrete logic, one or more hardware, one or more video decoding dedicated processors, or any combination thereof. The encoder 20 may be implemented by the processing circuit 46 to include reference Figure 2 The various modules described in the encoder 20 in the embodiment and / or any other encoder system or subsystem described herein. The decoder 30 may be implemented by the processing circuit 46 to include reference Figure 3 The various modules described in the decoder 30 of FIG. 1 and / or any other decoder system or subsystem described herein. The processing circuit may be used to perform the various operations described below. Figure 5 As shown, if the above-mentioned technology is partially implemented in software, a device can store the instructions of the software in a suitable non-transitory computer-readable medium, and can use one or more processors to execute these instructions in hardware to perform the technology in the present invention. The video encoder 20 or the video decoder 30 can be integrated into a single device as part of a combined codec (CODEC), such as Figure 1B shown.
[0124] Source device 12 and destination device 14 may include any of a variety of devices, including any type of handheld or fixed device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet or a tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content distribution server), a broadcast receiver device, a broadcast transmitter device, etc., and may not use or use any type of operating system. In some cases, source device 12 and destination device 14 may be used for wireless communication. Therefore, source device 12 and destination device 14 may be wireless communication devices.
[0125] In some cases, Figure 1A The video encoding and decoding system 10 shown is only an example, and the technology in the present application can be applied to video encoding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between encoding and decoding devices. In other examples, data can be retrieved from local storage, streamed over a network, etc. A video encoding device can encode data and store the data to a memory, and / or a video decoding device can retrieve data from a memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to a memory and / or retrieve data from a memory and decode data.
[0126] For ease of description, this document (for example) refers to the High-Efficiency Video Coding (HEVC) or the next-generation video coding standard Versatile Video Coding (VVC) reference software developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG) to describe the embodiments of the present invention. Those skilled in the art understand that the embodiments of the present invention are not limited to HEVC or VVC.
[0127] Encoders and encoding methods
[0128] Figure 2 FIG. 2 is a schematic block diagram of an exemplary video encoder 20 for implementing the techniques in the present application. Figure 2 In the example of , the video encoder 20 includes an input terminal 201 (or an input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output terminal 272 (or an output interface 272). The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a partitioning unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The illustrated video encoder 20 may also be referred to as a hybrid video encoder or a hybrid video codec-based video encoder.
[0129] The residual calculation unit 204, the transform processing unit 206, the quantization unit 208 and the mode selection unit 260 may constitute a forward signal path of the encoder 20, and the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 244 and the intra-frame prediction unit 254 may constitute a backward signal path of the video encoder 20, wherein the backward signal path of the video encoder 20 corresponds to the decoder (see Figure 3The inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 244 and the intra-frame prediction unit 254 also constitute the “built-in decoder” of the video encoder 20.
[0130] Images and image partitioning (images and blocks)
[0131] The encoder 20 may be configured to receive an image 17 (or image data 17) via an input 201 or the like. The image 17 may be an image in a series of images constituting a video or a video sequence. The received image or image data may also be a pre-processed image 19 (or pre-processed image data 19). For simplicity, the following description uses image 17. Image 17 may also be referred to as a current image or an image to be decoded (particularly in video decoding in order to distinguish the current image from other images (e.g., previously encoded and / or decoded images) in the same video sequence (i.e., a video sequence that also includes the current image)).
[0132] A (digital) image is or can be considered as a two-dimensional array or matrix consisting of samples with intensity values. The samples in the array can also be called pixels (pixel / pel) (short for picture elements). The number of samples in the horizontal and vertical directions (or axes) of the array or image defines the size and / or resolution of the image. In order to represent color, three color components are usually used, that is, the image can be represented as or can include three sample arrays. In the RGB format or color space, an image includes corresponding red, green and blue sample arrays. However, in video decoding, each pixel is usually represented in a brightness and chrominance format or color space, such as YCbCr, including a brightness component represented by Y (sometimes also represented by L) and two chrominance components represented by Cb and Cr. The brightness (luminance, abbreviated as luma) component Y represents the brightness or grayscale intensity (for example, the two are the same in grayscale images), while the two chrominance (chroma, abbreviated as chroma) components Cb and Cr represent the chrominance or color information components. Thus, an image in YCbCr format includes a luma sample array consisting of luma sample values (Y) and two chroma sample arrays consisting of chroma values (Cb and Cr). An image in RGB format can be converted or transformed into YCbCr format and vice versa. This process is also called color conversion or conversion. If the image is black and white, the image may include only a luma sample array. Accordingly, for example, the image may be a luma sample array in black and white format or a luma sample array and two corresponding chroma sample arrays in 4:2:0, 4:2:2 and 4:4:4 color formats.
[0133] An embodiment of the video encoder 20 may include an image partitioning unit ( Figure 2 ), for dividing the image 17 into a plurality of (usually non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264 / AVC), or (in H.265 / HEVC and VVC) coding tree blocks (CTB) or coding tree units (CTU). The image partitioning unit may be used to use the same block size for all images in a video sequence and a corresponding grid of defined block sizes, or to vary the block size between images or subsets or groups of images, and to divide each image into a plurality of corresponding blocks.
[0134] In other embodiments, the video encoder may be configured to directly receive the block 203 in the image 17, for example, one, several or all blocks constituting the image 17. The image block 203 may also be referred to as a current image block or an image block to be decoded.
[0135] Similar to the image 17, the image block 203 is also or can be regarded as a two-dimensional array or matrix composed of pixel points with intensity values (sample values), but the size of the image block 203 is smaller than that of the image 17. In other words, according to the color format applied, the block 203 may include (for example) one sample array (such as a brightness array in the case of the black and white image 17 or a brightness array or a chrominance array in the case of a color image) or three sample arrays (such as one brightness array and two chrominance arrays in the case of the color image 17) or any other number and / or type of arrays. The number of samples in the horizontal direction and the vertical direction (or axis) of the block 203 defines the size of the block 203. Accordingly, a block may be an M×N (M columns×N rows) sample array, or an M×N transform coefficient array, etc.
[0136] Figure 2 The illustrated embodiment of the video encoder 20 may be used to encode the image 17 block by block, for example performing encoding and prediction for each block 203 .
[0137] Figure 2 The illustrated embodiment of the video encoder 20 may also be used to divide and / or encode a picture using slices (also referred to as video slices). A picture may be divided into or encoded using one or more (usually non-overlapping) slices, each of which may include one or more blocks (e.g., CTUs).
[0138] Figure 2The illustrated embodiment of the video encoder 20 may also be used to partition and / or encode an image using tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles). An image may be partitioned into or encoded using one or more (usually non-overlapping) tile groups; each tile group may include one or more blocks (e.g., CTUs) or one or more tiles, etc.; each tile may be rectangular, etc. and may include one or more complete or partial blocks (e.g., CTUs), etc.
[0139] Residual calculation
[0140] The residual calculation unit 204 can be used to calculate the residual block 205 (also referred to as residual 205) based on the image block 203 and the prediction block 265 (the prediction block 265 is described in detail later) in the following manner to obtain the residual block 205 in the sample domain: for example, subtracting the sample value of the prediction block 265 from the sample value of the image block 203 sample by sample (pixel by pixel).
[0141] Transform
[0142] The transform processing unit 206 may be used to apply a transform such as discrete cosine transform (DCT) or discrete sine transform (DST) on the sample values of the residual block 205 to obtain a transform coefficient 207 in the transform domain. The transform coefficient 207 may also be referred to as a transform residual coefficient, representing the residual block 205 in the transform domain.
[0143] The transform processing unit 206 can be used to apply an integer approximation of a DCT / DST (e.g., a transform specified for H.265 / HEVC). Compared to an orthogonal DCT transform, this integer approximation is typically scaled by a factor. In order to maintain the norm of the residual block processed by the forward transform and the inverse transform, other scaling factors are applied as part of the transform process. The scaling factor is typically selected based on certain constraints, such as the scaling factor being a power of 2 for the shift operation, the bit depth of the transform coefficients, a trade-off between precision and implementation cost, etc. For example, a specific scaling factor is specified for the inverse transform by the inverse transform processing unit 212, etc. (and on the video decoder 30 side, the corresponding inverse transform by the inverse transform processing unit 312, etc.); accordingly, a corresponding scaling factor can be specified for the forward transform by the transform processing unit 206, etc. on the encoder 20 side.
[0144] Embodiments of the video encoder 20 (and accordingly, the transform processing unit 206) may be configured to output transform parameters (e.g., one or more transform types) directly or after encoding or compression through the entropy coding unit 270, such that the video decoder 30 may receive and use the transform parameters for decoding, and so on.
[0145] Quantification
[0146] The quantization unit 208 may be configured to quantize the transform coefficients 207 by applying scalar quantization or vector quantization, etc., to obtain quantized coefficients 209 . The quantized coefficients 209 may also be referred to as quantized transform coefficients 209 or quantized residual coefficients 209 .
[0147] The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different scaling may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to a finer quantization, while a larger quantization step size corresponds to a coarser quantization. A suitable quantization step size may be indicated by a quantization parameter (QP). For example, a quantization parameter may be an index to a predefined set of suitable quantization step sizes. For example, a smaller quantization parameter may correspond to fine quantization (a smaller quantization step size), a larger quantization parameter may correspond to coarse quantization (a larger quantization step size), and vice versa. Quantization may include dividing by a quantization step size and corresponding inverse quantization or dequantization, for example, performed by inverse quantization 210, or may include multiplying by a quantization step size. According to embodiments of some standards such as HEVC, a quantization parameter may be used to determine a quantization step size. In general, a quantization step size may be calculated based on a quantization parameter using a fixed-point approximation of an equation including division. Additional scaling factors may be introduced for quantization and dequantization to recover the norm of the residual block that may be modified due to the scaling used in the fixed-point approximation of the equations for the quantization step size and the quantization parameter. In one exemplary implementation, the scaling of the inverse transform and dequantization may be combined. Optionally, a custom quantization table may be used and signaled from the encoder to the decoder in a bitstream or the like. Quantization is a lossy operation, where the larger the quantization step size, the greater the loss.
[0148] Embodiments of the video encoder 20 (correspondingly, the quantization unit 208 ) may be configured to directly output a quantization parameter (QP) or output the quantization parameter after encoding through the entropy coding unit 270 , so that the video decoder 30 may receive and use the quantization parameter for decoding, and so on.
[0149] Dequantization
[0150] The inverse quantization unit 210 is used to apply the inverse quantization of the quantization unit 208 on the quantized coefficients to obtain dequantized coefficients 211, for example, according to or using the same quantization step size as the quantization unit 208, and applying an inverse quantization scheme of the quantization scheme applied by the quantization unit 208. The dequantized coefficients 211 may also be referred to as inverse quantized residual coefficients 211, which correspond to the transform coefficients 207, but are generally different from the transform coefficients due to losses caused by quantization.
[0151] Inverse Transform
[0152] The inverse transform processing unit 212 is used to apply the inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST) or other inverse transforms, to obtain a reconstructed residual block 213 (or a corresponding dequantized coefficient 213) in the sample domain. The reconstructed residual block 213 may also be referred to as a transform block 213.
[0153] reconstruction
[0154] The reconstruction unit 214 (e.g., an adder or summer 214) is used to add the transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain the reconstructed block 215 in the sample domain by, for example, adding the sample values of the reconstructed residual block 213 and the sample values of the prediction block 265 sample by sample.
[0155] Filtering
[0156] The loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstructed block 215 to obtain a filtered block 221, or is generally used to filter the reconstructed samples to obtain filtered sample values. For example, the loop filter unit is used to smoothly perform pixel conversion or otherwise improve video quality. The loop filter unit 220 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a collaborative filter, or any combination thereof. Although the loop filter unit 220 is Figure 2 2 is shown as an in-loop filter, but in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 may also be referred to as a filtered reconstructed block 221.
[0157] Embodiments of the video encoder 20 (and, accordingly, the loop filter unit 220) may be configured to output loop filter parameters (e.g., sample adaptive offset information), for example directly or after being encoded by the entropy coding unit 270, so that the decoder 30 may receive and apply the same loop filter parameters or the corresponding loop filter for decoding.
[0158] Decode image buffer
[0159] The decoded picture buffer (DPB) 230 may be a memory that stores reference images or generally stores reference image data for use by the video encoder 20 when encoding video data. The DPB 230 may be formed by any of a variety of memory devices, such as a dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM) or other types of memory devices. The decoded picture buffer (DPB) 230 may be used to store one or more filtered blocks 221. The decoded picture buffer 230 may also be used to store other previously filtered blocks (e.g., previously filtered reconstructed blocks 221) in the same current image or a different image (e.g., a previously reconstructed image), and may provide a previously complete reconstructed (i.e., decoded) image (and corresponding reference blocks and samples) and / or a partially reconstructed current image (and corresponding reference blocks and samples) for inter-frame prediction, etc. The decoded picture buffer (DPB) 230 may also be used to store one or more unfiltered reconstructed blocks 215, or generally to store unfiltered reconstructed samples, if the reconstructed block 215 is not filtered by the loop filter unit 220; and to store reconstructed blocks or reconstructed samples without any further processing.
[0160] Mode selection (partitioning and prediction)
[0161] The mode selection unit 260 includes a partitioning unit 262, an inter-frame prediction unit 244, and an intra-frame prediction unit 254, and is used to receive or obtain original image data such as an original block 203 (current block 203 of the current image 17) and reconstructed image data (e.g., filtered and / or unfiltered reconstructed samples or reconstructed blocks of the same (current) image and / or one or more previous decoded images) from the decoded image buffer 230 or other buffers (e.g., a row buffer, not shown in the figure). The reconstructed image data is used as reference image data required for prediction such as inter-frame prediction or intra-frame prediction to obtain a prediction block 265 or a prediction value 265.
[0162] The mode selection unit 260 can be used to determine or select a partitioning method for the current block prediction mode (including no partitioning) and determine or select a prediction mode (such as an intra-frame prediction mode or an inter-frame prediction mode), generate a corresponding prediction block 265, calculate the residual block 205 and reconstruct the reconstructed block 215.
[0163] Embodiments of the mode selection unit 260 may be used to select a partitioning method and a prediction mode (e.g., selected from those modes supported or available by the mode selection unit 260). The prediction mode provides the best match or minimum residual (minimum residual means better compression in transmission or storage), or provides minimum signaling overhead (minimum signaling overhead means better compression in transmission or storage), or considers or balances both of the above. The mode selection unit 260 may be used to determine the partitioning method and the prediction mode according to rate distortion optimization (RDO), that is, to select the prediction mode that provides the minimum rate distortion. The terms "best", "minimum", "optimal", etc. in this document do not necessarily mean "best", "minimum", "optimal", etc. in general, but may also refer to situations where termination or selection criteria are met, for example, values exceeding or below a threshold or other constraints may result in a "suboptimal selection", but will reduce complexity and reduce processing time.
[0164] In other words, the partitioning unit 262 can be used to divide the block 203 into smaller block partitions or sub-blocks (forming blocks again) in the following manner: for example, by iteratively using quad-tree (quad-tree, QT) partitioning, binary-tree (binary-tree, BT) partitioning or triple-tree (triple-tree, TT) partitioning or any combination thereof, and for performing prediction on each of the block partitions or sub-blocks, etc., wherein the mode selection includes selecting a tree structure for partitioning the block 203, and using a prediction mode for each of the block partitions or sub-blocks.
[0165] The partitioning (eg, performed by partition unit 260) and prediction processes (performed by inter-prediction unit 244 and intra-prediction unit 254) performed by exemplary video encoder 20 are described in detail below.
[0166] Division
[0167] The partitioning unit 262 can divide (or split) the current block 203 into smaller partitions, such as smaller blocks of square or rectangular size. These smaller blocks (also referred to as sub-blocks) can be further divided into even smaller partitions. This is also referred to as tree partitioning or hierarchical tree partitioning. The root block at root tree level 0 (hierarchical level 0, depth 0) etc. can be recursively divided into blocks of two or more next lower tree levels, such as nodes of tree level 1 (hierarchical level 1, depth 1). These blocks can be further divided into blocks of two or more next lower levels, such as tree level 2 (hierarchical level 2, depth 2), etc., until the partitioning ends (because the end criteria are met, such as reaching the maximum tree depth or minimum block size). Blocks that are not further divided are also referred to as leaf blocks or leaf nodes of the tree. A tree divided into two partitions is called a binary tree (binary-tree, BT), a tree divided into three partitions is called a ternary tree (ternary-tree, TT), and a tree divided into four partitions is called a quadtree (quad-tree, QT).
[0168] As described above, the term "block" used herein may be a portion of an image, in particular a square or rectangular portion. With reference to HEVC and VVC, etc., a block may be or may correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU), and / or to a plurality of corresponding blocks, such as a coding tree block (CTB), a coding block (CB), a transform block (TB), or a prediction block (PB).
[0169] For example, a coding tree unit (CTU) may be or may include a CTB consisting of luma samples and two corresponding CTBs consisting of chroma samples in an image with 3 sample arrays, or may be or may include a CTB consisting of samples in a black and white image or an image decoded using 3 separate color planes and syntax structures. These syntax structures are used to decode the samples. Accordingly, a coding tree block (CTB) may be an N×N sample block, where N may be set to a certain value so that a component is divided into multiple CTBs, which is one way of partitioning. A coding unit (CU) may be or may include a coding block consisting of luma samples and two corresponding coding blocks consisting of chroma samples in an image with 3 sample arrays, or a coding block consisting of samples in a black and white image or an image decoded using 3 separate color planes and syntax structures. These syntax structures are used to decode the samples. Accordingly, a coding block (CB) may be an M×N sample block, where M and N may be set to a certain value so that a CTB is divided into multiple coding blocks, which is one way of partitioning.
[0170] In an embodiment, for example according to HEVC, a coding tree unit (CTU) can be partitioned into multiple CUs using a quadtree structure represented as a coding tree. It is decided at the CU level whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to decode the image area. Each CU can be further partitioned into 1, 2 or 4 PUs according to the PU partition type. The same prediction process is performed within a PU, and relevant information is sent to the decoder in units of PUs. After the residual block is obtained by the prediction process according to the PU partition type, the CU can be divided into transform units (TUs) according to other quadtree structures similar to the coding tree for the CU.
[0171] In an embodiment, for example, according to the latest video coding standard currently under development called Versatile Video Coding (VVC), a quadtree combined with a binary tree (quad-tree and binary-tree, QTBT) partitioning is used to divide the coding block. In the QTBT block structure, a CU can be a square or a rectangle. For example, the coding tree unit (CTU) is first divided by a quadtree structure. The quadtree leaf nodes are further divided by a binary tree or a ternary / triple tree structure. The divided leaf nodes are called coding units (CUs), and this segmentation is used for prediction and transform processing without any further division. This means that in the QTBT coding block structure, the block sizes of CU, PU, and TU are the same. At the same time, multiple partitions such as ternary tree partitioning can be used with the QTBT block structure.
[0172] In one example, mode selection unit 260 in video encoder 20 may be used to perform any combination of the partitioning techniques described herein.
[0173] As described above, the video encoder 20 is used to determine or select the best or optimal prediction mode from a (predetermined) prediction mode set. The prediction mode set may include intra-frame prediction mode and / or inter-frame prediction mode, etc.
[0174] Intra prediction
[0175] The intra-frame prediction mode set may include 35 different intra-frame prediction modes, such as non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in HEVC, or may include 67 different intra-frame prediction modes, such as non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in VVC.
[0176] The intra prediction unit 254 is configured to generate an intra prediction block 265 using reconstructed samples of neighboring blocks of the same current image according to an intra prediction mode in an intra prediction mode set.
[0177] The intra-frame prediction unit 254 (or generally referred to as the mode selection unit 260) is also used to output intra-frame prediction parameters (or generally referred to as information representing the selected intra-frame prediction mode of the block) to the entropy coding unit 270 in the form of syntax elements 266 to be included in the encoded image data 21, so that the video decoder 30 can receive and use the prediction parameters for decoding, etc.
[0178] Inter prediction
[0179] The set of (possible) inter-frame prediction modes depends on the available reference image (i.e. (for example) at least part of the decoded image stored in DPB 230 as mentioned above) and other inter-frame prediction parameters, for example on whether the entire reference image or only a part of the reference image (e.g. a search window area around the area of the current block) is used to search for the best matching reference block, and / or for example on whether pixel interpolation is performed (e.g. half-pixel / half-pixel interpolation and / or quarter-pixel interpolation).
[0180] In addition to the above-mentioned prediction modes, skip mode and / or direct mode may also be used.
[0181] The inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (both in Figure 2 ). The motion estimation unit may be configured to receive or acquire an image block 203 (a current image block 203 of a current image 17) and a decoded image 231, or at least one or more previously reconstructed blocks (e.g., reconstructed blocks of one or more other / different previously decoded images 231) for motion estimation. For example, a video sequence may include a current image and a previously decoded image 231, or in other words, the current image and the previously decoded image 231 may be part of or constitute a series of images, which constitute a video sequence.
[0182] For example, the encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same or different images in a plurality of other images, and provide the reference image (or reference image index) and / or the offset (spatial offset) between the position (x coordinate, y coordinate) of the reference block and the position of the current block as an inter-frame prediction parameter to the motion estimation unit. This offset is also referred to as a motion vector (MV).
[0183] The motion compensation unit is used to obtain (e.g., receive) inter-frame prediction parameters, and perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain an inter-frame prediction block 265. The motion compensation performed by the motion compensation unit may include extracting or generating a prediction block based on a motion / block vector determined by motion estimation, and may also include performing interpolation to obtain sub-pixel accuracy. Interpolation filtering can generate other pixel samples based on known pixel samples, thereby potentially increasing the number of candidate prediction blocks that can be used to decode the image block. Once the motion vector corresponding to the PU of the current image block is received, the motion compensation unit can locate the prediction block pointed to by the motion vector in one of the reference image lists.
[0184] The motion compensation unit may also generate syntax elements associated with blocks and video slices for use by the video decoder 30 when decoding image blocks of the video slices. In addition to or as an alternative to slices and corresponding syntax elements, partition groups and / or partitions and corresponding syntax elements may also be generated or used.
[0185] Entropy Coding
[0186] The entropy coding unit 270 is used to apply or not apply an entropy coding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, binarization, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding or other entropy coding methods or techniques) to (uncompressed) quantized coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters and / or other syntax elements to obtain encoded image data 21 that can be output through an output terminal 272 in the form of an encoded bitstream 21, etc., so that the video decoder 30 can receive and use these parameters for decoding, etc. The encoded bitstream 21 can be sent to the video decoder 30, or stored in a memory to be sent or retrieved by the video decoder 30 later.
[0187] Other structural variations of the video encoder 20 may be used to encode the video stream. For example, a non-transform based encoder 20 may directly quantize the residual signal for certain blocks or frames without the transform processing unit 206. In another implementation, the encoder 20 may include a quantization unit 208 and an inverse quantization unit 210 combined into a single unit.
[0188] Decoder and decoding method
[0189] Figure 33 is an example of a video decoder 30 for implementing the technology in the present application. The video decoder 30 is used to receive the encoded image data 21 (e.g., the encoded code stream 21) encoded by the encoder 20 (e.g., to obtain a decoded image 331. The encoded image data or code stream includes information for decoding the encoded image data, such as data representing image blocks of encoded video slices (and / or partition groups or partitions) and related syntax elements.
[0190] exist Figure 3 In the example of FIG. 3 , the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a loop filter 320, a decoded picture buffer (DPB) 330, a mode application unit 360, an inter-frame prediction unit 344, and an intra-frame prediction unit 354. The inter-frame prediction unit 344 may be or may include a motion compensation unit. In some examples, the video decoder 30 may perform substantially the same as the reference Figure 2 The encoding process described in the video encoder 100 is the inverse of the decoding process.
[0191] As described with reference to encoder 20, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter prediction unit 344, and intra prediction unit 354 also constitute a "built-in decoder" of video encoder 20. Accordingly, inverse quantization unit 310 may be functionally identical to inverse quantization unit 110, inverse transform processing unit 312 may be functionally identical to inverse transform processing unit 212, reconstruction unit 314 may be functionally identical to reconstruction unit 214, loop filter 320 may be functionally identical to loop filter 220, and decoded picture buffer 330 may be functionally identical to decoded picture buffer 230. Therefore, the explanation of the corresponding units and functions of video encoder 20 applies correspondingly to the corresponding units and functions of video decoder 30.
[0192] Entropy decoding
[0193] The entropy decoding unit 304 is used to parse the bitstream 21 (or generally referred to as the encoded image data 21) and perform entropy decoding on the encoded image data 21 to obtain the quantized coefficients 309 and / or the decoded encoding parameters ( Figure 3The entropy decoding unit 304 may be used to apply a decoding algorithm or scheme corresponding to the coding scheme described with reference to the entropy coding unit 270 in the encoder 20. The entropy decoding unit 304 may also be used to provide the inter-frame prediction parameters, intra-frame prediction parameters and / or other syntax elements to the mode application unit 360, and to provide other parameters to other units in the decoder 30. The video decoder 30 may receive syntax elements at the video slice level and / or the video block level. In addition to or as an alternative to the slices and the corresponding syntax elements, block groups and / or blocks and corresponding syntax elements may also be received and / or used.
[0194] Dequantization
[0195] The inverse quantization unit 310 may be configured to receive a quantization parameter (QP) (or generally referred to as information related to inverse quantization) and quantized coefficients from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.), and inverse quantize the decoded quantized coefficients 309 according to the quantization parameters to obtain dequantized coefficients 311. The dequantized coefficients 311 may also be referred to as transform coefficients 311. The inverse quantization process may include determining a degree of quantization using a quantization parameter determined by the video encoder 20 for each video block in a video slice (or partition or group of partitions), and also determining a degree of inverse quantization that needs to be applied.
[0196] Inverse Transform
[0197] The inverse transform processing unit 312 may be configured to receive the dequantized coefficients 311 (also referred to as transform coefficients 311) and transform the dequantized coefficients 311 to obtain a reconstructed residual block 213 in the sample domain. The reconstructed residual block 213 may also be referred to as a transform block 313. The transform may be an inverse transform, such as an inverse DCT, an inverse DST, an inverse integer transform, or a conceptually similar inverse transform process. The inverse transform processing unit 312 may also be configured to receive transform parameters or corresponding information from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.) to determine the transform to be performed on the dequantized coefficients 311.
[0198] reconstruction
[0199] The reconstruction unit 314 (eg, adder or summer 314 ) may be configured to add the reconstructed residual block 313 to the prediction block 365 to obtain the reconstructed block 315 in the sample domain by, for example, adding sample values of the reconstructed residual block 313 and sample values of the prediction block 365 .
[0200] Filtering
[0201] The loop filter unit 320 (in or after the decoding loop) is used to filter the reconstructed block 315 to obtain the filtered block 321, so as to smoothly perform pixel conversion or otherwise improve video quality. The loop filter unit 320 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a collaborative filter, or any combination thereof. Although the loop filter unit 320 is in Figure 3 3. Although shown as an in-loop filter in FIG. 3, in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.
[0202] Decode image buffer
[0203] The decoded video block 321 of one picture is then stored in a decoded picture buffer 330, which stores the decoded picture 331 as a reference picture for subsequent motion compensation and / or output or display of other pictures.
[0204] The decoder 30 is used to output the decoded image 311 through an output terminal 312, etc., for display to a user or for viewing by the user.
[0205] predict
[0206] The inter-frame prediction unit 344 may be functionally identical to the inter-frame prediction unit 244 (particularly the motion compensation unit), and the intra-frame prediction unit 354 may be functionally identical to the intra-frame prediction unit 254, and may perform segmentation or partitioning decisions and predictions based on the partitioning mode and / or prediction parameters or corresponding information received from the coded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.). The mode application unit 360 may be configured to perform prediction (intra-frame prediction or inter-frame prediction) on each block based on the reconstructed image, block, or corresponding sample (filtered or unfiltered), and obtain a prediction block 365.
[0207] When the video slice is decoded as an intra-coded (I) slice, the intra-prediction unit 354 in the mode application unit 360 is used to generate a prediction block 365 for the image block of the current video slice based on the signaled intra-prediction mode and data from the previously decoded block of the current image. When the video image is decoded as an inter-coded (i.e., B or P) slice, the inter-prediction unit 344 (e.g., motion compensation unit) in the mode application unit 360 is used to generate the prediction block 365 for the video block of the current video slice based on the motion vector and other syntax elements received from the entropy decoding unit 304. For inter-prediction, these prediction blocks can be generated based on one of the reference pictures in one of the reference picture lists. The video decoder 30 can construct the reference frame list 0 and list 1 using a default construction technique based on the reference pictures stored in the DPB 330. The same or similar processes may be applied to or by embodiments using partition groups (e.g., video partition groups) and / or partitions (e.g., video partitions) in addition to or as an alternative to slices (e.g., video slices), e.g., video may be decoded using I, P, or B partition groups and / or partitions.
[0208] The mode application unit 360 is used to determine prediction information for the video blocks of the current video slice by parsing motion vectors or related information and other syntax elements, and use the prediction information to generate a prediction block for the current video block being decoded. For example, the mode application unit 360 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame or inter-frame prediction) for decoding the video blocks of the video slice, the inter-frame prediction slice type (e.g., B slice, P slice, or GPB slice), the construction information of one or more reference picture lists for the slice, the motion vector for each inter-frame encoded video block of the slice, the inter-frame prediction state for each inter-frame decoded video block of the slice, and other information to decode the video blocks in the current video slice. In addition to slices (e.g., video slices) or as an alternative to slices, the same or similar process can be applied to or by embodiments using block groups (e.g., video block groups) and / or blocks (e.g., video blocks), for example, the video can be decoded using I, P, or B block groups and / or blocks.
[0209] Figure 3 The embodiment of the video decoder 30 shown in FIG. 1 may be used to divide and / or decode a picture using slices (also referred to as video slices). A picture may be divided into or decoded using one or more (usually non-overlapping) slices, each of which may include one or more blocks (e.g., CTUs).
[0210] In an embodiment, Figure 3The video decoder 30 shown may be used to partition and / or decode an image using partition groups (also referred to as video partition groups) and / or partitions (also referred to as video partitions). An image may be partitioned into or decoded using one or more (usually non-overlapping) partition groups; each partition group may include one or more blocks (e.g., CTUs) or one or more partitions, etc.; each partition may be rectangular, etc. and may include one or more complete or partial blocks (e.g., CTUs), etc.
[0211] Other variations of the video decoder 30 may be used to decode the encoded image data 21. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, a non-transform based decoder 30 may directly dequantize the residual signal for certain blocks or frames without the inverse transform processing unit 312. In another implementation, the video decoder 30 may include the dequantization unit 310 and the inverse transform processing unit 312 combined into a single unit.
[0212] It should be understood that, in the encoder 20 and the decoder 30, the processing result of the current step may be further processed and then output to the next step. For example, after interpolation filtering, motion vector derivation or loop filtering, the processing result of interpolation filtering, motion vector derivation or loop filtering may be further operated, such as clipping or shifting operation.
[0213] It should be noted that the derived motion vector of the current block (including but not limited to the control point motion vector of the affine mode, the sub-block motion vector of the affine mode, the plane mode, the ATMVP mode, the temporal motion vector, etc.) can be further operated. For example, the value of the motion vector is limited to a predefined range according to the representation bit of the motion vector. If the representation bit of the motion vector is bitDepth, the range is –2^(bitDepth–1) to 2^(bitDepth–1)–1, where “^” represents a power. For example, if bitDepth is set to 16, the range is –32768 to 32767; if bitDepth is set to 18, the range is –131072 to 131071. For example, the value of the derived motion vector (for example, the MV of 4 4×4 sub-blocks in an 8×8 block) is limited so that the maximum difference between the integer parts of the MVs of the 4 4×4 sub-blocks does not exceed N pixels, for example, does not exceed 1 pixel. Two methods for limiting motion vectors according to bitDepth are provided here.
[0214] Method 1: Remove the most significant bit (MSB) of the overflow by smoothing
[0215] ux= (mvx+2 bitDepth ) % 2 bitDepth (1)
[0216] mvx = (ux >= 2 bitDepth–1 ) ? (ux – 2 bitDepth ) : ux (2)
[0217] uy= (mvy+2 bitDepth ) % 2 bitDepth (3)
[0218] mvy = (uy >= 2 bitDepth–1 ) ? (uy – 2 bitDepth ) : uy (4)
[0219] Among them, mvx is the horizontal component of the motion vector of an image block or sub-block; mvy is the vertical component of the motion vector of an image block or sub-block; ux and uy represent intermediate values.
[0220] For example, if the value of mvx is –32769, the value obtained by using formulas (1) and (2) is 32767. In computer systems, decimal numbers are stored in the form of two's complement. The two's complement of –32769 is 1,0111,1111,1111,1111 (17 bits), and if the MSB is discarded, the two's complement obtained is 0111,1111,1111,1111 (the decimal number is 32767), which is the same as the output result obtained by using formulas (1) and (2).
[0221] ux= (mvpx + mvdx +2 bitDepth ) % 2 bitDepth (5)
[0222] mvx = (ux >= 2 bitDepth–1 ) ? (ux – 2 bitDepth ) : ux (6)
[0223] uy= (mvpy + mvdy +2 bitDepth ) % 2 bitDepth (7)
[0224] mvy = (uy >= 2 bitDepth–1 ) ? (uy – 2 bitDepth ) : uy (8)
[0225] These operations can be performed in the process of summing mvp and mvd, as shown in equations (5) to (8).
[0226] Method 2: Clip the value to remove the overflowed MSB
[0227] vx=Clip3(–2 bitDepth–1 ,2 bitDepth–1 –1,vx)
[0228] vy=Clip3(–2 bitDepth–1 ,2 bitDepth–1 –1,vy)
[0229] Where vx is the horizontal component of the motion vector of an image block or sub-block, vy is the vertical component of the motion vector of an image block or sub-block; x, y and z correspond to the three input values of the MV clipping process respectively. The function Clip3 is defined as follows:
[0230]
[0231] Figure 4 Schematic diagram of a video decoding device 400 provided in accordance with an embodiment of the present invention. The video decoding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video decoding device 400 may be a decoder (e.g. Figure 1A 30) or an encoder (e.g. Figure 1A 20 in the video encoder).
[0232] The video decoding device 400 includes: an input port 410 (or input port 410) and a receiving unit (Rx) 420 for receiving data; a processor, a logic unit or a central processing unit (CPU) 430 for processing the data; a transmitting unit (Tx) 440 and an output port 450 (or output port 450) for transmitting the data; and a memory 460 for storing the data. The video decoding device 400 may also include an optical-to-electrical (OE) component and an electrical-to-optical (EO) component coupled to the input port 410, the receiving unit 420, the transmitting unit 440 and the output port 450, which are used as an outlet or an inlet of an optical signal or an electrical signal.
[0233] The processor 430 is implemented by hardware and software. The processor 430 can be implemented as one or more CPU chips, one or more cores (e.g., multi-core processors), one or more FPGAs, one or more ASICs, and one or more DSPs. The processor 430 communicates with the input port 410, the receiving unit 420, the sending unit 440, the output port 450, and the memory 460. The processor 430 includes a decoding module 470. The decoding module 470 implements the disclosed embodiments described above. For example, the decoding module 470 performs, processes, prepares, or provides various decoding operations. Therefore, the inclusion of the decoding module 470 provides substantial improvements to the functions of the video decoding device 400 and affects the conversion of the video decoding device 400 to different states. Optionally, the decoding module 470 is implemented with instructions stored in the memory 460 and executed by the processor 430.
[0234] The memory 460 may include one or more disks, one or more tape drives, and one or more solid-state hard disks, and may be used as an overflow data storage device to store programs when they are selected for execution and to store instructions and data read during the execution of the programs. For example, the memory 460 may be volatile and / or non-volatile, and may be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random-access memory (SRAM).
[0235] Figure 5 A simplified block diagram of an apparatus 500 is provided for an exemplary embodiment. The apparatus 500 may be used as Figure 1A Either or both of the source device 12 and the destination device 14.
[0236] The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices that are currently available or will be developed in the future and are capable of operating or processing information. Although the disclosed implementations may be implemented using a single processor such as the processor 502 shown in the figure, using multiple processors may improve speed and efficiency.
[0237] In one implementation, the memory 504 in the apparatus 500 may be a read only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 accessed by the processor 502 via a bus 512. The memory 504 may also include an operating system 508 and an application 510, wherein the application 510 includes at least one program that enables the processor 502 to perform the method described herein. For example, the application 510 may include applications 1 to N, and may also include a video decoding application that performs the method described herein.
[0238] The apparatus 500 may also include one or more output devices, such as a display 518. In one example, the display 518 may be a touch-sensitive display that combines a display with a touch-sensitive element that can be used to sense touch input. The display 518 may be coupled to the processor 502 via the bus 512.
[0239] Although the bus 512 in the device 500 is described herein as a single bus, the bus 512 may include multiple buses. In addition, the auxiliary memory 514 may be directly coupled to other components in the device 500 or may be accessed through a network, and may include a single integrated unit (e.g., one memory card) or multiple units (e.g., multiple memory cards). Therefore, the device 500 may be implemented in a variety of configurations.
[0240] Triangular partitioning mode (TPM) and geometric motion partitioning (GEO), also known as triangular fusion mode and geometric fusion mode, respectively, are partitioning techniques that implement non-horizontal and non-vertical boundaries between prediction partitions, where prediction units PU1 and prediction units PU2 are combined in a region using a weighted averaging process of their sample subsets associated with different color components. TPM enables the boundaries between prediction partitions to be along the rectangular block diagonal, while the boundaries according to GEO can be located at arbitrary positions. In the region where the weighted averaging process is applied, the integer number in the square represents the weight W applied to the brightness component of the prediction unit PU1. PU1 In one example, the weight W applied to the luma component of prediction unit PU2 is PU2 The calculation is as follows:
[0241] W PU2 =8–W PU1 .
[0242] The weight applied to the chroma components of the corresponding prediction unit may be different from the weight applied to the luma component of the corresponding prediction unit.
[0243] The detailed information about TPM syntax is shown in Table 1, where four syntax elements are used to indicate information about TPM:
[0244] MergeTriangleFlag is a flag to determine whether to select TPM ("0" means not to select TPM; otherwise, select TPM);
[0245] merge_triangle_split_dir is the split direction flag of the TPM ("0" means the split direction is from the upper left corner to the lower right corner; otherwise, the split direction is from the upper right corner to the lower left corner);
[0246] merge_triangle_idx0 and merge_triangle_idx1 are the indices of merge candidates 0 and 1 for the TPM.
[0247] Table 1: Fusion data syntax including TPM syntax
[0248]
[0249]
[0250] In one example, TPM is detailed in the following proposal: "CE10.3.1.b: Triangular prediction unit mode" by RL.Liao and C.S.Lim, published in the 12th JVET meeting in Macau, China, October 2018, JVET-L0124. GEO is explained in the following paper: S.Esenlik, H.Gao, A.Filippov, V.Rufitskiy, A.M.Kotra, B.Wang, E.Alshina, M. and J. Sauer, “Non-CE4: Geometrical partitioning for inter blocks,” JVET-O0489, 15th JVET conference, Gothenburg, Sweden, July 2019.
[0251] The disclosed method for coordinating TPM and / or GEO with WP is to disable TPM and / or GEO when applying WP. The first implementation is shown in Table 2, which checks whether the value of the variable weightedPredFlag of the coding unit is equal to 0.
[0252] The variable weightedPredFlag is derived as follows:
[0253] – If slice_type is P, weightedPredFlag is set to pps_weighted_pred_flag.
[0254] – Otherwise (slice_type is B), weightedPredFlag is set to pps_weighted_bipred_flag.
[0255] The weighted prediction process can be switched between picture level and slice level using the syntax elements pps_weighted_pred_flag and sps_weighted_pred_flag.
[0256] As described above, the variable weightedPredFlag indicates whether slice-level weighted prediction can be used when obtaining inter-frame prediction samples for the slice.
[0257] Table 2: Syntax of fusion data to coordinate TPM with WP
[0258]
[0259]
[0260] ciip_flag[x0][y0] indicates whether combined inter-frame blending and intra-frame prediction are applied to the current coding unit. The array indices x0 and y0 indicate the position (x0, y0) of the top left luma sample of the current coding block relative to the top left luma sample of the image.
[0261] When ciip_flag[x0][y0] does not exist, ciip_flag[x0][y0] is inferred as follows:
[0262] – ciip_flag[x0][y0] is inferred to be 1 if all of the following are true:
[0263] –sps_ciip_enabled_flag is equal to 1,
[0264] –general_merge_flag[x0][y0] is equal to 1,
[0265] –merge_subblock_flag[x0][y0] is equal to 0,
[0266] –regular_merge_flag[x0][y0] is equal to 0,
[0267] –cbWidth is less than 128,
[0268] –cbHeight is less than 128,
[0269] –cbWidth*cbHeight is greater than or equal to 64;
[0270] – Otherwise, ciip_flag[x0][y0] is inferred to be 0.
[0271] When ciip_flag[x0][y0] is equal to 1, the variable IntraPredModeY[x][y] (where x=x0..x0+cbWidth−1, y=y0..y0+cbHeight−1) is set to INTRA_PLANAR.
[0272] The variable MergeTriangleFlag[x0][y0] indicates whether triangle-based motion compensation is used to generate prediction samples for the current coding unit when decoding the B slice. The derivation of this variable is as follows:
[0273] –MergeTriangleFlag[x0][y0] is set to 1 if all of the following conditions are true:
[0274] –sps_triangle_enabled_flag is equal to 1,
[0275] –slice_type is B,
[0276] –general_merge_flag[x0][y0] is equal to 1,
[0277] –MaxNumTriangleMergeCand is greater than or equal to 2,
[0278] –cbWidth*cbHeight is greater than or equal to 64,
[0279] –regular_merge_flag[x0][y0] is equal to 0,
[0280] –merge_subblock_flag[x0][y0] is equal to 0,
[0281] –ciip_flag[x0][y0] is equal to 0,
[0282] –weightedPredFlag is equal to 0;
[0283] – Otherwise, MergeTriangleFlag[x0][y0] is set to 0.
[0284] The second implementation is shown in Table 3. If weightedPredFlag is equal to 1, the syntax element max_num_merge_cand_minus_max_num_triangle_cand does not exist and is inferred to have a value such that MaxNumTriangleMergeCand is less than 2.
[0285] Table 3: Generic Stripe Header Syntax for Coordinating TPM with WP
[0286]
[0287]
[0288]
[0289]
[0290]
[0291]
[0292] Specifically, the following semantics can be used for the second implementation:
[0293] max_num_merge_cand_minus_max_num_triangle_cand indicates the maximum number of triangle merging mode candidates supported in a stripe minus MaxNumMergeCand.
[0294] When max_num_merge_cand_minus_max_num_triangle_cand is not present, sps_triangle_enabled_flag is equal to 1, slice_type is B, weightedPredFlag is equal to 0, and MaxNumMergeCand is greater than or equal to 2, max_num_merge_cand_minus_max_num_triangle_cand is inferred to be pps_max_num_merge_cand_minus_max_num_triangle_cand_minus1+1.
[0295] When max_num_merge_cand_minus_max_num_triangle_cand is not present, sps_triangle_enabled_flag is equal to 1, slice_type is B, weightedPredFlag is equal to 1, and MaxNumMergeCand is greater than or equal to 2, max_num_merge_cand_minus_max_num_triangle_cand is inferred to be MaxNumMergeCand or xNumMergeCand–1.
[0296] The maximum number of triangle fusion mode candidates MaxNumTriangleMergeCand is derived as follows:
[0297] MaxNumTriangleMergeCand=MaxNumMergeCand–
[0298] max_num_merge_cand_minus_max_num_triangle_cand
[0299] When max_num_merge_cand_minus_max_num_triangle_cand exists, the value range of MaxNumTriangleMergeCand should be 2 to MaxNumMergeCand (including the end values).
[0300] When max_num_merge_cand_minus_max_num_triangle_cand does not exist (sps_triangle_enabled_flag is equal to 0 or MaxNumMergeCand is less than 2), MaxNumTriangleMergeCand is set to 0.
[0301] When MaxNumTriangleMergeCand is equal to 0, triangle merging mode is not allowed for the current strip.
[0302] The mechanism disclosed in the present invention is not only applicable to TPM and GEO, but also to other non-rectangular prediction and partitioning modes, such as combined intra-frame and inter-frame prediction using triangular partitions.
[0303] Since TPM and GEO are only applied to B slices, the variable weightedPredFlag in the above embodiment can be directly replaced by the variable pps_weighted_bipred_flag.
[0304] The third implementation is shown in Table 6, which checks whether the value of the variable weightedPredFlag of the coding unit is equal to 0.
[0305] The variable weightedPredFlag is derived as follows:
[0306] – weightedPredFlag is set to 0 if all of the following are true:
[0307] luma_weight_l0_flag[i] is equal to 0, where i ranges from 0 to NumRefIdxActive[0],
[0308] luma_weight_l1_flag[i] is equal to 0, where i ranges from 0 to NumRefIdxActive[1],
[0309] chroma_weight_l0_flag[i] is equal to 0, where i ranges from 0 to NumRefIdxActive[0],
[0310] chroma_weight_l0_flag[i] is equal to 0, where i ranges from 0 to NumRefIdxActive[1];
[0311] – Otherwise, weightedPredFlag is set to 1.
[0312] The derivation process of weightedPredFlag indicates that if all weighted flags of the luminance component and the chrominance component and all weighted flags of all reference indices of the current slice are 0, weighted prediction is disabled in the current slice; otherwise, weighted prediction can be used for the current slice.
[0313] As described above, the variable weightedPredFlag indicates whether slice-level weighted prediction can be used when obtaining inter-frame prediction samples for the slice.
[0314] The fourth implementation is shown in Table 2, where weightedPredFlag is replaced by slice_weighted_pred_flag, and slice_weighted_pred_flags is indicated in the slice header, as shown in Table 4.
[0315] As described above, the syntax slice_weighted_pred_flag indicates whether slice-level weighted prediction can be used when obtaining inter-frame prediction samples of a slice.
[0316] Table 4: Generic slice header syntax indicating slice-level weighted prediction flag
[0317]
[0318]
[0319]
[0320]
[0321]
[0322]
[0323]
[0324] Specifically, the following semantics can be used for the fourth implementation:
[0325] slice_weighted_pred_flag equal to 0 indicates that weighted prediction is not applied to the current slice. slice_weighted_pred_flag equal to 1 indicates that weighted prediction is applied to the current slice. When slice_weighted_pred_flag is not present, the value of slice_weighted_pred_flag is inferred to be 0.
[0326] The fifth implementation is to disable block-level TPM through consistency constraints. In the case of a TPM coded block, the inter-frame prediction value P 0 710 and P 1 720 (such as Figure 7 The weighting factors of the luminance component and chrominance component of the corresponding reference image (as shown) may not exist.
[0327] Regarding more details, refIdxA and predListFlagA represent the reference index and reference picture list of the inter-frame prediction value P0; refIdxB and predListFlagB represent the reference index and reference picture list of the inter-frame prediction value P1.
[0328] The variables lumaWeightedFlag and chromaWeightedFlag are derived as follows:
[0329] lumaWeightedFlagA=predListFlagA? luma_weight_l1_flag[refIdxA]:luma_weight_l0_flag[refIdxA]
[0330] lumaWeightedFlagB=predListFlagB? luma_weight_l1_flag[refIdxB]:luma_weight_l0_flag[refIdxB]
[0331] chromaWeightedFlagA=predListFlagA? chroma_weight_l1_flag[refIdxA]:chromaweight_l0_flag[refIdxA]
[0332] chromaWeightedFlagB=predListFlagB? chroma_weight_l1_flag[refIdxB]:chromaweight_l0_flag[refIdxB]
[0333] lumaWeightedFlag=lumaWeightedFlagA||lumaWeightedFlagB
[0334] chromaWeightedFlag=chromaWeightedFlagA||chromaWeightedFlagB
[0335] The requirement for code stream consistency is: lumaWeightedFlag and chromaWeightedFlag can be equal to 0.
[0336] A sixth implementation is to disable the hybrid weighted sample prediction process for a TPM coded block when using explicit weighted prediction.
[0337] Figure 7 and Figure 8 The examples of TPM and GEO are described respectively. It should be noted that the embodiments of TPM can also be used in GEO mode.
[0338] In the case of a TPM coded block, if the inter-frame prediction value P 0 710 or P 1 If the weighting factor of the luminance component or chrominance component of the reference image corresponding to 720 exists, then according to the WP parameter (P 0 and P 1 WP parameters 730{w 0 ,O 0} and WP parameters 740{w 1 ,O 1}) is used to generate the inter-frame prediction value block; otherwise, the weighting process according to the mixed weighting parameters is used to generate the inter-frame prediction value of block 750. Fig. 9 As shown, the inter-frame prediction value 901 requires two prediction blocks P0 911 and P1 912, which have an overlapping area 921, wherein non-zero weights are applied to blocks 911 and 912 to partially mix the prediction values P0 911 and P1 912. Fig. 9 , blocks adjacent to block 901 are represented as 931 , 932 , 933 , 934 , 935 , and 936 . Figure 8 850. Some differences between the TPM fusion mode and the GEO fusion mode are shown. In the case of the GEO fusion mode, the overlapping area between the prediction values 851 and 852 can not only be located on the diagonal line of the inter-frame prediction block 850. The reception of the prediction values P0 851 and P1 852 can be achieved by adding the weights and offsets {w 0 ,O 0}830 and {w 1 ,O 1}840 is applied or not applied to blocks 810 and 820 respectively, and is achieved by copying blocks 810 and 820 from other images.
[0339] In one example, refIdxA and predListFlagA represent the reference index and reference image list of the inter-frame prediction value P0; refIdxB and predListFlagB represent the reference index and reference image list of the inter-frame prediction value P1.
[0340] The variables lumaWeightedFlag and chromaWeightedFlag are derived as follows:
[0341] lumaWeightedFlagA=predListFlagA? luma_weight_l1_flag[refIdxA]:luma_weight_l0_flag[refIdxA]
[0342] lumaWeightedFlagB=predListFlagB? luma_weight_l1_flag[refIdxB]:luma_weight_l0_flag[refIdxB]
[0343] chromaWeightedFlagA=predListFlagA? chroma_weight_l1_flag[refIdxA]:chromaweight_l0_flag[refIdxA]
[0344] chromaWeightedFlagB=predListFlagB? chroma_weight_l1_flag[refIdxB]:chromaweight_l0_flag[refIdxB]
[0345] lumaWeightedFlag=lumaWeightedFlagA||lumaWeightedFlagB
[0346] chromaWeightedFlag=chromaWeightedFlagA||chromaWeightedFlagB
[0347] Then, if lumaWeightedFlag is true, the explicit weighting process is called; if lumaWeightedFlag is false, the hybrid weighting process is called. In addition, the chroma component is also determined by chromaWeightedFlag.
[0348] In an optional implementation, the weighted flags of all components are used uniformly. If one of lumaWeightedFlag or chromaWeightedFlag is true, the explicit weighting process is called; if both lumaWeightedFlag and chromaWeightedFlag are false, the hybrid weighting process is called.
[0349] The explicit weighting process for rectangular blocks predicted using the bi-prediction mechanism is performed as follows.
[0350] Inputs to this process include:
[0351] – Two variables nCbW and nCbH, representing the width and height of the current coding block;
[0352] – two (nCbW) × (nCbH) arrays predSamplesA and predSamplesB;
[0353] – Prediction list flags predListFlagA and predListFlagB;
[0354] – reference indices refIdxA and refIdxB;
[0355] – Variable cIdx, indicating the color component index;
[0356] – Sample bit depth bitDepth.
[0357] The output of this process is a (nCbW)×(nCbH) array pbSamples of prediction sample values.
[0358] The variable shift1 is set to Max(2,14 – bitDepth).
[0359] The variables log2Wd, o0, o1, w0, and w1 are derived as follows:
[0360] – If the luma sample’s cIdx is equal to 0, the following applies:
[0361] log2Wd=luma_log2_weight_denom+shift1
[0362] w0=predListFlagA? LumaWeightL1[refIdxA]:LumaWeightL0[refIdxA]
[0363] w1=predListFlagB? LumaWeightL1[refIdxB]:LumaWeightL0[refIdxB]
[0364] o0=(predListFlagA?luma_offset_l1[refIdxA]:luma_offset_l0[refIdxA])<<(BitDepth Y –8)
[0365] o1=(predListFlagB?luma_offset_l1[refIdxB]:luma_offset_l0[refIdxB])<<(BitDepth Y –8)
[0366] – Otherwise (cIdx of the chroma sample is not equal to 0), the following applies:
[0367] log2Wd=ChromaLog2WeightDenom+shift1
[0368] w0=predListFlagA? ChromaWeightL1[refIdxA][cIdx–1]:ChromaWeightL0[refIdxA][cIdx–1]
[0369] w1=predListFlagA? ChromaWeightL1[refIdxB][cIdx–1]:ChromaWeightL0[refIdxB][cIdx–1]
[0370] o0=(predListFlagA?ChromaOffsetL1[refIdxA][cIdx–1]:ChromaOffsetL0[refIdxA][cIdx–1])<<(BitDepth C –8)
[0371] o1=(predListFlagB?ChromaOffsetL1[refIdxB][cIdx–1]:ChromaOffsetL0[refIdxB][cIdx–1])<<(BitDepth C –8)
[0372] The prediction samples pbSamples[x][y] (where x = 0..nCbW–1, y = 0..nCbH–1) are derived as follows:
[0373] pbSamples[x][y]=Clip3(0,(1< <bitDepth)–1,(predSamplesA[x][y]*w0+predSamplesB[x][y]*w1+((o0+o1+1)<<log2Wd))> >(log2Wd+1))
[0374] The parameters for slice-level weighted prediction can be represented as a set of variables that are assigned to each element in the reference picture list. The index of the element is further represented as "i". These parameters may include:
[0375] –LumaWeightL0[i]
[0376] – luma_offset_l0[i] is an additional offset applied to the luma prediction value used for prediction of list 0 using RefPicList[0][i]. luma_offset_l0[i] shall be in the range –128 to 127 (inclusive). When luma_weight_l0_flag[i] is equal to 0, luma_offset_l0[i] is inferred to be 0.
[0377] The variable LumaWeightL0[i] is derived as (1 << luma_log2_weight_denom) + delta_luma_weight_l0[i]. When luma_weight_l0_flag[i] is equal to 1, the value range of delta_luma_weight_l0[i] shall be from –128 to 127 (including the end values). When luma_weight_l0_flag[i] is equal to 0, LumaWeightL0[i] is inferred as 2 luma_log2_weight_denom 。
[0378] The hybrid weighting process for the rectangular block predicted using the bidirectional prediction mechanism is performed as described below.
[0379] The inputs to this process include:
[0380] – Two variables nCbW and nCbH, representing the width and height of the current coding block;
[0381] – Two (nCbW) × (nCbH) arrays predSamplesLA and predSamplesLB;
[0382] – The variable triangleDir, representing the partitioning direction;
[0383] – The variable cIdx, representing the color component index.
[0384] The output of this process is an (nCbW) × (nCbH) array pbsamples composed of predicted sample values.
[0385] The variable nCbR is derived as follows:
[0386] nCbR = (nCbW > nCbH)? (nCbW / nCbH) : (nCbH / nCbW)
[0387] The variable bitDepth is derived as follows:
[0388] – If cIdx is equal to 0, bitDepth is set to BitDepth Y 。
[0389] – Otherwise, bitDepth is set to BitDepth C 。
[0390] The variables shift1 and offset1 are derived as follows:
[0391] – The variable shift1 is set to Max(5, 17 – bitDepth).
[0392] –The variable offset1 is set to 1<<(shift1–1).
[0393] Based on the values of triangleDir, wS and cIdx, the prediction samples pbSamples[x][y] (where x = 0..nCbW–1, y = 0..nCbH–1) are derived as follows:
[0394] –The variable wIdx is derived as follows:
[0395] – If cIdx is equal to 0 and triangleDir is equal to 0, the following applies:
[0396] wIdx=(nCbW>nCbH)? (Clip3(0,8,(x / nCbR–y)+4)):(Clip3(0,8,(x–y / nCbR)+4))
[0397] – If cIdx is equal to 0 and triangleDir is equal to 1, the following applies:
[0398] wIdx=(nCbW>nCbH)? (Clip3(0,8,(nCbH–1–x / nCbR–y)+4))(Clip3(0,8,(nCbW–
[0399] 1–x–y / nCbR)+4))
[0400] – If cIdx is greater than 0 and triangleDir is equal to 0, the following applies:
[0401] wIdx=(nCbW>nCbH)? (Clip3(0,4,(x / nCbR–y)+2)):(Clip3(0,4,(x–y / nCbR)+2))
[0402] – Otherwise (if cIdx is greater than 0 and triangleDir is equal to 1), the following applies:
[0403] wIdx=(nCbW>nCbH)? (Clip3(0,4,(nCbH–1–x / nCbR–y)+2))(Clip3(0,4,(nCbW–
[0404] 1–x–y / nCbR)+2))
[0405] –The variable wValue representing the predicted sample weight is derived using wIdx and cIdx as follows:
[0406] wValue=(cIdx==0)? Clip3(0,8,wIdx):Clip3(0,8,wIdx*2)
[0407] –The derivation of the predicted sample value is as follows:
[0408] pbSamples[x][y]=Clip3(0,(1<<bitDepth)–1,(predSamplesLA[x][y]*wValue+predSamplesLB[x][y]*(8–wValue)+offset1)> >shift1)
[0409] For the geometry mode, the hybrid weighting process for rectangular blocks predicted using the bi-prediction mechanism is performed as follows.
[0410] Inputs to this process include:
[0411] – Two variables nCbW and nCbH, representing the width and height of the current coding block;
[0412] – two (nCbW) × (nCbH) arrays predSamplesLA and predSamplesLB;
[0413] – The variable angleIdx represents the angle index of the geometric partition;
[0414] – The variable distanceIdx represents the distance index of the geometric partition;
[0415] – The variable cIdx represents the color component index.
[0416] The output of this process includes the (nCbW)×(nCbH) array pbSamples of predicted sample values and the variable partIdx.
[0417] The variable bitDepth is derived as follows:
[0418] – If cIdx is equal to 0, bitDepth is set to BitDepth Y .
[0419] – Otherwise, bitDepth is set to BitDepth C .
[0420] The variables shift1 and offset1 are derived as follows:
[0421] – The variable shift1 is set to Max(5,17–bitDepth).
[0422] –The variable offset1 is set to 1<<(shift1–1).
[0423] The brightness weight array sampleWeight L [x][y] and the chroma weight array sampleWeight C [x][y] (where x = 0..nCbW–1, y = 0..nCbH–1) is derived as follows:
[0424] The values of the following variables are set as follows:
[0425] –hwRatio is set to nCbH / nCbW
[0426] –displacementX is set to angleIdx
[0427] – displacementY is set to (displacementX+8)%32
[0428] – partIdx is set to angleIdx>=13&&angleIdx<=27? 1:0
[0429] –rho is set to the following values via a lookup table denoted Dis (described in Table 8-12):
[0430] rho=(Dis[displacementX]<<8)+(Dis[displacementY]<<8)
[0431] The variable shiftHor is set to 0 if one of the following conditions is true:
[0432] angleIdx%16 is equal to 8,
[0433] angleIdx%16 is not equal to 0 and hwRatio≥1;
[0434] Otherwise, shiftHor is set to 1.
[0435] If shiftHor is equal to 0, offsetX and offsetY are derived as follows:
[0436] offsetX=(256–nCbW)>>1,
[0437] offsetY=(256–nCbH)>>1+angleIdx<16? (distanceIdx*nCbH)>>3:–((distanceIdx*nCbH)>>3)
[0438] If shiftHor is equal to 1, offsetX and offsetY are derived as follows:
[0439] offsetX=(256–nCbW)>>1+angleIdx<16? (distanceIdx*nCbW)>>3:–((distanceIdx*nCbW)>>3)
[0440] offsetY=(256–nCbH)>>1
[0441] The variables weightIdx and weightIdxAbs are calculated by looking up the table (Table 9) (where x = 0..nCbW–1, y = 0..nCbH–1) as follows:
[0442] weightIdx=(((x+offsetX)<<1)+1)*Dis[displacementX]+(((y+offsetY)<<1)+1))*Dis[displacementY]–rho.
[0443] weightIdxAbs=Clip3(0,26,abs(weightIdx)).
[0444] sampleWeight L The values of [x][y] (where x = 0..nCbW–1, y = 0..nCbH–1) are set according to Table 10 represented as GeoFilter as follows:
[0445] sampleWeight L [x][y]=weightIdx<=0? GeoFilter[weightIdxAbs]:8–GeoFilter[weightIdxAbs]
[0446] sampleWeight C The values of [x][y] (where x = 0..nCbW–1, y = 0..nCbH–1) are set as follows:
[0447] sampleWeight C [x][y]=sampleWeight L [(x<<(SubWidthC–1))][(y<<(SubHeightC–1))]
[0448] Note: sampleWeight L The values of [x][y] can also be obtained from sampleWeightL [x–shiftX][y–shiftY]. If angleIdx is greater than 4 and less than 12, or angleIdx is greater than 20 and less than 24, then shiftX is the tangent of the split angle and shiftY is 1; otherwise, shiftX is 1 and shiftY is the cotangent of the split angle. If the tangent (relative to the cotangent) value is infinite, then shiftX is 1 (relative to 0) or shiftY is 0 (relative to 1).
[0449] The derivation of the predicted sample value is as follows, where X is represented as L or C, and cIdx is equal to 0 or not equal to 0:
[0450] pbSamples[x][y]=partIdx? Clip3(0,(1< <bitDepth)–1,(predSamplesLA[x][y]*(8–
[0451] sampleWeight X [x][y])+predSamplesLB[x][y]*sampleWeight X [x][y]+offset1)>>shift1):Clip3(0,
[0452] (1< <bitDepth)–1,(predSamplesLA[x][y]*sampleWeight X [x][y]+predSamplesLB[x][y]*(8–sampleWeight X [x][y])+offset1)>>shift1)
[0453] Table 5: Lookup table Dis for deriving geometric partition distance
[0454] idx 0 1 2 4 6 7 8 9 10 12 14 15 Dis[idx] 8 8 8 8 4 2 0 –2 –4 –8 –8 –8 idx 16 17 18 20 22 23 24 25 26 28 30 31 Dis[idx] –8 –8 –8 –8 –4 –2 0 2 4 8 8 8
[0455] Table 6: Filter weight lookup table for deriving geometric partitioning filter weights GeoFilter
[0456] idx 0 1 2 3 4 5 6 7 8 9 10 11 12 13 GeoFilter[idx] 4 4 4 4 5 5 5 5 5 5 5 6 6 6 idx 14 15 16 17 18 19 20 21 22 23 24 25 26 GeoFilter[idx] 6 6 6 6 7 7 7 7 7 7 7 7 8
[0457] In the VVC specification draft 7 (document JVET-P2001-vE: "Versatile Video Coding (Draft 7)" published by B. Bross, J. Chen, S. Liu, Y.-K. Wang, which is the output document JVET-P2001 of the 16th JVET meeting in Geneva, Switzerland; the document is contained in the file JVET-P2001-v14: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 16_Geneva / wg11 / JVET-P2001-v14.zip), the concept of picture header (PH) is introduced as follows: a part of the syntax elements are moved from the slice header (SH) to the PH to reduce the signaling overhead caused by assigning the same or similar values to the same syntax elements in each SH associated with the PH. As shown in Table 7, the syntax element for controlling the maximum number of fusion candidates of the TPM fusion mode is indicated in PH, while the weighted prediction parameter is still indicated in SH as shown in Tables 8 and 10. The semantics of the syntax elements used in Tables 8 and 9 are described as follows.
[0458] Table 7: Image header RBSP syntax
[0459]
[0460] Image header RBSP semantics
[0461] The PH includes information common to all slices of the coded picture associated with the PH.
[0462] non_reference_picture_flag equal to 1 indicates that the picture associated with the PH is never used as a reference picture. non_reference_picture_flag equal to 0 indicates that the picture associated with the PH may or may not be used as a reference picture.
[0463] gdr_pic_flag equal to 1 indicates that the picture associated with PH is a gradual decoding refresh (GDR) picture. gdr_pic_flag equal to 0 indicates that the picture associated with PH is not a GDR picture.
[0464] After decoding a coded layer videosequence start (CLVSS) picture that is not the first picture in the codestream, no_output_of_prior_pics_flag affects the output of previously decoded pictures in the decoded picture buffer (DPB).
[0465] recovery_poc_cnt indicates the recovery point of the decoded image in the output order. If the current image is a GDR image associated with PH, and there is an image picA in the coded layer video sequence (CLVS) that is after the current GDR image in the decoding order and the PicOrderCntVal of image picA is equal to the PicOrderCntVal of the current GDR image plus the value of recovery_poc_cnt, then the image picA is called the recovery point image. Otherwise, the first image in the output order whose PicOrderCntVal is greater than the PicOrderCntVal of the current image plus the value of recovery_poc_cnt is called the recovery point image. The recovery point image should not be before the current GDR image in the decoding order. The value range of recovery_poc_cnt should be 0 to MaxPicOrderCntLsb–1 (including the end value).
[0466] NOTE 1: When gdr_enabled_flag is equal to 1 and the PicOrderCntVal of the current picture is greater than or equal to the RpPicOrderCntVal of the associated GDR picture, the current decoded picture and subsequent decoded pictures in output order exactly match the corresponding pictures resulting from starting the decoding process from the previous intra random access point (IRAP) picture (which, when present, precedes the associated GDR picture in the decoding order).
[0467] ph_pic_parameter_set_id indicates the value of pps_pic_parameter_set_id of the PPS currently used. The value range of ph_pic_parameter_set_id should be 0 to 63 (inclusive).
[0468] The requirement for code stream consistency is: the value of TemporalId of PH should be greater than or equal to the value of TemporalId of the picture parameter set (PPS) with pps_pic_parameter_set_id equal to ph_pic_parameter_set_id.
[0469] sps_poc_msb_flag equal to 1 indicates that the syntax element ph_poc_msb_cycle_present_flag is present in the PH of the referenced Sequence Parameter Set (SPS). sps_poc_msb_flag equal to 0 indicates that the syntax element ph_poc_msb_cycle_present_flag is not present in the PH of the referenced SPS.
[0470] ph_poc_msb_present_flag equal to 1 indicates that the syntax element poc_msb_val is present in the PH. ph_poc_msb_present_flag equal to 0 indicates that the syntax element poc_msb_val is not present in the PH. The value of ph_poc_msb_present_flag shall be equal to 0 when vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 0 and there is a picture in the current access unit (Access Unit, AU) in the reference layer of the current layer.
[0471] poc_msb_val represents the most significant bit (MSB) value of the picture order count (POC) of the current picture. The length of the syntax element poc_msb_val is (poc_msb_len_minus1+1) bits.
[0472] sps_triangle_enabled_flag indicates whether triangle-based motion compensation can be used for inter prediction. sps_triangle_enabled_flag equal to 0 indicates that the syntax should be constrained so that triangle-based motion compensation is not used in coded layer video sequence (CLVS) and merge_triangle_split_dir, merge_triangle_idx0 and merge_triangle_idx1 are not present in the coding unit syntax of CLVS. sps_triangle_enabled_flag equal to 1 indicates that triangle-based motion compensation can be used in CLVS.
[0473] pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 equal to 0 indicates that pic_max_num_merge_cand_minus_max_num_triangle_cand exists in the PH of the reference picture parameter set (PPS). pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 greater than 0 indicates that pic_max_num_merge_cand_minus_max_num_triangle_cand does not exist in the PH of the reference PPS. The value range of pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 should be 0 to MaxNumMergeCand–1.
[0474] pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 equal to 0 indicates that pic_max_num_merge_cand_minus_max_num_triangle_cand exists in the PH of the slice that references the PPS. pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 greater than 0 indicates that pic_max_num_merge_cand_minus_max_num_triangle_cand does not exist in the PH that references the PPS. The value range of pps_max_num_merge_cand_minus_max_num_triangle_cand_plus1 shall be 0 to MaxNumMergeCand–1.
[0475] pic_six_minus_max_num_merge_cand represents the maximum number of merged motion vector prediction (MVP) candidates supported in the slice associated with PH minus 6. The maximum number of merged MVP candidates MaxNumMergeCand is derived as follows:
[0476] MaxNumMergeCand=6–picsix_minus_max_num_merge_cand
[0477] MaxNumMergeCand shall be in the range 1 to 6, inclusive. When pic_six_minus_max_num_merge_cand is not present, the value of pic_six_minus_max_num_merge_cand is inferred to be pps_six_minus_max_num_merge_cand_plus1–1.
[0478] Table 8: Generic Stripe Header Syntax
[0479]
[0480]
[0481]
[0482] General Stripe Header Semantics
[0483] When the slice header syntax element slice_pic_order_cnt_lsb is present, the value of this syntax element shall be the same for all slice headers of a coded picture.
[0484] The variable CuQpDeltaVal represents the difference between the luma quantization parameter of the coding unit including cu_qp_delta_abs and its prediction parameter, and is set to 0. The variable CuQpOffset Cb 、CuQpOffset Cr and CuQpOffset CbCr Indicates the quantization parameter Qp' of the coding unit including cu_chroma_qp_offset_flag. Cb , Qp' Cr and Qp' CbCr These variables are all set to 0.
[0485] slice_pic_order_cnt_lsb represents the picture order number of the current picture modulo MaxPicOrderCntLsb. The length of the syntax element slice_pic_order_cnt_lsb is (log2_max_pic_order_cnt_lsb_minus4+4) bits. The value range of slice_pic_order_cnt_lsb shall be 0 to MaxPicOrderCntLsb–1 (inclusive).
[0486] When the current image is a GDR image, the derivation of the variable RpPicOrderCntVal is as follows:
[0487] RpPicOrderCntVal=PicOrderCntVal+recovery_poc_cnt.
[0488] slice_subpic_id represents the sub-picture identifier of the sub-picture comprising the slice. If slice_subpic_id exists, the value of the variable SubPicIdx is derived so that SubpicIdList[SubPicIdx] is equal to slice_subpic_id. Otherwise (slice_subpic_id does not exist), the variable SubPicIdx is derived to 0. The length (in bits) of slice_subpic_id is derived as follows:
[0489] – If sps_subpic_id_signalling_present_flag is equal to 1, the length of slice_subpic_id is equal to sps_subpic_id_len_minus1+1.
[0490] – Otherwise, if ph_subpic_id_signalling_present_flag is equal to 1, the length of slice_subpic_id is equal to ph_subpic_id_len_minus1+1.
[0491] – Otherwise, if pps_subpic_id_signalling_present_flag is equal to 1, the length of slice_subpic_id is equal to pps_subpic_id_len_minus1+1.
[0492] – Otherwise, the length of slice_subpic_id is equal to Ceil(Log2(sps_num_subpics_minus1+1)).
[0493] slice_address indicates the slice address of the slice. When slice_address does not exist, the value of slice_address is inferred to be 0.
[0494] If rect_slice_flag is equal to 0, the following applies:
[0495] – Stripe address is the raster scan block index.
[0496] – The length of slice_address is Ceil(Log2(NumTilesInPic)) bits.
[0497] –slice_address should range from 0 to NumTilesInPic–1 (inclusive).
[0498] Otherwise (rect_slice_flag is equal to 1), the following applies:
[0499] – The slice address is the slice index of the slice within the SubPicIdx-th sub-picture.
[0500] – The length of the stripe address is Ceil(Log2(NumSlicesInSubpic[SubPicIdx])) bits.
[0501] –slice_address shall have a value range of 0 to NumSlicesInSubpic[SubPicIdx]–1 (inclusive).
[0502] The codestream consistency requirement is that the following constraints apply:
[0503] – If rect_slice_flag is equal to 0 or subpics_present_flag is equal to 0, the value of slice_address shall not be equal to the value of slice_address of any other coded slice Network Abstraction Layer (NAL) unit of the same coded picture.
[0504] – Otherwise, the pair of slice_subpic_id and slice_address values shall not be equal to the pair of slice_subpic_id and slice_address values of any other coded slice NAL unit of the same coded picture.
[0505] – When rect_slice_flag is equal to 0, the slices in the picture shall be arranged in ascending order according to the slice_address values of the slices.
[0506] – The shape of the slices in a picture shall be such that each Coding Tree Unit (CTU) shall have its entire left and upper boundaries consisting of picture boundaries or of boundaries of one or more previously decoded CTUs when decoded.
[0507] num_tiles_in_slice_minus1+1 (when present) indicates the number of tiles in the slice. The value range of num_tiles_in_slice_minus1 should be 0 to NumTilesInPic–1 (inclusive).
[0508] The variable NumCtuInCurrSlice represents the number of CTUs in the current slice, and the list CtbAddrInCurrSlice[i] (i ranges from 0 to NumCtuInCurrSlice–1 (including the end value)) represents the image raster scan address of the i-th coding tree block (CTB) in the slice. The derivation of the two is as follows:
[0509]
[0510] The variables SubPicLeftBoundaryPos, SubPicTopBoundaryPos, SubPicRightBoundaryPos, and SubPicBotBoundaryPos are derived as follows:
[0511]
[0512] slice_type indicates the encoding type of the slice, as shown in Table 9.
[0513] Table 9: Association between name and slice_type
[0514] slice_type The name of slice_type 0 B (B band) 1 P(P stripe) 2 I(I strip)
[0515] slice_rpl_sps_flag[i] equal to 1 indicates that the reference picture list i of the current slice is derived from one of the syntax structures ref_pic_list_struct(listIdx, rplsIdx) with listIdx equal to i in the SPS. slice_rpl_sps_flag[i] equal to 0 indicates that the reference picture list i of the current slice is derived from the syntax structure ref_pic_list_struct(listIdx, rplsIdx) with listIdx equal to i directly included in the slice header of the current picture.
[0516] When slice_rpl_sps_flag[i] is not present, the following applies:
[0517] – If pic_rpl_present_flag is equal to 1, the value of slice_rpl_sps_flag[i] is inferred to be pic_rpl_sps_flag[i];
[0518] – Otherwise, if num_ref_pic_lists_in_sps[i] is equal to 0, the value of ref_pic_list_sps_flag[i] is inferred to be 0;
[0519] – Otherwise, if num_ref_pic_lists_in_sps[i] is greater than 0 and if rpl1_idx_present_flag is equal to 0, the value of slice_rpl_sps_flag[1] is inferred to be slice_rpl_sps_flag[0].
[0520] slice_rpl_idx[i] represents the index in the list of syntax structures ref_pic_list_struct(listIdx,rplsIdx) with listIdx equal to i included in the SPS of the syntax structure ref_pic_list_struct(listIdx,rplsIdx) with listIdx equal to i used to derive the reference picture list i of the current picture. The syntax element slice_rpl_idx[i] is represented by Ceil(Log2(num_ref_pic_lists_in_sps[i])) bits. When slice_rpl_idx[i] does not exist, the value of slice_rpl_idx[i] is inferred to be 0. The value range of slice_rpl_idx[i] should be 0 to num_ref_pic_lists_in_sps[i]–1 (including the end value). When slice_rpl_sps_flag[i] is equal to 1 and num_ref_pic_lists_in_sps[i] is equal to 1, the value of slice_rpl_idx[i] is inferred to be 0. When slice_rpl_sps_flag[i] is equal to 1 and rpl1_idx_present_flag is equal to 0, the value of slice_rpl_idx[1] is inferred to be slice_rpl_idx[0].
[0521] The derivation of the variable RplsIdx[i] is as follows:
[0522]
[0523] slice_poc_lsb_lt[i][j] represents the value of the picture order number of the j-th LTRP entry in the i-th reference picture list modulo MaxPicOrderCntLsb. The length of the syntax element slice_poc_lsb_lt[i][j] is (log2_max_pic_order_cnt_lsb_minus4+4) bits.
[0524] The variable PocLsbLt[i][j] is derived as follows:
[0525]
[0526] slice_delta_poc_msb_present_flag[i][j] is equal to 1, indicating that slice_delta_poc_msb_cycle_lt[i][j] is present. slice_delta_poc_msb_present_flag[i][j] is equal to 0, indicating that slice_delta_poc_msb_cycle_lt[i][j] is not present.
[0527] Assume that prevTid0Pic is the previous picture in decoding order, the previous picture has the same nuh_layer_id as the current picture, the previous picture's TemporalId is equal to 0, and the previous picture is not a Random Access Skipped Leading (RASL) picture or a Random Access Decodable Leading (RADL) picture. Assume that setOfPrevPocVals is the set consisting of:
[0528] –PicOrderCntVal of prevTid0Pic,
[0529] – PicOrderCntVal of each picture referenced by an entry in RefPicList[0] or RefPicList[1] of prevTid0Pic and having the same nuh_layer_id as the current picture,
[0530] – PicOrderCntVal of each picture after prevTid0Pic in decoding order, with the same nuh_layer_id as the current picture and before the current picture in decoding order.
[0531] When pic_rpl_present_flag is equal to 0 and there is a multiple value in setOfPrevPocVals (whose value is equal to PocLsbLt[i][j] modulo MaxPicOrderCntLsb), the value of slice_delta_poc_msb_present_flag[i][j] shall be equal to 1.
[0532] slice_delta_poc_msb_cycle_lt[i][j] represents the value of the variable FullPocLt[i][j], as shown below:
[0533]
[0534] The value range of slice_delta_poc_msb_cycle_lt[i][j] shall be 0 to 2. (32 –log2 _ max _ pic _ order _ cnt _ lsb _ minus4–4) (Inclusive of the end value). When slice_delta_poc_msb_cycle_lt[i][j] is not present, the value of slice_delta_poc_msb_cycle_lt[i][j] is inferred to be 0.
[0535] num_ref_idx_active_override_flag equal to 1 indicates that the syntax element num_ref_idx_active_minus1[0] is present for P slices and B slices and the syntax element num_ref_idx_active_minus1[1] is present for B slices. num_ref_idx_active_override_flag equal to 0 indicates that the syntax elements num_ref_idx_active_minus1[0] and num_ref_idx_active_minus1[1] are not present. When num_ref_idx_active_override_flag is not present, the value of num_ref_idx_active_override_flag is inferred to be 1.
[0536] num_ref_idx_active_minus1[i] is used to derive the variable NumRefIdxActive[i] shown in equation 145. The value range of num_ref_idx_active_minus1[i] shall be 0 to 14 (inclusive).
[0537] When i is equal to 0 or 1, when the current slice is a B slice, num_ref_idx_active_override_flag is equal to 1, and num_ref_idx_active_minus1[i] does not exist, num_ref_idx_active_minus1[i] is inferred to be 0.
[0538] When the current slice is a P slice, num_ref_idx_active_override_flag is equal to 1, and num_ref_idx_active_minus1[0] does not exist, num_ref_idx_active_minus1[0] is inferred to be 0.
[0539] The derivation of the variable NumRefIdxActive[i] is as follows:
[0540]
[0541] The value of NumRefIdxActive[i]-1 indicates the maximum reference index in reference picture list i that can be used for decoding slices. When the value of NumRefIdxActive[i] is equal to 0, the reference index in reference picture list i cannot be used for decoding slices.
[0542] When the current slice is a P slice, the value of NumRefIdxActive[0] should be greater than 0.
[0543] When the current stripe is the B stripe, both NumRefIdxActive[0] and NumRefIdxActive[1] should be greater than 0.
[0544] Weighted prediction parameter syntax
[0545]
[0546]
[0547] Weighted prediction parameter semantics
[0548] luma_log2_weight_denom is the logarithm of the denominator of all luma weighting factors to base 2. The value range of luma_log2_weight_denom should be 0 to 7 (inclusive).
[0549] delta_chroma_log2_weight_denom is the difference of the denominators of all chroma weighting factors with base 2 logarithms. When delta_chroma_log2_weight_denom is not present, delta_chroma_log2_weight_denom is inferred to be 0.
[0550] The variable ChromaLog2WeightDenom is derived as luma_log2_weight_denom+delta_chroma_log2_weight_denom, and the value range of this variable should be 0 to 7 (including the end values).
[0551] When luma_weight_l0_flag[i] equals 1, it indicates that there exists a weighting factor for the luma component used for prediction in List 0 using RefPicList[0][i]. When luma_weight_l0_flag[i] equals 0, it indicates that these weighting factors do not exist.
[0552] When chroma_weight_l0_flag[i] equals 1, it indicates that there exists a weighting factor for the chroma prediction value used for prediction in List 0 using RefPicList[0][i]. When chroma_weight_l0_flag[i] equals 0, it indicates that these weighting factors do not exist. When chroma_weight_l0_flag[i] does not exist, chroma_weight_l0_flag[i] is inferred to be 0.
[0553] delta_luma_weight_l0[i] is the difference of the weighting factor applied to the luma prediction value used for prediction in List 0 using RefPicList[0][i].
[0554] The variable LumaWeightL0[i] is derived as (1<<luma_log2_weight_denom)+delta_luma_weight_l0[i]. When luma_weight_l0_flag[i] equals 1, the value range of delta_luma_weight_l0[i] shall be from -128 to 127 (including the end values). When luma_weight_l0_flag[i] equals 0, LumaWeightL0[i] is inferred to be 2luma_log2_weight_denom.
[0555] luma_offset_l0[i] is an additional offset applied to the luma prediction value used for prediction in List 0 using RefPicList[0][i]. The value range of luma_offset_l0[i] shall be from -128 to 127 (including the end values). When luma_weight_l0_flag[i] equals 0, luma_offset_l0[i] is inferred to be 0.
[0556] delta_chroma_weight_l0[i][j] is the difference of the weighting factor applied to the chroma prediction value used for prediction in List 0 using RefPicList[0][i], where for Cb, j equals 0, and for Cr, j equals 1.
[0557] The variable ChromaWeightL0[i][j] is derived as (1<<ChromaLog2WeightDenom)+delta_chroma_weight_l0[i][j]. When chroma_weight_l0_flag[i] equals 1, the value range of delta_chroma_weight_l0[i][j] shall be from –128 to 127 (including the end values). When chroma_weight_l0_flag[i] equals 0, ChromaWeightL0[i][j] is inferred to be 2ChromaLog2WeightDenom.
[0558] delta_chroma_offset_l0[i][j] is the difference of the additional offset applied to the chroma prediction value used for prediction in list 0 using RefPicList[0][i], where for Cb, j equals 0, and for Cr, j equals 1.
[0559] The derivation of the variable ChromaOffsetL0[i][j] is as follows:
[0560] ChromaOffsetL0[i][j] = Clip3(–128, 127, (128 + delta_chroma_offset_l0[i][j] – ((128 *
[0561] ChromaWeightL0[i][j]) >> ChromaLog2WeightDenom))
[0562] The value range of delta_chroma_offset_l0[i][j] shall be from –4 * 128 to 4 * 127 (including the end values). When chroma_weight_l0_flag[i] equals 0, ChromaOffsetL0[i][j] is inferred to be 0.
[0563] The semantics of luma_weight_l1_flag[i], chroma_weight_l1_flag[i], delta_luma_weight_l1[i], luma_offset_l1[i], delta_chroma_weight_l1[i][j] and delta_chroma_offset_l1[i][j] are the same as luma_weight_l0_flag[i], chroma_weight_l0_flag[i], delta_luma_weight_l0[i], luma_offset_l0[i], delta_chroma_weight_l0[i][j] and delta_chroma_offset_l0[i][j], respectively, where l0, L0, List0 and List0 are replaced by l1, L1, List1 and List1, respectively.
[0564] The variable sumWeightL0Flags is derived as the sum of luma_weight_l0_flag[i]+2*chroma_weight_l0_flag[i], where i=0..NumRefIdxActive[0]–1.
[0565] When slice_type is B, the variable sumWeightL1Flags is derived as the sum of luma_weight_l1_flag[i]+2*chroma_weight_l1_flag[i], where i=0..NumRefIdxActive[1]–1.
[0566] The requirements for code stream consistency are: when slice_type is P, sumWeightL0Flags should be less than or equal to 24; when slice_type is B, the sum of sumWeightL0Flags and sumWeightL1Flags should be less than or equal to 24.
[0567] Reference picture list structure semantics
[0568] The syntax structure ref_pic_list_struct(listIdx, rplsIdx) can exist in the SPS or in the slice header. Depending on whether the syntax structure is included in the slice header or in the SPS, the following applies:
[0569] – If the syntax structure ref_pic_list_struct (listIdx, rplsIdx) is present in the slice header, this syntax structure indicates the reference picture list listIdx of the current picture (including pictures of the slice).
[0570] – Otherwise (present in the SPS), the syntax structure ref_pic_list_struct(listIdx, rplsIdx) indicates a candidate for the reference picture list listIdx. The term "current picture" in the semantics specified in the rest of this clause refers to: (1) each picture including one or more slices, where one or more slices include ref_pic_list_idx[listIdx] equal to the index in the list of the syntax structure ref_pic_list_struct(listIdx, rplsIdx) included in the SPS, and (2) each picture in a Coded Video Sequence (CVS) that references the SPS.
[0571] num_ref_entries[listIdx][rplsIdx] indicates the number of entries in the syntax structure ref_pic_list_struct(listIdx, rplsIdx). The value range of num_ref_entries[listIdx][rplsIdx] should be 0 to MaxDecPicBuffMinus1+14 (including the end value).
[0572] ltrp_in_slice_header_flag[listIdx][rplsIdx] is equal to 0, indicating that the POC LSB of the LTRP entry in the syntax structure ref_pic_list_struct(listIdx, rplsIdx) exists in the syntax structure ref_pic_list_struct(listIdx, rplsIdx). ltrp_in_slice_header_flag[listIdx][rplsIdx] is equal to 1, indicating that the POC LSB of the Long-Term Reference Picture (LTRP) entry in the syntax structure ref_pic_list_struct(listIdx, rplsIdx) does not exist in the syntax structure ref_pic_list_struct(listIdx, rplsIdx).
[0573] inter_layer_ref_pic_flag[listIdx][rplsIdx][i] equal to 1 indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx, rplsIdx) is an Inter-Layer Reference Picture (ILRP) entry. inter_layer_ref_pic_flag[listIdx][rplsIdx][i] equal to 0 indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx, rplsIdx) is not an ILRP entry. When inter_layer_ref_pic_flag[listIdx][rplsIdx][i] does not exist, the value of inter_layer_ref_pic_flag[listIdx][rplsIdx][i] is inferred to be 0.
[0574] st_ref_pic_flag[listIdx][rplsIdx][i] equal to 1 indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx, rplsIdx) is a STRP entry. st_ref_pic_flag[listIdx][rplsIdx][i] equal to 0 indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx, rplsIdx) is a LTRP entry. When inter_layer_ref_pic_flag[listIdx][rplsIdx][i] is equal to 0 and st_ref_pic_flag[listIdx][rplsIdx][i] is not present, the value of st_ref_pic_flag[listIdx][rplsIdx][i] is inferred to be 1.
[0575] The variable NumLtrpEntries[listIdx][rplsIdx] is derived as follows:
[0576] for(i=0,NumLtrpEntries[listIdx][rplsIdx]=0; i <num_ref_entries[listIdx][rplsIdx];i++)
[0577] if(!inter_layer_ref_pic_flag[listIdx][rplsIdx][i]&&!st_ref_pic_flag[listIdx][rplsIdx][i])
[0578] NumLtrpEntries[listIdx][rplsIdx]++
[0579] abs_delta_poc_st[listIdx][rplsIdx][i] represents the value of the variable AbsDeltaPocSt[listIdx][rplsIdx][i], as shown below:
[0580]
[0581] strp_entry_sign_flag[listIdx][rplsIdx][i] equal to 1 indicates that the value of the i-th entry in the syntax structure ref_pic_list_struct(listIdx, rplsIdx) is greater than or equal to 0. strp_entry_sign_flag[listIdx][rplsIdx][i] equal to 0 indicates that the value of the i-th entry in the syntax structure ref_pic_list_struct(listIdx, rplsIdx) is less than 0. When strp_entry_sign_flag[listIdx][rplsIdx][i] is not present, the value of strp_entry_sign_flag[listIdx][rplsIdx][i] is inferred to be 1.
[0582] The derivation of the list DeltaPocValSt[listIdx][rplsIdx] is as follows:
[0583]
[0584] rpls_poc_lsb_lt[listIdx][rplsIdx][i] represents the value of the picture order number of the picture referenced by the i-th entry in the syntax structure ref_pic_list_struct(listIdx, rplsIdx) modulo MaxPicOrderCntLsb. The length of the syntax element rpls_poc_lsb_lt[listIdx][rplsIdx][i] is (log2_max_pic_order_cnt_lsb_minus4+4) bits.
[0585] ilrp_idx[listIdx][rplsIdx][i] represents the index of the ILRP in the direct reference layer list of the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx). The value range of ilrp_idx[listIdx][rplsIdx][i] shall be 0 to NumDirectRefLayers[GeneralLayerIdx[nuh_layer_id]]–1 (inclusive).
[0586] Therefore, different mechanisms can be used to control the GEO / TPM fusion mode, depending on whether WP is applied to the reference image from which reference blocks P0 and P1 are obtained in the following way:
[0587] – Move the WP parameters listed in Table 14 from SH to PH;
[0588] – Move GEO / TPM parameters from PH back to SH;
[0589] – Change the semantics of MaxNumTriangleMergeCand, i.e., set MaxNumTriangleMergeCand to 0 or 1 for these slices when a WP-passed reference picture can be used (eg, at least one flag lumaWeightedFlag is true).
[0590] For TPM fusion mode, the exemplary reference blocks P0 and P1 are Figure 7 In FIG. 7 , the exemplary reference blocks P0 and P1 are represented as 710 and 720 respectively. For the GEO fusion mode, the exemplary reference blocks P0 and P1 are represented as 710 and 720 respectively. Figure 8 They are represented as 810 and 820 respectively.
[0591] Therefore, different mechanisms can be used to control the GEO / TPM fusion mode, depending on whether WP is applied to the reference image from which reference blocks P0 and P1 are obtained in the following way:
[0592] – Move the WP parameters listed in Table 14 from SH to PH;
[0593] – Move GEO / TPM parameters from PH back to SH;
[0594] – Change the semantics of MaxNumTriangleMergeCand, i.e., set MaxNumTriangleMergeCand to 0 or 1 for these slices when a WP-passed reference picture can be used (eg, at least one flag lumaWeightedFlag is true).
[0595] For TPM fusion mode, the exemplary reference blocks P0 and P1 are Figure 7 In FIG. 7 , the exemplary reference blocks P0 and P1 are represented as 710 and 720 respectively. For the GEO fusion mode, the exemplary reference blocks P0 and P1 are represented as 710 and 720 respectively. Figure 8 They are represented as 810 and 820 respectively.
[0596] In one embodiment, when the enablement of non-rectangular modes (e.g., GEO and TPM) and the WP parameter are indicated in the image header, the following syntax may be used, as shown in the following table:
[0597] Table: Image header RBSP syntax
[0598]
[0599]
[0600]
[0601] …
[0602] The variable WPDisabled is set to 1 when all values of luma_weight_l0_flag[i], chroma_weight_l0_flag[i], luma_weight_l1_flag[j] and chroma_weight_l1_flag[j] are set to 0, where i=0..NumRefIdxActive[0], j=0..NumRefIdxActive[1]; otherwise, the value of WPDisabled is set to 0.
[0603] When the variable WPDisabled is set to 0, the value of pic_max_num_merge_cand_minus_max_num_triangle_cand is set to MaxNumMergeCand.
[0604] In one example, the indication of the WP parameters and the enabling of non-rectangular modes (eg, GEO and TPM) are performed in the stripe header. The following table gives an exemplary syntax.
[0605]
[0606]
[0607]
[0608]
[0609]
[0610]
[0611] …
[0612] The variable WPDisabled is set to 1 when all values of luma_weight_l0_flag[i], chroma_weight_l0_flag[i], luma_weight_l1_flag[j] and chroma_weight_l1_flag[j] are set to 0, where i=0..NumRefIdxActive[0], j=0..NumRefIdxActive[1]; otherwise, the value of WPDisabled is set to 0.
[0613] When the variable WPDisabled is set to 0, the value of max_num_merge_cand_minus_max_num_triangle_cand is set to MaxNumMergeCand.
[0614] In the above disclosed embodiments, the weighted prediction parameters may be indicated in a picture header or a slice header.
[0615] In one example, whether TPM or GEO is enabled is determined when considering the reference picture lists that the block can use for non-rectangular weighted prediction. When the fused list of the block includes only elements from one reference picture list k, the value of the variable WPDisabled[k] determines whether the fused mode is enabled.
[0616] In one example, the merge list for the non-rectangular inter prediction mode is constructed such that the merge list only includes elements for which weighted prediction is not enabled.
[0617] The following section of the instructions illustrates this example:
[0618] Inputs to this process include:
[0619] – The luminance position (xCb, yCb) of the upper left sample of the current luminance coding block relative to the upper left luminance sample of the current image;
[0620] – The variable cbWidth indicates the width of the current coding block in units of luma samples;
[0621] – The variable cbHeight indicates the height of the current coding block in units of luma samples.
[0622] The output of this process is as follows, where X is either 0 or 1:
[0623] –Availability flag of the adjacent coding unit availableFlagA 0 、availableFlagA 1 、availableFlagB 0 ,
[0624] availableFlagB 1 and availableFlagB 2 ;
[0625] – Reference index refIdxLXA of the neighboring coding unit 0 ,refIdxLXA 1 ,refIdxLXB 0 ,refIdxLXB 1 and refIdxLXB 2 ;
[0626] – The prediction list of adjacent coding units uses the flag predFlagLXA 0 、predFlagLXA 1 、predFlagLXB 0 、predFlagLXB 1 and predFlagLXB 2 ;
[0627] – Motion vector mvLXA of the adjacent coding unit with 1 / 16 fractional sample accuracy 0 、mvLXA 1 、mvLXB 0 、mvLXB 1 and mvLXB 2 ;
[0628] – Half-sample interpolation filter index hpelIfIdxA 0 、hpelIfIdxA 1 、hpelIfIdxB 0 、hpelIfIdxB 1 and hpelIfIdxB 2 ;
[0629] – Bidirectional prediction weight index bcwIdxA 0 ,bcwIdxA 1 ,bcwIdxB 0 ,bcwIdxB 1 and bcwIdxB 2 .
[0630] To derive availableFlagB1 ,refIdxLXB 1 、predFlagLXB 1 、mvLXB 1 、hpelIfIdxB 1 and bcwIdxB 1 , the following apply:
[0631] – Luminance position within adjacent luminance coding blocks (xNbB 1 ,yNb 1 ) is set to (xCb+cbWidth–1,yCb–1).
[0632] – The neighboring block availability derivation process detailed in Section 6.4.4 is called, where the inputs include the current luma position (xCurr, yCurr) set to (xCb, yCb), the neighboring luma position (xNbB 1 ,yNb 1 ), checkPredModeY is set to true and cIdx is set to 0, the output is assigned to the block availability flag availableB 1 .
[0633] –Variable availableFlagB 1 ,refIdxLXB 1 、predFlagLXB 1 、mvLXB 1 、hpelIfIdxB 1 and bcwIdxB 1 The derivation is as follows:
[0634] – If availableB 1 If it is false, availableFlagB 1 is set to 0, mvLXB 1 Both components of are set to 0,
[0635] refIdxLXB 1 is set to –1 and predFlagLXB 1 is set to 0 (where X is 0 or 1), hpelIfIdxB 1 is set to 0, bcwIdxB 1 is set to 0.
[0636] – Otherwise, availableFlagB 1 is set to 1, and the following assignments are made:
[0637] mxDV 1= MvLX[ xNbB 1 ][ 1 ] (501)
[0638] refIdxLXB 1 = RefIdxLX[ xNbB 1 ][ 1 ] (502)
[0639] predFlagLXB 1 = PredFlagLX[ xNbB 1 ][ 1 ] (503)
[0640] hpelIfIdxB 1 = HpelIfIdx[ xNbB 1 ][ 1 ] (504)
[0641] bxf 1 = BcwIdx[ xNbB 1 ][ 1 ] (505)
[0642] To derive availableFlagA 1 ,refIdxLXA 1 、predFlagLXA 1 、mvLXA 1 、hpelIfIdxA 1 and bcwIdxA 1 , the following apply:
[0643] – Luminance position within adjacent luminance coding blocks (xNbA 1 ,y 1 ) is set to (xCb–1,yCb+cbHeight–1).
[0644] – The neighboring block availability derivation process detailed in Section 6.4.4 is called, where the inputs include the current luma position (xCurr, yCurr) set to (xCb, yCb), the neighboring luma position (xNbA 1 ,y 1 ), checkPredModeY is set to true and cIdx is set to 0, the output is assigned to the block availability flag availableA 1 .
[0645] –Variable availableFlagA 1 ,refIdxLXA 1 、predFlagLXA 1 、mvLXA 1 、hpelIfIdxA 1 and bcwIdxA 1 The derivation is as follows:
[0646] – If one or more of the following conditions are true, availableFlagA 1 is set to 0, mvLXA 1 Both components of refIdxLXA are set to 0, 1 is set to –1 and predFlagLXA 1 is set to 0 (where X is 0 or 1), hpelIfIdxA 1 is set to 0, bcwIdxA 1 is set to 0:
[0647] –availableA 1 is false,
[0648] –availableB 1 True, the brightness position (xNbA 1 ,y 1 ) and (xNbB 1 ,yNb 1 ) have the same motion vector and the same reference index,
[0649] –WPDisabledX[RefIdxLX[xNbA 1 ][yNbA 1 ]] is set to 0, the blending mode is non-rectangular blending mode (for example, the triangle flag of the block is set to 1 at the current brightness position (xCurr, yCurr)),
[0650] –WPDisabledX[RefIdxLX[xNbB 1 ][yNb 1 ]] is set to 0, the blending mode is non-rectangular blending mode (for example, the triangle flag of the block is set to 1 at the current brightness position (xCurr, yCurr));
[0651] – Otherwise, availableFlagA 1 is set to 1, and the following assignments are made:
[0652] mxDV 1 = MvLX[ xNbA1 ][yNb 1 ] (506)
[0653] refIdxLXA 1 = RefIdxLX[ xNbA 1 ][yNb 1 ] (507)
[0654] predFlagLXA 1 = PredFlagLX[ xNbA 1 ][yNb 1 ] (508)
[0655] hpelIfIdxA 1 = HpelIfIdx[ xNbA 1 ][yNb 1 ] (509)
[0656] wxya 1 = BcwIdx[ xNbA 1 ][yNb 1 ] (510)
[0657] To derive availableFlagB 0 ,refIdxLXB 0 、predFlagLXB 0 、mvLXB 0 、hpelIfIdxB 0 and bcwIdxB 0 , the following applies:
[0658] – Luminance position within adjacent luminance coding blocks (xNbB 0 ,yNb 0 ) is set to (xCb+cbWidth,yCb–1).
[0659] – The neighboring block availability derivation process detailed in Section 6.4.4 is called, where the inputs include the current luma position (xCurr, yCurr) set to (xCb, yCb), the neighboring luma position (xNbB 0 ,yNb 0 ), checkPredModeY is set to true and cIdx is set to 0, the output is assigned to the block availability flag availableB 0 .
[0660] –Variable availableFlagB0 ,refIdxLXB 0 、predFlagLXB 0 、mvLXB 0 、hpelIfIdxB 0 and bcwIdxB 0 The derivation is as follows:
[0661] – If one or more of the following conditions are true, availableFlagB 0 is set to 0, mvLXB 0 Both components of refIdxLXB are set to 0, 0 is set to –1 and predFlagLXB 0 is set to 0 (where X is 0 or 1), hpelIfIdxB 0 is set to 0, bcwIdxB 0 is set to 0:
[0662] –availableB 0 is false,
[0663] –availableB 1 True, the brightness position (xNbB 1 ,yNb 1 ) and (xNbB 0 ,yNb 0 ) have the same motion vector and the same reference index,
[0664] –WPDisabledX[RefIdxLX[xNbB 0 ][yNb 0 ]] is set to 0, the blending mode is non-rectangular blending mode (for example, the triangle flag of the block is set to 1 at the current brightness position (xCurr, yCurr)),
[0665] –WPDisabledX[RefIdxLX[xNbB 1 ][yNb 1 ]] is set to 0, the blending mode is non-rectangular blending mode (for example, the triangle flag of the block is set to 1 at the current brightness position (xCurr, yCurr));
[0666] – Otherwise, availableFlagB 0 is set to 1, and the following assignments are made:
[0667] mxDV 0 = MvLX[ xNbB 0 ][0 ] (511)
[0668] refIdxLXB 0 = RefIdxLX[ xNbB 0 ][ 0 ] (512)
[0669] predFlagLXB 0 = PredFlagLX[ xNbB 0 ][ 0 ] (513)
[0670] hpelIfIdxB 0 = HpelIfIdx[ xNbB 0 ][ 0 ] (514)
[0671] bxf 0 = BcwIdx[ xNbB 0 ][ 0 ] (515)
[0672] To derive availableFlagA 0 ,refIdxLXA 0 、predFlagLXA 0 、mvLXA 0 、hpelIfIdxA 0 and bcwIdxA 0 , the following apply:
[0673] – Luminance position within adjacent luminance coding blocks (xNbA 0 ,y 0 ) is set to (xCb–1,yCb+cbWidth).
[0674] – The neighboring block availability derivation process detailed in Section 6.4.4 is called, where the inputs include the current luma position (xCurr, yCurr) set to (xCb, yCb), the neighboring luma position (xNbA 0 ,y 0 ), checkPredModeY is set to true and cIdx is set to 0, the output is assigned to the block availability flag availableA 0 .
[0675] –Variable availableFlagA 0 ,refIdxLXA0 、predFlagLXA 0 、mvLXA 0 、hpelIfIdxA 0 and bcwIdxA 0 The derivation is as follows:
[0676] – If one or more of the following conditions are true, availableFlagA 0 is set to 0, mvLXA 0 Both components of refIdxLXA are set to 0, 0 is set to –1 and predFlagLXA 0 is set to 0 (where X is 0 or 1), hpelIfIdxA 0 is set to 0, bcwIdxA 0 is set to 0:
[0677] –availableA 0 is false,
[0678] –availableA 1 True, the brightness position (xNbA 1 ,y 1 ) and (xNbA 0 ,y 0 ) have the same motion vector and the same reference index,
[0679] –WPDisabledX[RefIdxLX[xNbA 0 ][yNbA 0 ]] is set to 0, the blending mode is non-rectangular blending mode (for example, the triangle flag of the block is set to 1 at the current brightness position (xCurr, yCurr)),
[0680] –WPDisabledX[RefIdxLX[xNbA 1 ][yNbA 1 ]] is set to 0, the blending mode is non-rectangular blending mode (for example, the triangle flag of the block is set to 1 at the current brightness position (xCurr, yCurr));
[0681] – Otherwise, availableFlagA 0 is set to 1, and the following assignments are made:
[0682] mxDV 0 = MvLX[ xNbA 0 ][yNb 0 ] (516)
[0683] refIdxLXA 0 = RefIdxLX[ xNbA 0 ][yNb 0 ] (517)
[0684] predFlagLXA 0 = PredFlagLX[ xNbA 0 ][yNb 0 ] (518)
[0685] hpelIfIdxA 0 = HpelIfIdx[ xNbA 0 ][yNb 0 ] (519)
[0686] wxya 0 = BcwIdx[ xNbA 0 ][yNb 0 ] (520)
[0687] To derive availableFlagB 2 ,refIdxLXB 2 、predFlagLXB 2 、mvLXB 2 、hpelIfIdxB 2 and bcwIdxB 2 , the following applies:
[0688] – Luminance position within adjacent luminance coding blocks (xNbB 2 ,yNb 2 ) is set to (xCb–1,yCb–1).
[0689] – The neighboring block availability derivation process detailed in Section 6.4.4 is called, where the inputs include the current luma position (xCurr, yCurr) set to (xCb, yCb), the neighboring luma position (xNbB 2 ,yNb 2 ), checkPredModeY is set to true and cIdx is set to 0, the output is assigned to the block availability flag availableB 2 .
[0690] –Variable availableFlagB 2 ,refIdxLXB 2 、predFlagLXB2 、mvLXB 2 、hpelIfIdxB 2 and bcwIdxB 2 The derivation is as follows:
[0691] – If one or more of the following conditions are true, availableFlagB 2 is set to 0, mvLXB 2 Both components of refIdxLXB are set to 0, 2 is set to –1 and predFlagLXB 2 is set to 0 (where X is 0 or 1), hpelIfIdxB 2 is set to 0, bcwIdxB 2 is set to 0:
[0692] –availableB 2 is false,
[0693] –availableA 1 True, the brightness position (xNbA 1 ,y 1 ) and (xNbB 2 ,yNb 2 ) have the same motion vector and the same reference index,
[0694] –availableB 1 True, the brightness position (xNbB 1 ,yNb 1 ) and (xNbB 2 ,yNb 2 ) have the same motion vector and the same reference index,
[0695] –availableFlagA 0 +availableFlagA 1 +availableFlagB 0 +availableFlagB 1 Equals 4.
[0696] –WPDisabledX[RefIdxLX[xNbB 1 ][yNb 1 ]] is set to 0, the blending mode is non-rectangular blending mode (for example, the triangle flag of the block is set to 1 at the current brightness position (xCurr, yCurr)),
[0697] –WPDisabledX[RefIdxLX[xNbB2 ][yNb 2 ]] is set to 0, the blending mode is non-rectangular blending mode (for example, the triangle flag of the block is set to 1 at the current brightness position (xCurr, yCurr));
[0698] – Otherwise, availableFlagB 2 is set to 1, and the following assignments are made:
[0699] mxDV 2 = MvLX[ xNbB 2 ][ 2 ] (521)
[0700] refIdxLXB 2 = RefIdxLX[ xNbB 2 ][ 2 ] (522)
[0701] predFlagLXB 2 = PredFlagLX[ xNbB 2 ][ 2 ] (523)
[0702] hpelIfIdxB 2 = HpelIfIdx[ xNbB 2 ][ 2 ] (524)
[0703] bxf 2 = BcwIdx[ xNbB 2 ][ 2 ] (525)
[0704] In the example exposed above, the following variable definitions are used:
[0705] When all values of luma_weight_l0_flag[i] and chroma_weight_l0_flag[i] are set to 0, the variable WPDisabled0[i] is set to 1, where i=0..NumRefIdxActive[0]. Otherwise, the value of WPDisabled0[i] is set to 0.
[0706] When all values of luma_weight_l1_flag[i] and chroma_weight_l1_flag[i] are set to 0, the variable WPDisabled1[i] is set to 1, where i=0..NumRefIdxActive[1]. Otherwise, the value of WPDisabled1[1] is set to 0.
[0707] In another example, a variable SliceMaxNumTriangleMergeCand is defined at the slice header according to one of the following conditions:
[0708] –SliceMaxNumTriangleMergeCand=(lumaWeightedFlag||chromaWeightedFlag)? 0:
[0709] MaxNumTriangleMergeCand;
[0710] –SliceMaxNumTriangleMergeCand=(lumaWeightedFlag||chromaWeightedFlag)? 1:
[0711] MaxNumTriangleMergeCand;
[0712] –SliceMaxNumTriangleMergeCand=slice_weighted_pred_flag? 0:MaxNumTriangleMergeCand;
[0713] or
[0714] –SliceMaxNumTriangleMergeCand=slice_weighted_pred_flag? 1:MaxNumTriangleMergeCand.
[0715] The value of SliceMaxNumTriangleMergeCand is further used to resolve the merge information at the block level. The following table gives an exemplary syntax.
[0716]
[0717]
[0718] When the non-rectangular inter prediction mode is the GEO mode, the following example is further described.
[0719] Different mechanisms can be used to control the GEO / TPM fusion mode, depending on whether WP should be used for the reference picture from which reference blocks P0 and P1 are obtained by:
[0720] – Move the WP parameters listed in Table 14 from SH to PH;
[0721] – Move GEO parameters from PH back to SH;
[0722] – Change the semantics of MaxNumGeoMergeCand, e.g., set MaxNumGeoMergeCand to 0 or 1 for those slices when a WP-passed reference picture is available (e.g., at least one flag lumaWeightedFlag is true).
[0723] For GEO fusion mode, the exemplary reference blocks P0 and P1 are Figure 8 They are represented as 810 and 820 respectively.
[0724] In one example, when the enablement of non-rectangular modes (e.g., GEO and TPM) and the WP parameter are indicated in the image header, the following syntax may be used, as shown in the following table:
[0725] Table: Image header RBSP syntax
[0726]
[0727]
[0728] …
[0729] The variable WPDisabled is set to 1 when all values of luma_weight_l0_flag[i], chroma_weight_l0_flag[i], luma_weight_l1_flag[j] and chroma_weight_l1_flag[j] are set to 0, where i=0..NumRefIdxActive[0], j=0..NumRefIdxActive[1]; otherwise, the value of WPDisabled is set to 0.
[0730] When the variable WPDisabled is set to 0, the value of pic_max_num_merge_cand_minus_max_num_geo_cand is set to MaxNumMergeCand.
[0731] In another example, pic_max_num_merge_cand_minus_max_num_geo_cand is set to MaxNumMergeCand–1.
[0732] In one example, the indication of the WP parameters and the enabling of non-rectangular modes (eg, GEO and TPM) are performed in the stripe header. The following table gives an exemplary syntax.
[0733]
[0734]
[0735]
[0736]
[0737]
[0738] The variable WPDisabled is set to 1 when all values of luma_weight_l0_flag[i], chroma_weight_l0_flag[i], luma_weight_l1_flag[j] and chroma_weight_l1_flag[j] are set to 0, where i=0..NumRefIdxActive[0], j=0..NumRefIdxActive[1]; otherwise, the value of WPDisabled is set to 0.
[0739] When the variable WPDisabled is set to 0, the value of max_num_merge_cand_minus_max_num_geo_cand is set to MaxNumMergeCand.
[0740] In another embodiment, when the variable WPDisabled is set to 0, the value of max_num_merge_cand_minus_max_num_geo_cand is set to MaxNumMergeCand−1.
[0741] In the examples disclosed above, the weighted prediction parameters may be indicated in a picture header or a slice header.
[0742] In other embodiments, a variable SliceMaxNumGeoMergeCand is defined at the slice header according to one of the following conditions:
[0743] –SliceMaxNumGeoMergeCand=(lumaWeightedFlag||chromaWeightedFlag)? 0:
[0744] MaxNumGeoMergeCand;
[0745] –SliceMaxNumGeoMergeCand=(lumaWeightedFlag||chromaWeightedFlag)? 1:
[0746] MaxNumGeoMergeCand;
[0747] –SliceMaxNumGeoMergeCand=slice_weighted_pred_flag? 0:MaxNumGeoMergeCand;
[0748] Or SliceMaxNumGeoMergeCand=slice_weighted_pred_flag? 1:MaxNumGeoMergeCand.
[0749] Different embodiments use different scenarios listed above.
[0750] The value of the variable SliceMaxNumGeoMergeCand is further used to resolve the merge information at the block level. The following table gives an exemplary syntax.
[0751] 7.3.9.7 Fusion Data Syntax
[0752]
[0753]
[0754] The relevant image header semantics are as follows:
[0755] pic_max_num_merge_cand_minus_max_num_geo_cand indicates the maximum number of geo merge mode candidates supported in the slice associated with the picture header minus MaxNumMergeCand.
[0756] When pic_max_num_merge_cand_minus_max_num_geo_cand is not present, sps_geo_enabled_flag is equal to 1, and MaxNumMergeCand is greater than or equal to 2, pic_max_num_merge_cand_minus_max_num_geo_cand is inferred to be pps_max_num_merge_cand_minus_max_num_geo_cand_plus1–1.
[0757] The maximum number of geo fusion mode candidates MaxNumGeoMeoCand is derived as follows:
[0758] MaxNumGeoMergeCand=MaxNumMergeCand–pic_max_num_merge_cand_minus_max_num_geo_cand
[0759] When pic_max_num_merge_cand_minus_max_num_geo_cand exists, the value range of MaxNumGeoMergeCand should be 2 to MaxNumMergeCand (including the end values).
[0760] When pic_max_num_merge_cand_minus_max_num_geo_cand does not exist (sps_geo_enabled_flag is equal to 0 or MaxNumMergeCand is less than 2), MaxNumGeoMergeCand is set to 0.
[0761] When MaxNumGeoMergeCand is equal to 0, geo merge mode is not allowed for the strips associated with PH.
[0762] In the following examples, several aspects related to indication are considered. These aspects are described below:
[0763] – The syntax element ( ) related to the number of fusion mode candidates is indicated in the sequence parameters set (SPS),
[0764] This allows a specific implementation to derive the number of non-rectangular pattern fusion candidates (MaxNumGeoMergeCand) at the SPS level;
[0765] – When the picture consists of only one strip, PH may be indicated in SH;
[0766] – Define a PH / SH parameter coverage mechanism, including the following:
[0767] The PPS flag is used to indicate whether the syntax element of the relevant coding tool exists in the PH or in the SH (but can exist in both at the same time).
[0768] Specifically, reference picture lists and weighted prediction tables can use this mechanism.
[0769] – Prediction weight table, which is the fifth type of data (such as ALF, deblocking filter, RPL and SAO) that can be indicated in PH or SH;
[0770] – When weighted prediction is enabled for a picture, all slices in the picture need to have the same reference picture list;
[0771] – If only some slice types are used in the picture associated with the PH, inter- and intra-related syntax elements are indicated under certain conditions.
[0772] Specifically, two flags are introduced, namely pic_inter_slice_present_flag and pic_intra_slice_present_flag.
[0773] In one example, a syntax element () related to the number of fusion mode candidates is indicated in a sequence parameter set (SPS), which enables a specific implementation to derive the number of non-rectangular mode fusion candidates (MaxNumGeoMergeCand) at the SPS level. This aspect can be implemented by an encoding process or a decoding process according to the following syntax.
[0774] 7.3.2.3 Sequence parameter set RBSP syntax
[0775]
[0776]
[0777] The syntax described above has the following semantics.
[0778] sps_six_minus_max_num_merge_cand_plus1 equal to 0 indicates that pic_six_minus_max_num_merge_cand exists in the PH of the reference PPS. sps_six_minus_max_num_merge_cand_plus1 greater than 0 indicates that pic_six_minus_max_num_merge_cand does not exist in the PH of the reference PPS. The value range of sps_six_minus_max_num_merge_cand_plus1 should be 0 to 6 (inclusive).
[0779] sps_max_num_merge_cand_minus_max_num_geo_cand_plus1 equal to 0 indicates that pic_max_num_merge_cand_minus_max_num_geo_cand exists in the PH of the slice that references the PPS. sps_max_num_merge_cand_minus_max_num_geo_cand_plus1 greater than 0 indicates that pic_max_num_merge_cand_minus_max_num_geo_cand does not exist in the PH that references the PPS. The value range of sps_max_num_merge_cand_minus_max_num_geo_cand_plus1 shall be 0 to MaxNumMergeCand–1.
[0780] The semantics of the corresponding elements of PH are as follows:
[0781] pic_six_minus_max_num_merge_cand represents the maximum number of merged motion vector prediction (MVP) candidates supported in the slice associated with PH minus 6. The maximum number of merged MVP candidates MaxNumMergeCand is derived as follows:
[0782] MaxNumMergeCand=6–pic_six_minus_max_num_merge_cand
[0783] MaxNumMergeCand shall be in the range 1 to 6, inclusive. When pic_six_minus_max_num_merge_cand is not present, the value of pic_six_minus_max_num_merge_cand is inferred to be sps_six_minus_max_num_merge_cand_plus1–1.
[0784] pic_max_num_merge_cand_minus_max_num_geo_cand indicates the maximum number of geo merge mode candidates supported in the slice associated with the picture header minus MaxNumMergeCand.
[0785] When sps_max_num_merge_cand_minus_max_num_geo_cand is not present, sps_geo_enabled_flag is equal to 1, and MaxNumMergeCand is greater than or equal to 2, pic_max_num_merge_cand_minus_max_num_geo_cand is inferred as sps_max_num_merge_cand_minus_max_num_geo_cand_plus1–1.
[0786] The maximum number of geo fusion mode candidates MaxNumGeoMeoCand is derived as follows:
[0787] MaxNumGeoMergeCand=MaxNumMergeCand–pic_max_num_merge_cand_minus_max_num_geo_cand
[0788] When pic_max_num_merge_cand_minus_max_num_geo_cand exists, the value range of MaxNumGeoMergeCand should be 2 to MaxNumMergeCand (including the end values).
[0789] When pic_max_num_merge_cand_minus_max_num_geo_cand does not exist (sps_geo_enabled_flag is equal to 0 or MaxNumMergeCand is less than 2), MaxNumGeoMergeCand is set to 0.
[0790] When MaxNumGeoMergeCand is equal to 0, geo merge mode is not allowed for the strips associated with PH.
[0791] Optionally, max_num_merge_cand_minus_max_num_geo_cand indicates that the maximum number of GEO merge mode candidates supported in the SPS is subtracted from MaxNumMergeCand.
[0792] When sps_geo_enabled_flag is equal to 1 and MaxNumMergeCand is greater than or equal to 3, the maximum number of GEO fusion mode candidates MaxNumGeoMergeCand is derived as follows:
[0793] MaxNumGeoMergeCand=MaxNumMergeCand–max_num_merge_cand_minus_max_num_geo_cand
[0794] If the value of sps_geo_enabled_flag is equal to 1, the value range of MaxNumGeoMergeCand should be 2 to MaxNumMergeCand (including the end value).
[0795] If sps_geo_enabled_flag is equal to 1 and MaxNumMergeCand is equal to 2, MaxNumGeoMergeCand is set to 2.
[0796] Otherwise, MaxNumGeoMergeCand is set to 0.
[0797] The optional syntax and semantics of this example are as follows:
[0798]
[0799] sps_six_minus_max_num_merge_cand represents the maximum number of merged motion vector prediction (MVP) candidates supported in the slice associated with the PH minus 6. The maximum number of merged MVP candidates MaxNumMergeCand is derived as follows:
[0800] MaxNumMergeCand=6–sps_six_minus_max_num_merge_cand
[0801] The value range of MaxNumMergeCand should be 1 to 6 (including the end value).
[0802] sps_six_minus_max_num_merge_cand indicates the maximum number of geo merge mode candidates supported in the strip associated with the picture header minus MaxNumMergeCand.
[0803] The maximum number of geo fusion mode candidates MaxNumGeoMeoCand is derived as follows:
[0804] MaxNumGeoMergeCand=MaxNumMergeCand–sps_max_num_merge_cand_minus_max_num_geo_cand
[0805] When sps_max_num_merge_cand_minus_max_num_geo_cand exists, the value range of MaxNumGeoMergeCand should be 2 to MaxNumMergeCand (including the end values).
[0806] When sps_max_num_merge_cand_minus_max_num_geo_cand does not exist (sps_geo_enabled_flag is equal to 0 or MaxNumMergeCand is less than 2), MaxNumGeoMergeCand is set to 0.
[0807] When MaxNumGeoMergeCand is equal to 0, geo merging mode is not used.
[0808] For the above example and the two optional syntax definitions, a check is made to see if weighted prediction is enabled. This check affects the derivation of the variable MaxNumGeoMergeCand, and the value of MaxNumGeoMergeCand is set to 0 in one of the following cases:
[0809] – when i=0..NumRefIdxActive[0], j=0..NumRefIdxActive[1], luma_weight_l0_flag[i], chroma_weight_l0_flag[i], luma_weight_l1_flag[j] and chroma_weight_l1_flag[j] are set to 0 or do not exist;
[0810] – A flag in the SPS or PPS indicates the presence of bidirectional weighted prediction (pps_weighted_bipred_flag);
[0811] – Indicates the presence of bidirectional weighted prediction in the picture header (PH) or slice header (SH).
[0812] The SPS level flag indicating the presence of weighted prediction parameters may be as follows:
[0813]
[0814]
[0815] The syntax element "sps_wp_enabled_flag" determines whether weighted prediction can be enabled on a low level (PPS, PH or SH). An exemplary implementation is given below.
[0816]
[0817] In the above table, pps_weighted_pred_flag and pps_weighted_bipred_flag are flags in the bitstream, indicating whether weighted prediction is enabled for unidirectional prediction blocks and bidirectional prediction blocks.
[0818] In one example, when the weighted prediction flags are specified in the picture header as pic_weighted_pred_flag and pic_weighted_bipred_flag, etc., the following dependency on sps_wp_enabled_flag can be specified in the codestream syntax:
[0819]
[0820] In one example, if the weighted prediction flags are specified in the slice header as weighted_pred_flag and weighted_bipred_flag, etc., then the following dependency on sps_wp_enabled_flag can be specified in the codestream syntax:
[0821]
[0822]
[0823] In one example, the reference picture list may be indicated in the PPS or the PH or the SH (but not both at the same time). In some examples, the indication of the reference picture list relies on syntax elements indicating the presence of weighted prediction (e.g., pps_weighted_pred_flag and pps_weighted_bipred_flag). Therefore, depending on whether the reference picture list is indicated in the PPS, PH, or SH, the weighted prediction parameters are indicated before indicating the reference picture list in the PPS, PH, or SH, respectively.
[0824] For this example, the following syntax may be specified:
[0825] Image Parameter Set Syntax
[0826]
[0827] …
[0828] rpl_present_in_ph_flag equal to 1 indicates that the reference picture list indication is not present in the slice header of the referenced PPS, but may be present in the PH of the referenced PPS. rpl_present_in_ph_flag equal to 0 indicates that the reference picture list indication is not present in the PH of the referenced PPS, but may be present in the slice header of the referenced PPS.
[0829] sao_present_in_ph_flag equal to 1 indicates that the syntax elements for enabling SAO usage are not present in the slice header of the referenced PPS, but may be present in the PH of the referenced PPS. sao_present_in_ph_flag equal to 0 indicates that the syntax elements for enabling SAO usage are not present in the PH of the referenced PPS, but may be present in the slice header of the referenced PPS.
[0830] alf_present_in_ph_flag equal to 1 indicates that the syntax elements for enabling ALF usage are not present in the slice header of the referenced PPS, but may be present in the PH of the referenced PPS. alf_present_in_ph_flag equal to 0 indicates that the syntax elements for enabling ALF usage are not present in the PH of the referenced PPS, but may be present in the slice header of the referenced PPS.
[0831] …
[0832] weighted_pred_table_present_in_ph_flag equal to 1 indicates that the weighted prediction table is not present in the slice header of the referenced PPS, but may be present in the PH of the referenced PPS. weighted_pred_table_present_in_ph_flag equal to 0 indicates that the weighted prediction table is not present in the PH of the referenced PPS, but may be present in the slice header of the referenced PPS. When weighted_pred_table_present_in_ph_flag is not present, the value of weighted_pred_table_present_in_ph_flag is inferred to be 0.
[0833] …
[0834] deblocking_filter_override_enabled_flag equal to 1 indicates that a deblocking filter override may be present in the PH or slice header of the referenced PPS. deblocking_filter_override_enabled_flag equal to 0 indicates that a deblocking filter override is not present in the PH or slice of the referenced PPS. When deblocking_filter_override_enabled_flag is not present, the value of deblocking_filter_override_enabled_flag is inferred to be 0.
[0835] deblocking_filter_override_present_in_ph_flag equal to 1 indicates that the deblocking filter override is not present in the slice header of the referenced PPS, but may be present in the PH of the referenced PPS. deblocking_filter_override_present_in_ph_flag equal to 0 indicates that the deblocking filter override is not present in the PH of the referenced PPS, but may be present in the slice header of the referenced PPS.
[0836] …
[0837]
[0838]
[0839]
[0840]
[0841]
[0842]
[0843] An alternative syntax for the image header is as follows:
[0844]
[0845]
[0846]
[0847] In another example, indications of picture header and slice header elements may be combined in a single process.
[0848] This example introduces a flag (picture_header_in_slice_header_flag) that indicates whether the picture header and slice header are combined. According to this example, the syntax of the codestream is as follows:
[0849] Image header RBSP syntax
[0850]
[0851] Image header structure syntax
[0852]
[0853] Common Strip Header Syntax
[0854]
[0855]
[0856] The semantics of picture_header_in_slice_header_flag and the associated codestream constraints are as follows:
[0857] picture_header_in_slice_header_flag equal to 1 indicates that the picture header syntax structure is present in the slice header. picture_header_in_slice_header_flag equal to 0 indicates that the picture header syntax structure is not present in the slice header.
[0858] The requirement for codestream consistency is that the value of picture_header_in_slice_header_flag is the same for all slices in CLVS.
[0859] When picture_header_in_slice_header_flag is equal to 1, the code stream consistency requirement is: there is no NAL unit with NAL unit type PH_NUT in CLVS.
[0860] When picture_header_in_slice_header_flag is equal to 0, the code stream consistency requirement is: there is a NAL unit with a NAL unit type of PH_NUT in the PU, and the NAL unit is before the first VCL NAL unit of the PU.
[0861] Various aspects of these examples are combined below.
[0862] When picture_header_in_slice_header_flag is equal to 0, these flags indicate whether the syntax elements of the associated coding tool are present in the PH or SH (but not in both);
[0863] Otherwise (when picture_header_in_slice_header_flag is equal to 1), these flags are inferred to be 0, indicating slice-level tool parameter indication.
[0864] An optional combination is as follows:
[0865] When picture_header_in_slice_header_flag is equal to 0, these flags indicate whether the syntax elements of the associated coding tool are present in the PH or SH (but not in both);
[0866] Otherwise (when picture_header_in_slice_header_flag is equal to 1), these flags are inferred to be 0, indicating tool parameter indication at the picture header level.
[0867] This combination has the following syntax:
[0868] Image Parameter Set Syntax
[0869]
[0870]
[0871] In this example, it is checked whether weighted prediction is enabled by indicating the number of entries in the reference picture list that are referenced by weighted prediction. The syntax and semantics in this example are defined as follows:
[0872]
[0873]
[0874] …
[0875] num_l0_weighted_ref_pics indicates the number of weighted reference pictures in reference picture list 0. The value range of num_l0_weighted_ref_pics should be 0 to MaxDecPicBuffMinus1+14 (including the end value).
[0876] The codestream conformance requirement is that, when num_l0_weighted_ref_pics is present, the value of num_l0_weighted_ref_pics shall not be less than the number of active reference pictures for L0 of any slice in the picture associated with the picture header.
[0877] num_l1_weighted_ref_pics indicates the number of weighted reference pictures in reference picture list 1. The value range of num_l1_weighted_ref_pics should be 0 to MaxDecPicBuffMinus1+14 (including the end value).
[0878] The codestream conformance requirement is that, when present, the value of num_l1_weighted_ref_pics shall not be less than the number of active reference pictures of L1 for any slice in the picture associated with the picture header.
[0879] …
[0880] When num_l0_weighted_ref_pics or num_l1_weighted_ref_pics is non-zero, MaxNumGeoMergeCand is set to 0. The following syntax is an example of how to take advantage of this dependency:
[0881]
[0882] The semantics of pic_max_num_merge_cand_minus_max_num_geo_cand in this embodiment is the same as that in the above embodiment.
[0883] In one example, if only certain slice types are used in a picture associated with a PH, inter- and intra-related syntax elements are indicated under certain conditions.
[0884] The syntax for this example is given below.
[0885]
[0886]
[0887]
[0888]
[0889]
[0890] 7.3.7.1 General Strip Header Syntax
[0891]
[0892] 7.4.3.6 Image Header RBSP Semantics
[0893] pic_inter_slice_present_flag equal to 1 indicates that one or more slices with slice_type equal to 0 (B) or 1 (P) may be present in the picture associated with the PH. pic_inter_slice_present_flag equal to 0 indicates that slices with slice_type equal to 0 (B) or 1 (P) may not be present in the picture associated with the PH.
[0894] pic_intra_slice_present_flag equal to 1 indicates that one or more slices with slice_type equal to 2 (I) may be present in the picture associated with the PH. pic_intra_slice_present_flag equal to 0 indicates that slices with slice_type equal to 2 (I) may not be present in the picture associated with the PH. When pic_intra_slice_only_flag is not present, the value of pic_intra_slice_only_flag is inferred to be 1.
[0895] NOTE: The values of pic_inter_slice_present_flag and pic_intra_slice_present_flag are both set to 1 in a picture header associated with a picture that includes one or more sub-pictures that include one or more intra-coded slices and that may be merged with one or more sub-pictures that include one or more inter-coded slices.
[0896] 7.4.8.1 General Strip Header Syntax
[0897] slice_type specifies the encoding type of the slice, as shown in Table 7-5.
[0898] Table 7-5: Association between name and slice_type
[0899] slice_type The name of slice_type 0 B (B band) 1 P(P stripe) 2 I(I strip)
[0900] When the value of nal_unit_type ranges from IDR_W_RADL to CRA_NUT (including the end value) and the current picture is the first picture in the access unit, slice_type should be equal to 2.
[0901] When slice_type is not present, the value of slice_type is inferred to be 2.
[0902] When pic_intra_slice_present_flag is equal to 0, the value range of slice_type should be 0 to 1 (including the end value).
[0903] This example can be combined with the indication of pred_weight_table() in the image header. The above example discloses the indication of pred_weight_table() in the image header.
[0904] An example syntax is as follows:
[0905]
[0906] When indicating that pred_weight_table() is present in the image header, the following syntax may be used.
[0907]
[0908] Optional examples can use the following syntax:
[0909]
[0910]
[0911] Optional examples can use the following syntax:
[0912]
[0913] In the above syntax, pic_inter_bipred_slice_present_flag indicates that all slice types of the reference picture header are present: I slice, B slice, and P slice.
[0914] When pic_inter_bipred_slice_present_flag is 0, the picture includes only I-type slices or B-type slices.
[0915] In this case, non-rectangular mode is not used.
[0916] In one example, a combination of the above examples is disclosed. An exemplary syntax is as follows:
[0917]
[0918]
[0919]
[0920] In one example, a non-rectangular (eg, GEO) mode of a reference image is selected without using a weighted predictor.
[0921] In this example, the semantics are defined as follows:
[0922] 7.4.10.7 Fusion Data Semantics
[0923] …
[0924] The variable MergeGeoFlag[x0][y0] indicates whether triangle-based motion compensation is used to generate prediction samples for the current coding unit when decoding a B slice. The derivation of this variable is as follows:
[0925] –MergeGeoFlag[x0][y0] is set to 1 if all of the following conditions are true:
[0926] –sps_geo_enabled_flag is equal to 1,
[0927] –slice_type is B,
[0928] –general_merge_flag[x0][y0] is equal to 1,
[0929] –MaxNumGeoMergeCand is greater than or equal to 2,
[0930] –cbWidth is greater than or equal to 8,
[0931] –cbHeight is greater than or equal to 8,
[0932] –cbWidth is less than 8*cbHeight,
[0933] –cbHeight is less than 8*cbWidth,
[0934] –regular_merge_flag[x0][y0] is equal to 0,
[0935] –merge_subblock_flag[x0][y0] is equal to 0,
[0936] –ciip_flag[x0][y0] is equal to 0;
[0937] – Otherwise, MergeGeoFlag[x0][y0] is set to 0.
[0938] The requirement for code stream consistency is: if one of the CU's luma explicit weighting flag or chroma explicit weighting flag is true, then MergeGeoFlag[x0][y0] should be equal to 0.
[0939] In one example, a portion of the VVC specification explains:
[0940] 8.5.7 Decoding process of GEO inter-frame blocks
[0941] 8.5.7.1 Overview
[0942] This process is called when decoding a coding unit with MergeGeoFlag[xCb][yCb] equal to 1.
[0943] Inputs to this process include:
[0944] – Luma position (xCb, yCb), which represents the top left sample of the current coding block relative to the top left luminance sample of the current image;
[0945] – The variable cbWidth indicates the width of the current coding block in units of luma samples;
[0946] – The variable cbHeight indicates the height of the current coding block in units of luma samples;
[0947] – Luma motion vectors mvA and mvB with 1 / 16 fractional sample accuracy;
[0948] – Chroma motion vectors mvCA and mvCB;
[0949] – reference indices refIdxA and refIdxB;
[0950] –Prediction list flags predListFlagA and predListFlagB.
[0951] …
[0952] Assume predSamplesLA L and predSamplesLB L predSamplesLA is a (cbWidth)×(cbHeight) array of predicted brightness sample values. Cb 、predSamplesLB Cb 、predSamplesLACr and predSamplesLB Cr It is a (cbWidth / SubWidthC)×(cbHeight / SubHeightC) array of predicted chroma sample values.
[0953] predSamples L 、predSamples Cb and predSamples Cr Derived by following the steps in order:
[0954] 1. When N is each of A and B, the following applies:
[0955] …
[0956] 2. The partition angle and distance variables of the merged geo mode angleIdx and distanceIdx are set according to the value of merge_geo_partition_idx[xCb][yCb] detailed in Table 36.
[0957] 3. The derivation of the variable explicitWeightedFlag is as follows:
[0958] lumaWeightedFlagA=predListFlagA? luma_weight_l1_flag[refIdxA]:luma_weight_l0_flag[refIdxA]
[0959] lumaWeightedFlagB=predListFlagB? luma_weight_l1_flag[refIdxB]:luma_weight_l0_flag[refIdxB]
[0960] chromaWeightedFlagA=predListFlagA? chroma_weight_l1_flag[refIdxA]:chroma weight_l0_flag[refIdxA]
[0961] chromaWeightedFlagB=predListFlagB? chroma_weight_l1_flag[refIdxB]:chroma weight_l0_flag[refIdxB]
[0962] weightedFlag=lumaWeightedFlagA||lumaWeightedFlagB||chromaWeightedFlagA||chromaWeightedFlagB
[0963] 4. Prediction samples predSamples in the current brightness coding block L [x L ][y L ](where x L =0..cbWidth–1,y L =0..cbHeight
[0964] –1) is derived in the following way: if weightedFlag is equal to 0, the weighted sample prediction process of geo fusion mode detailed in Section 8.5.7.2 is called; if weightedFlag is equal to 1, the explicit weighted sample prediction process detailed in Section 8.5.6.6.3 is called, where the input includes the coding block width nCbW set to cbWidth, the coding block height nCbH set to cbHeight, and the sample array predSamplesLA L and predSamplesLB L , variables angleIdx and distanceIdx and cIdx set to 0.
[0965] 5. Prediction samples predSamples in the current chrominance component Cb coding block Cb [x C ][y C ](where x C =0..cbWidth / SubWidthC
[0966] –1,y C =0..cbHeight / SubHeightC–1) is derived as follows: If weightedFlag is equal to 0, then call
[0967] The weighted sample prediction process for geo fusion mode detailed in 8.5.7.2; if weightedFlag is equal to 1, the explicit weighted sample prediction process detailed in Section 8.5.6.6.3 is called, where the input includes the coding block width nCbW set to cbWidth / SubWidthC, the coding block height nCbH set to cbHeight / SubHeightC, and the sample array predSamplesLA Cb and predSamplesLB Cb, variables angleIdx and distanceIdx and cIdx set to 1.
[0968] 6. Prediction samples predSamples in the current chrominance component Cr coding block Cr [x C ][y C ](where x C =0..cbWidth / SubWidthC
[0969] –1,y C =0..cbHeight / SubHeightC–1) is derived as follows: If weightedFlag is equal to 0, then call
[0970] The weighted sample prediction process for geo fusion mode detailed in 8.5.7.2; if weightedFlag is equal to 1, the explicit weighted sample prediction process detailed in Section 8.5.6.6.3 is called, where the input includes the coding block width nCbW set to cbWidth / SubWidthC, the coding block height nCbH set to cbHeight / SubHeightC, and the sample array predSamplesLA Cr and predSamplesLB Cr , variables angleIdx and distanceIdx and cIdx set to 2.
[0971] 7. Call the motion vector storage process of the fused geo mode detailed in Section 8.5.7.3, where the input includes the luma coding block position (xCb, yCb), luma coding block width cbWidth, luma coding block height cbHeight, division directions angleIdx and distanceIdx, luma motion vectors mvA and mvB, reference indices refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB.
[0972] Table 36: Details of angleIdx and distanceIdx values based on merge_geo_partition_idx values
[0973]
[0974] 8.5.6.6.3 Explicit Weighted Sample Prediction Process
[0975] Inputs to this process include:
[0976] – Two variables nCbW and nCbH, representing the width and height of the current coding block;
[0977] – two (nCbW) × (nCbH) arrays predSamplesL0 and predSamplesL1;
[0978] – The prediction list uses flags predFlagL0 and predFlagL1;
[0979] – reference indices refIdxL0 and refIdxL1;
[0980] – Variable cIdx, indicating the color component index;
[0981] – Sample bit depth bitDepth.
[0982] The output of this process is a (nCbW)×(nCbH) array pbSamples of prediction sample values.
[0983] The variable shift1 is set to Max(2,14 – bitDepth).
[0984] The variables log2Wd, o0, o1, w0, and w1 are derived as follows:
[0985] – If the luma sample’s cIdx is equal to 0, the following applies:
[0986] log2Wd = luma_log2_weight_denom + shift1 (1010)
[0987] w0 = LumaWeightL0[ refIdxL0 ] (1011)
[0988] w1 = LumaWeightL1[refIdxL1] (1012)
[0989] o0 = luma_offset_l0[ refIdxL0 ] << (bitDepth – 8) (1013)
[0990] o1 = luma_offset_l1[ refIdxL1 ] << (bitDepth – 8) (1014)
[0991] – Otherwise (cIdx of the chroma sample is not equal to 0), the following applies:
[0992] log2Wd = ChromaLog2WeightDenom + shift1 (1015)
[0993] w0 = ChromaWeightL0[ refIdxL0 ][ cIdx – 1 ] (1016)
[0994] w1 = ChromaWeightL1[ refIdxL1 ][ cIdx – 1 ] (1017)
[0995] o0 = ChromaOffsetL0[ refIdxL0 ][ cIdx – 1 ] << (bitDepth – 8) (1018)
[0996] o1 = ChromaOffsetL1[ refIdxL1 ][ cIdx – 1 ] <<(bitDepth – 8 ) (1019)
[0997] The prediction samples pbSamples[x][y] (where x = 0..nCbW–1, y = 0..nCbH–1) are derived as follows:
[0998] – If predFlagL0 is equal to 1 and predFlagL1 is equal to 0, the predicted sample value is derived as follows:
[0999] if(log2Wd>=1)
[1000] pbSamples[x][y]=Clip3(0,(1< <bitDepth)–1,
[1001] ( ( predSamplesL0[ x ][ y ] * w0 + 2 log2Wd – 1 ) >> log2Wd ) + o0 )(1020)
[1002] else
[1003] pbSamples[x][y]=Clip3(0,(1< <bitDepth)–1,predSamplesL0[x][y]*w0+o0)
[1004] – If predFlagL0 is equal to 0 and predFlagL1 is equal to 1, the predicted sample value is derived as follows:
[1005] if(log2Wd>=1)
[1006] pbSamples[x][y]=Clip3(0,(1< <bitDepth)–1,
[1007] ( ( predSamplesL1[ x ][ y ] * w1 + 2 log2Wd – 1 ) >> log2Wd ) + o1 )(1021)
[1008] else
[1009] pbSamples[x][y]=Clip3(0,(1< <bitDepth)–1,predSamplesL1[x][y]*w1+o1)
[1010] – Otherwise (predFlagL0 is equal to 1, predFlagL1 is equal to 1), the predicted sample value is derived as follows:
[1011] pbSamples[x][y]=Clip3(0,(1< <bitDepth)–1,
[1012] (predSamplesL0[x][y]*w0+predSamplesL1[x][y]*w1+
[1013] ( ( o0 + o1 + 1 ) << log2Wd ) ) >> ( log2Wd + 1 ) ) (1022)
[1014] In this example, the syntax of the fused data parameter is disclosed, which includes a check variable indicating the presence of a non-rectangular fused mode (eg, GEO mode). An example of the syntax is given below.
[1015]
[1016]
[1017] The variable MaxNumGeoMergeCand is derived from any of the above examples.
[1018] An optional variable SliceMaxNumGeoMergeCand derived from the variable MaxNumGeoMergeCand may be used. The value of MaxNumGeoMergeCand is obtained at a higher indication level such as, for example, PH, PPS or SPS.
[1019] In one example, SliceMaxNumGeoMergeCand is derived from the value of MaxNumGeoMergeCand and other checks performed on the slice.
[1020] For example, SliceMaxNumGeoMergeCand=(num_l0_weighted_ref_pics>0||num_l1_weighted_ref_pics>0)? 0:MaxNumGeoMergeCand.
[1021] In another example, the following expression is used to determine the MaxNumGeoMergeCand value:
[1022] SliceMaxNumGeoMergeCand=(!pic_inter_slice_present_flag)? 0:MaxNumGeoMergeCand.
[1023] In one example,
[1024] The following syntax tables are defined:
[1025]
[1026] The variable MaxNumGeoMergeCand is derived as follows:
[1027] SliceMaxNumGeoMergeCand=(!pic_inter_bipred_slice_present_flag)? 0:MaxNumGeoMergeCand.
[1028] The present invention discloses a method for indicating the number of fusion candidates of rectangular mode and non-rectangular mode. The number of fusion candidates of rectangular mode and non-rectangular mode are related. If the number of fusion candidates indicating the rectangular mode is less than a threshold, it may not be necessary to indicate the number of fusion candidates of the non-rectangular mode.
[1029] Especially for TPM fusion mode or Geo fusion mode, since a block predicted using any non-rectangular fusion mode requires two inter prediction values with different specified MVs, there should be at least two candidates for the fusion mode. In one embodiment, when the number of fusion mode candidates is indicated in a sequence parameter set (SPS), the following syntax can be used:
[1030] 7.3.2.3 Sequence parameter set RBSP syntax
[1031]
[1032]
[1033]
[1034]
[1035]
[1036]
[1037]
[1038] According to one embodiment of the present invention, the following steps are performed to indicate the number of fusion mode candidates in the SPS:
[1039] – Indicates the number of merging mode candidates for normal mode (MaxNumMergeCand);
[1040] – Indicates whether non-rectangular mode is enabled via the non-rectangular fusion enable flag (sps_geo_enabled_fla);
[1041] – When the non-rectangular fusion enabling flag value is non-zero and the number of fusion mode candidates of the conventional fusion mode is greater than the first threshold, it indicates that the non-rectangular fusion enabling flag value is non-zero and the number of fusion mode candidates of the conventional fusion mode is greater than the first threshold.
[1042] The number of geo modes (sps_max_num_merge_cand_minus_max_num_geo_cand).
[1043] When the number of fusion mode candidates of the normal mode is greater than a second threshold (eg, 1), a non-rectangular fusion enabling flag is indicated.
[1044] In Embodiment 1, the order of steps is as shown in the following part of the SPS syntax of the VVC specification:
[1045]
[1046]
[1047] In this embodiment, two sequential checks are performed, the second check depending on the value of a flag that indicates or does not indicate depending on the result of the first check.
[1048] Compared to the process described in Example 1, Example 2 performs the second check in a different way. Specifically, Example 1 uses the condition "greater than" instead of "greater than or equal to". The order of steps is shown in the following part of the SPS syntax of the VVC specification:
[1049]
[1050] Embodiment 3 differs from Embodiment 1 in that when the result of the first check is a false value, the second check is not performed, and since the value of sps_geo_enabled_flag does not refer to some value of MaxNumMergeCand, the non-rectangular fusion enabling flag value (sps_geo_enabled_flag) is determined after the process of deriving the MaxNumMergeCand value according to the syntax element sps_six_minus_max_num_merge_cand is completed, which is a technical advantage, so the processing of the parsing process is skipped. The order of steps performed according to Embodiment 3 is shown in the following part of the SPS syntax of the VVC specification:
[1051]
[1052] Embodiment 4 is a combination of various aspects of Embodiment 2 and Embodiment 3. The order of steps performed according to Embodiment 4 is shown in the following part of the SPS syntax of the VVC specification:
[1053]
[1054]
[1055] Examples 5 to 8 disclose different expressions of the first inspection and the second inspection. These examples can be explained as follows:
[1056] Example 5
[1057]
[1058] Example 6
[1059]
[1060] Example 7
[1061]
[1062] Example 8
[1063]
[1064]
[1065] exist Fig.15 In one implementation shown, a method for obtaining a maximum number of geometric partition fusion mode candidates for video decoding is disclosed. The method includes:
[1066] S1501: Obtain the bitstream of the video sequence.
[1067] The bit stream can be obtained from a wireless network or a wired network. The bit stream can be sent from a website, a server or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, microwave, Wi-Fi, Bluetooth, LTE or 5G.
[1068] In one embodiment, the code stream is a bit sequence, which is in the form of a network abstraction layer (NAL) unit stream or a byte stream and constitutes a representation of a series of access units (AU), and the series of access units constitute one or more coded video sequences (CVS).
[1069] In some embodiments, for the decoding process, the decoding end reads the bitstream and derives the decoded image from the bitstream; for the encoding process, the encoding end generates the bitstream.
[1070] Typically, a codestream includes syntax elements consisting of syntax structures. Syntax element: An element of data represented in a codestream.
[1071] Syntax structure: 0 or more syntax elements appear together in the codestream in a specified order.
[1072] In a specific example, the code stream format represents the relationship between a network abstraction layer (NAL) unit stream and a byte stream, and the network abstraction layer unit stream or the byte stream is called a code stream.
[1073] Codestreams can have two formats: NAL unit stream format and byte stream format. The NAL unit stream format is conceptually the more "basic" type. The NAL unit stream format includes a sequence of syntax structures called NAL units. This sequence is sorted in decoding order. Some constraints are applied to the decoding order (and content) of NAL units in a NAL unit stream.
[1074] The NAL units are sorted in decoding order, and a start code prefix and 0 or more zero-valued bytes are added before each NAL unit to form a byte stream, thereby constructing a byte stream format according to the NAL unit stream format. The position of the unique start code prefix pattern is searched in the byte stream, thereby extracting the NAL unit stream format from the byte stream format.
[1075] This section details the relationship between the source images given by the codestream and the decoded images.
[1076] The video source represented by the bitstream is a sequence of images arranged in decoding order.
[1077] The source image and the decoded image each include one or more sample arrays:
[1078] – an array of only luminance (Y) (monochrome) samples;
[1079] – One luma sample array and two chroma sample arrays (YCbCr or YCgCo);
[1080] – Green-Blue-Red (GBR, also called RGB) sample array;
[1081] – represents an array of other unspecified monochrome or tristimulus color samples (e.g., YZX, also called XYZ).
[1082] The variables and terms associated with these arrays are called luminance (or L or Y) and chrominance, where the two chrominance arrays are called Cb and Cr, regardless of the actual color representation method used. The actual color representation method used can be indicated in the syntax specified in the VUI parameters detailed in ITU-TH.SEI|ISO / IEC 23002-7.
[1083] S1502: Obtain a value of a first indicator according to the code stream.
[1084] The first indicator represents a maximum number of fused motion vector prediction (MVP) candidates.
[1085] In one example, the first indicator is represented by a variable MaxNumMergeCand.
[1086] In one example, the maximum number of merged MVP candidates MaxNumMergeCand is derived as follows:
[1087] MaxNumMergeCand=6–sps_six_minus_max_num_merge_cand.
[1088] sps_six_minus_max_num_merge_cand indicates that the maximum number of merged motion vector prediction (MVP) candidates supported in the SPS is subtracted from 6. The value range of sps_six_minus_max_num_merge_cand should be 0 to 5 (including the end value).
[1089] In an example, sps_six_minus_max_num_merge_cand is parsed from the sequence parameter set RBSP syntax structure in the codestream.
[1090] S1503: Obtain a value of a second indicator according to the code stream.
[1091] The second indicator represents whether geometric partitioning-based motion compensation is enabled for the video sequence.
[1092] In one example, the second indicator is represented by sps_geo_enabled_flag (sps_gpm_enabled_flag). sps_geo_enabled_flag equal to 1 indicates that motion compensation based on geometric partitioning is enabled for CLVS, and merge_gpm_partition_idx, merge_gpm_idx0, and merge_gpm_idx1 may be present in the coding unit syntax of CLVS. sps_geo_enabled_flag equal to 0 indicates that motion compensation based on geometric partitioning is disabled for CLVS, and merge_gpm_partition_idx, merge_gpm_idx0, and merge_gpm_idx1 are not present in the coding unit syntax of CLVS. When sps_geo_enabled_flag is not present, the value of sps_geo_enabled_flag is inferred to be 0.
[1093] In one implementation, the step of obtaining the value of the second indicator is performed after the step of obtaining the value of the first indicator.
[1094] In one implementation, the value of the second indicator is obtained from a sequence parameter set (SPS) of the code stream.
[1095] In one implementation, when the value of the first indicator is greater than or equal to a threshold, the value of the second indicator is parsed from a sequence parameter set (SPS) of the code stream. The threshold is an integer value, and in one example, the threshold is 2.
[1096] For example, the value of the second indicator sps_gpm_enabled_flag is based on
[1097] The sequence parameter set is obtained using the RBSP syntax.
[1098]
[1099] S1504: Parse the value of the third indicator from the code stream.
[1100] In one implementation, when the value of the first indicator is greater than a threshold and the value of the second indicator is equal to a preset value, the value of the third indicator is parsed from the code stream, wherein the third indicator represents the maximum number of geometric partitioning fusion mode candidates subtracted from the value of the first indicator.
[1101] The threshold is an integer value, and the preset value is an integer value. In one example, the threshold is 2.
[1102] In one example, the threshold is 1.
[1103] In an example, the value of the third indicator is obtained from a sequence parameter set (SPS) of the code stream.
[1104] In one example, the third indicator is represented by sps_max_num_merge_cand_minus_max_num_geo_cand (sps_max_num_merge_cand_minus_max_num_gpm_cand).
[1105] For example, the value of the third indicator sps_max_num_merge_cand_minus_max_num_gpm_cand is obtained according to the sequence parameter set RBSP syntax.
[1106]
[1107] In one implementation, the method further includes: when the value of the first indicator is equal to the threshold and the value of the second indicator is equal to the preset value, setting the value of the maximum number of the geometric partitioning fusion mode candidates to 2.
[1108] In one implementation, the method further includes: when the value of the first indicator is less than the threshold or the value of the second indicator is not equal to the preset value, setting the value of the maximum number of geometric partitioning fusion mode candidates to 0.
[1109] In one example, sps_max_num_merge_cand_minus_max_num_gpm_cand indicates that the maximum number of partition merge mode candidates supported in the SPS is subtracted from MaxNumMergeCand. The value range of sps_max_num_merge_cand_minus_max_num_gpm_cand should be 0 to MaxNumMergeCand-2 (including the end value).
[1110] The maximum number of geometric partition fusion mode candidates MaxNumGpmMergeCand (MaxNumGeoMergeCand) is derived as follows:
[1111]
[1112]
[1113] exist Fig.16 In the implementation shown, a video decoding device 1600 is disclosed. The video decoding device includes: a receiving module 1601, used to obtain a code stream of a video sequence; an acquisition module 1602, used to obtain a value of a first indicator according to the code stream, wherein the first indicator indicates a maximum number of fused motion vector prediction (motion vector prediction, MVP) candidates; the acquisition module 1602 is used to obtain a value of a second indicator according to the code stream, wherein the second indicator indicates whether geometric partition-based motion compensation is enabled for the video sequence; and a parsing module 1603, used to: when the value of the first indicator is greater than a threshold value and the value of the second indicator is equal to a preset value, parse the value of a third indicator from the code stream, wherein the third indicator indicates the maximum number of geometric partition fusion mode candidates subtracted from the value of the first indicator.
[1114] In one implementation, the acquisition module 1602 is used to: when the value of the first indicator is equal to the threshold and the value of the second indicator is equal to the preset value, set the value of the maximum number of the geometric partitioning fusion mode candidates to 2.
[1115] In one implementation, the acquisition module 1602 is used to: when the value of the first indicator is less than the threshold or the value of the second indicator is not equal to the preset value, set the value of the maximum number of geometric partitioning fusion mode candidates to 0.
[1116] In one implementation, the threshold is 2.
[1117] In one implementation, the threshold is 1.
[1118] In one implementation, the step of obtaining the value of the second indicator is performed after the step of obtaining the value of the first indicator.
[1119] In one implementation, when the value of the first indicator is greater than or equal to the threshold, the value of the second indicator is parsed from a sequence parameter set (SPS) of the code stream.
[1120] In one implementation, the value of the second indicator is obtained from a sequence parameter set (SPS) of the code stream.
[1121] In one implementation, the value of the third indicator is obtained from a sequence parameter set (SPS) of the code stream.
[1122] For more details of the receiving module 1601 , the obtaining module 1602 and the parsing module 1603 , reference may be made to the above method examples and implementations.
[1123] Example 1. The video encoding method includes indicating the number of fusion mode candidates, wherein the method includes:
[1124] – Indicates the number of merging mode candidates for normal mode (MaxNumMergeCand);
[1125] – Indicates whether non-rectangular mode is enabled via the non-rectangular fusion enable flag (sps_geo_enabled_flag);
[1126] – When the non-rectangular merging enable flag value is non-zero and the number of merging mode candidates of the normal merging mode is greater than the first threshold, the number of non-rectangular modes (sps_max_num_merge_cand_minus_max_num_geo_cand) is indicated,
[1127] When the number of fusion mode candidates of the conventional mode is greater than a second threshold value (1), the non-rectangular fusion enabling flag is indicated.
[1128] Example 2. The method of Example 1, wherein the non-rectangular merge enable flag value is determined after a process of deriving a MaxNumMergeCand value from a syntax element sps_six_minus_max_num_merge_cand is completed.
[1129] Example 3. The method according to any of the above examples, wherein the threshold check is a comparison of whether the number of fusion mode candidates of the regular fusion mode is greater than 2.
[1130] Example 4. The method according to Example 1 or Example 2, wherein the check of the first threshold is a comparison of whether the number of fusion mode candidates of the regular fusion mode is greater than or equal to 3.
[1131] In one example, an inter-frame prediction method is disclosed, including: determining whether a non-rectangular inter-frame prediction mode is allowed to be used for a block group; obtaining one or more inter-frame prediction mode parameters and weighted prediction parameters of the block group; and obtaining a prediction value of a current block based on the one or more inter-frame prediction mode parameters and weighted prediction parameters, wherein one inter-frame prediction mode parameter of the one or more inter-frame prediction mode parameters represents reference image information of the current block, and the block group includes the current block.
[1132] In one example, the reference picture information includes whether weighted prediction is enabled for the reference picture index; if weighted prediction is enabled, the non-rectangular inter prediction mode is disabled.
[1133] In a possible implementation, if weighted prediction is disabled, the non-rectangular inter prediction mode is enabled.
[1134] In one example, the determining whether to allow the non-rectangular inter prediction mode includes indicating that a maximum number of triangle merge candidates (MaxNumTriangleMergeCand) is greater than 1.
[1135] In one example, the block group includes a picture, and the weighted prediction parameter and indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a picture header of the picture.
[1136] In an example, the block group includes a slice, and the weighted prediction parameter and indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a slice header of the slice.
[1137] In one example, the non-rectangular inter-frame prediction mode is a triangulation mode.
[1138] In one example, the non-rectangular inter prediction mode is a geographic (GEO) partitioning mode.
[1139] In one example, the weighted prediction parameters are used for slice-level brightness compensation.
[1140] In one example, the weighted prediction parameters are used for block level brightness compensation.
[1141] In one example, the weighted prediction parameters include: a flag indicating whether the weighted prediction is applied to the luminance component and / or chrominance component of the prediction block; and a linear model parameter specifying a linear transformation of the value of the prediction block.
[1142] In one example, an apparatus for inter-frame prediction is disclosed, comprising: a non-transitory memory storing processor-executable instructions; and a processor coupled to the memory and configured to execute the processor-executable instructions to implement any of the above method examples.
[1143] In one example, a code stream for inter-frame prediction is disclosed, including: indication information for determining whether a non-rectangular inter-frame prediction mode is allowed to be used for a block group; one or more inter-frame prediction mode parameters and weighted prediction parameters of the block group, wherein a prediction value of a current block is obtained based on the one or more inter-frame prediction mode parameters and weighted prediction parameters, one of the one or more inter-frame prediction mode parameters represents reference image information of the current block, and the block group includes the current block.
[1144] In one example, the reference picture information includes whether weighted prediction is enabled for the reference picture index; if weighted prediction is enabled, the non-rectangular inter prediction mode is disabled.
[1145] In one example, if weighted prediction is disabled, the non-rectangular inter prediction mode is enabled.
[1146] In an example, the indication information includes that the maximum number of triangle fusion candidates (MaxNumTriangleMergeCand) is greater than 1.
[1147] In one example, the block group includes a picture, and the weighted prediction parameter and the indication information are in a picture header of the picture.
[1148] In an example, the block group consists of slices, and the weighted prediction parameter and the indication information are in a slice header of the slice.
[1149] In one example, the non-rectangular inter-frame prediction mode is a triangulation mode.
[1150] In one example, the non-rectangular inter prediction mode is a geographic (GEO) partitioning mode.
[1151] In one example, the weighted prediction parameters are used for slice-level brightness compensation.
[1152] In one example, the weighted prediction parameters are used for block level brightness compensation.
[1153] In one example, the weighted prediction parameters include: a flag indicating whether the weighted prediction is applied to the luminance component and / or chrominance component of the prediction block; and a linear model parameter specifying a linear transformation of the value of the prediction block.
[1154] In one example, an inter-frame prediction device is disclosed, including: a determination module, which determines whether a non-rectangular inter-frame prediction mode is allowed to be used for a block group; an acquisition module, which is used to obtain one or more inter-frame prediction mode parameters and weighted prediction parameters of the block group; a prediction module, which is used to obtain a prediction value of a current block based on the one or more inter-frame prediction mode parameters and weighted prediction parameters, wherein one inter-frame prediction mode parameter of the one or more inter-frame prediction mode parameters represents reference image information of the current block, and the block group includes the current block.
[1155] In one example, the reference picture information includes whether weighted prediction is enabled for the reference picture index; if weighted prediction is enabled, the non-rectangular inter prediction mode is disabled.
[1156] In one example, if weighted prediction is disabled, the non-rectangular inter prediction mode is enabled.
[1157] In an example, the determination module is specifically configured to: indicate that the maximum number of triangle fusion candidates (MaxNumTriangleMergeCand) is greater than 1.
[1158] In one example, the block group includes a picture, and the weighted prediction parameter and indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a picture header of the picture.
[1159] In an example, the block group includes a slice, and the weighted prediction parameter and indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a slice header of the slice.
[1160] In one example, the non-rectangular inter-frame prediction mode is a triangulation mode.
[1161] In one example, the non-rectangular inter prediction mode is a geographic (GEO) partitioning mode.
[1162] In one example, the weighted prediction parameters are used for slice-level brightness compensation.
[1163] In one example, the weighted prediction parameters are used for block level brightness compensation.
[1164] In one example, the weighted prediction parameters include: a flag indicating whether the weighted prediction is applied to the luminance component and / or chrominance component of the prediction block; and a linear model parameter specifying a linear transformation of the value of the prediction block.
[1165] In the embodiment of the present invention, only signal-related information in a slice header of a slice that allows or enables bidirectional inter-frame prediction (eg, a bidirectional (B) prediction slice (also called a B slice)) may be used for efficient encoding and / or decoding.
[1166] The following explains the application of the encoding method and the decoding method shown in the above embodiments and the system using these applications.
[1167] Fig.10 31 is a block diagram of a content supply system 3100 for implementing a content distribution service. The content supply system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the above-mentioned communication channel 13. The communication link 3104 includes, but is not limited to, Wi-Fi, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.
[1168] Capture device 3102 generates data, and can encode data by the coding method shown in the above-mentioned embodiment.Alternatively, capture device 3102 can distribute data to streaming media server (not shown in the figure), and the server encodes data and sends encoded data to terminal device 3106.Capture device 3102 includes but is not limited to video camera, smart phone or tablet computer, computer or notebook computer, video conferencing system, PDA, vehicle-mounted device, or any combination thereof, etc.For example, capture device 3102 can include source device 12 as described above.When data includes video, the video encoder 20 included in capture device 3102 can actually perform video encoding process.When data includes audio (i.e. sound), the video encoder included in capture device 3102 can actually perform audio encoding process.For some actual scenes, capture device 3102 distributes encoded video data and encoded audio data by multiplexing encoded video data and encoded audio data together.For other actual scenes, for example, in a video conferencing system, encoded audio data and encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106 respectively.
[1169] In the content supply system 3100, the terminal device 310 receives and regenerates the encoded data. The terminal device 3106 may be a device with data receiving and recovery capabilities, such as a smart phone or tablet computer 3108, a computer or laptop computer 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a set top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, a vehicle-mounted device 3124, or any combination of the above devices capable of decoding the above-mentioned encoded data. For example, the terminal device 3106 may include the destination device 14 described above. When the encoded data includes video, the video decoder 30 included in the terminal device preferentially performs video decoding. When the encoded data includes audio, the audio decoder included in the terminal device preferentially performs audio decoding processing.
[1170] For a terminal device with a display, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a personal digital assistant (PDA) 3122, or a vehicle-mounted device 3124, the terminal device can feed the decoded data to its display. For a terminal device without a display, such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, an external display 3126 is connected therein to receive and display the decoded data.
[1171] When each device in the system performs encoding or decoding, the image encoding device or the image decoding device as shown in the above-mentioned embodiments can be used.
[1172] Fig.1131 is a schematic diagram of an exemplary structure of the terminal device 3106. After the terminal device 3106 receives the stream from the capture device 3102, the protocol processing unit 3202 analyzes the transmission protocol of the stream. The protocol includes but is not limited to Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG-DASH, Real-time Transport Protocol (RTP), Real-time Messaging Protocol (RTMP), or any combination thereof.
[1173] After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can divide the multiplexed data into encoded audio data and encoded video data. As mentioned above, for other actual scenes, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. In this case, the encoded data is sent to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.
[1174] Through the demultiplexing process, a video elementary stream (ES), an audio ES, and optional subtitles are generated. The video decoder 3206, including the video decoder 30 described in the above embodiment, decodes the video ES to generate video frames through the decoding method shown in the above embodiment, and feeds this data to the synchronization unit 3212. The audio decoder 3208 decodes the audio ES to generate audio frames, and feeds this data to the synchronization unit 3212. Optionally, the video frames can be stored in a buffer ( Fig.11 Similarly, the audio frames may be stored in a buffer (not shown) before being fed to the synchronization unit 3212. Fig.11 (not shown).
[1175] The synchronization unit 3212 synchronizes the video frames and the audio frames and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of the video and audio information. The information may be coded in syntax using timestamps associated with the presentation of the decoded audio and visual data and timestamps associated with the transmission of the data stream itself.
[1176] If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles, synchronizes the subtitles with the video frames and the audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 3216.
[1177] The present invention is not limited to the above-mentioned system, and the image encoding device or the image decoding device in the above-mentioned embodiments may be included in other systems such as automobile systems.
[1178] Mathematical operators
[1179] The mathematical operators used in this application are similar to those used in the C programming language. However, this application accurately defines the results of integer division and arithmetic shift operations, and also defines other operations, such as power operations and real-valued division. Numbering and counting specifications usually start from 0, for example, "first" is equivalent to the 0th, "second" is equivalent to the 1st, and so on.
[1180] Arithmetic operators
[1181] The arithmetic operators are defined as follows:
[1182]
[1183] Logical operators
[1184] The logical operators are defined as follows:
[1185] x&&y Boolean logic "AND" operation of x and y
[1186] x||y Boolean logical "OR" operation of x and y
[1187] ! Boolean logic "not" operation
[1188] x? y:z If x is true or not equal to 0, find the value of y, otherwise find the value of z.
[1189] Relational operators
[1190] The relational operators are defined as follows:
[1191] > Greater than
[1192] >= greater than or equal to
[1193] < less than
[1194] <= less than or equal to
[1195] == is equal to
[1196] ! = Not equal to
[1197] When a relational operator is applied to a syntax element or variable that has been assigned the value "na" (not applicable), the value "na" is treated as a distinct value for that syntax element or variable. The value "na" is considered not equal to any other value.
[1198] Bitwise operators
[1199] The definitions of bitwise operators are as follows:
[1200] & Bitwise AND operation. When operating on integer arguments, it operates on the two's complement representation of the integer values. When operating on binary arguments, if it contains fewer bits than the other argument, the shorter argument is extended by adding more significant bits equal to 0.
[1201] | Bitwise OR operation. When operating on integer arguments, it operates on the two's complement representation of the integer values. When operating on binary arguments, if it contains fewer bits than the other argument, the shorter argument is extended by adding more significant bits equal to 0.
[1202] ^ Bitwise XOR operation. When operating on integer arguments, it operates on the two's complement representation of the integer values. When operating on binary arguments, if it contains fewer bits than the other argument, the shorter argument is extended by adding more significant bits equal to 0.
[1203] x >> y shifts x arithmetically to the right by y binary bits in its two's complement integer representation. This function is defined only when y is a non-negative integer value. The result of the right shift is that the bit shifted into the most significant bit (MSB) is equal to the MSB of x before the shift operation.
[1204] x << y shifts x arithmetically to the left by y binary bits in its two's complement integer representation. This function is defined only when y is a non-negative integer value. The result of the left shift is that the bit shifted into the least significant bit (LSB) is equal to 0.
[1205] Assignment operators
[1206] The definitions of arithmetic operators are as follows:
[1207] = Assignment operator
[1208] ++ Increment, i.e., x++ is equivalent to x = x + 1; when used as an array subscript, the value of the variable is evaluated before the increment operation.
[1209] –– Decrement, i.e., x–– is equivalent to x = x - 1; when used as an array subscript, the value of the variable is evaluated before the decrement operation.
[1210] += adds the specified value, that is, x+=3 is equivalent to x=x+3, and x+=(–3) is equivalent to x=x+(–3).
[1211] –= decrements the specified value, that is, x–=3 is equivalent to x=x–3, and x–=(–3) is equivalent to x=x–(–3).
[1212] Range Notation
[1213] The following notation is used to indicate the range of values:
[1214] x=y..zx takes integer values from y to z (including y and z), where x, y and z are integers and z is greater than y.
[1215] Mathematical functions
[1216] The mathematical function is defined as follows:
[1217]
[1218] Asin(x) trigonometric inverse sine function, operates on parameter x, x is between –1.0 and 1.0 (inclusive), and the output value is –π
[1219] The range is from ÷2 to π÷2 (inclusive), and the unit is radians.
[1220] Atan(x) is the trigonometric inverse tangent function that operates on the parameter x and produces an output value in the range of –π÷2 to π÷2 (inclusive) in radians.
[1221]
[1222] Ceil(x) is the smallest integer greater than or equal to x.
[1223] Clip1 Y (x)=Clip3(0,(1< <BitDepth Y )–1,x)
[1224] Clip1 C (x)=Clip3(0,(1< <BitDepth C )–1,x)
[1225]
[1226] Cos(x) is the trigonometric cosine function, which operates on the parameter x in radians.
[1227] Floor(x) is the largest integer less than or equal to x.
[1228]
[1229] Ln(x) returns the natural logarithm of x (logarithm to base e, where e is the base constant 2.718281828...). Log2(x) returns the base 2 logarithm of x.
[1230] Log10(x) is the base 10 logarithm of x.
[1231]
[1232]
[1233] Round(x)=Sign(x)*Floor(Abs(x)+0.5)
[1234]
[1235] Sin(x) represents the trigonometric sine function, which operates on the parameter x in radians.
[1236]
[1237] Swap(x,y)=(y,x)
[1238] Tan(x) represents the trigonometric tangent function, which operates on the parameter x in radians.
[1239] Operation priority order
[1240] When parentheses are not used to explicitly indicate the order of precedence in an expression, the following rules apply:
[1241] – Operations with higher priority are evaluated before any operations with lower priority.
[1242] – Operations of the same priority are evaluated from left to right.
[1243] The following table describes the priority of operations from highest to lowest. The higher the position in the table, the higher the priority.
[1244] For operators that are also used in the C programming language, the operator precedence order in this specification is the same as the precedence order in the C programming language.
[1245] Table: Operation priority is sorted from highest (top of table) to lowest (bottom of table)
[1246]
[1247] Text description of the logical operation
[1248] In the text, the following logical operation statements are described in mathematical form:
[1249] if(condition 0)
[1250] statement 0
[1251] else if(condition 1)
[1252] statement 1 ...
[1254] else / *informative remark on remaining condition* /
[1255] statement
[1256] It can be described in the following way:
[1257] ...as follows / ...as follows:
[1258] – If condition 0 then statement 0
[1259] – Else if condition 1 then statement 1
[1260] –……
[1261] – Otherwise (informative explanation of the remaining conditions), then statement n
[1262] Each "if ... else, if ... else, ..." statement in the text is introduced by "... as follows" or "... as follows" and is immediately followed by "if ...". The last condition of "if ... else, if ... else, ..." always has an "otherwise, ..." Statements with "if ... else, if ... else" in the middle can be identified by matching "... as follows" or "... as follows" with the ending "otherwise ...".
[1263] In the text, the following logical operation statements are described in mathematical form:
[1264] if(condition 0a&&condition 0b)
[1265] statement 0
[1266] else if(condition 1a||condition 1b)
[1267] statement 1 ...
[1269] else
[1270] statement
[1271] It can be described in the following way:
[1272] ...as follows / ...as follows:
[1273] – Statement 0 if all of the following are true:
[1274] –Condition 0a
[1275] – Condition 0b
[1276] – Otherwise, if one or more of the following conditions are met, then statement 1:
[1277] –Condition 1a
[1278] – Condition 1b
[1279] –……
[1280] – Otherwise, statement n
[1281] In the text, the following logical operation statements are described in mathematical form:
[1282] if(condition 0)
[1283] statement 0
[1284] if(condition 1)
[1285] statement 1
[1286] It can be described in the following way:
[1287] When condition 0, statement 0
[1288] When condition 1, statement 1.
[1289] Embodiments of encoders 20, decoders 30, etc. and the functions described herein with reference to encoders 20, decoders 30, etc. may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, these functions may be stored as one or more instructions or codes in a computer-readable medium or sent via a communication medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media (e.g., data storage media), or any communication media that facilitates the transfer of computer programs from one place to another (e.g., according to a communication protocol). In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media or (2) communication media such as signals or carrier waves. Data storage media may be any available media accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in the present invention. A computer program product may include a computer-readable medium.
[1290] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory or any other medium that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwave are used to transmit instructions from a website, server or other remote source, then the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other transient media, but rather involve non-transient tangible storage media. Disks and optical disks used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs) and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.
[1291] Instructions can be executed by one or more processors such as one or more digital signal processors (DSP), one or more general-purpose microprocessors, one or more application specific integrated circuits (ASIC), one or more field programmable logic arrays (FPGA), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" used herein may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the various functions described herein may be provided in dedicated hardware and / or software modules for encoding and decoding, or incorporated in a combined codec. Moreover, these techniques may be fully implemented in one or more circuits or logic elements.
[1292] The techniques of the present invention may be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or a set of ICs (e.g., chipsets). The present invention describes various components, modules, or units to emphasize functional aspects of devices for performing the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units may be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by a collection of interoperable hardware units including one or more processors as described above.
[1293] Other embodiments of the present invention are provided below. It should be noted that the numbers used in the following parts do not necessarily need to be consistent with the numbers used in the above parts.
[1294] Embodiment 1: A method for inter-frame prediction of an image block, wherein indication of weighted prediction parameters and enabling of non-rectangular inter-frame prediction are performed on a group of prediction blocks, the method comprising: obtaining inter-frame prediction mode parameters of the block, wherein the obtaining comprises checking whether the non-rectangular inter-frame prediction mode is enabled for the block group, the block group comprising the prediction block; obtaining weighted prediction parameters associated with the block, inter-frame prediction mode parameters of a block relative to a reference image indicated for the block, and weighted prediction parameters specified for the block group.
[1295] Embodiment 2: The method according to embodiment 1, wherein enabling of non-rectangular inter prediction is performed by indicating a maximum number of triangle merge candidates (MaxNumTriangleMergeCand) greater than 1.
[1296] Embodiment 3: The method according to embodiment 1 or 2, wherein when the weighted prediction parameter indicates that weighted prediction is enabled for at least one reference index, non-rectangular inter prediction is inferred to be disabled.
[1297] Embodiment 4: The method according to any one of embodiments 1 to 3, wherein the block group is a picture, and the enabling of the inter-frame prediction non-rectangular mode parameters and the weighted prediction parameters are indicated in the picture header.
[1298] Embodiment 5: The method according to any one of embodiments 1 to 4, wherein the block group is a slice, and the enabling of the inter-frame prediction non-rectangular mode parameter and the weighted prediction parameters are indicated in the slice header.
[1299] Embodiment 6: A method according to any one of embodiments 1 to 5, wherein the inter-frame prediction mode parameters include a reference index for determining the reference image and motion vector information for determining the position of the reference block in the reference image.
[1300] Embodiment 7: The method according to any one of embodiments 1 to 6, wherein the non-rectangular fusion mode is a triangulation mode.
[1301] Embodiment 8: The method according to any one of embodiments 1 to 7, wherein the non-rectangular fusion mode is a GEO mode.
[1302] Embodiment 9: The method according to any one of embodiments 1 to 8, wherein the weighted prediction is a slice-level brightness compensation mechanism (eg, global weighted prediction).
[1303] Embodiment 10: The method according to any one of embodiments 1 to 9, wherein the weighted prediction is a block-level brightness compensation mechanism, such as local illumination compensation (LIC).
[1304] Embodiment 11: A method according to any one of embodiments 1 to 10, wherein the weighted prediction parameters include: a set of flags indicating whether weighted prediction is applied to the luminance component and chrominance component of the prediction block; linear model parameters α and β specifying the linear transformation of the value of the prediction block.
[1305] In the first aspect of the present application, Fig.12As shown, an inter-frame prediction method 1200 is disclosed, including: S1201: determining whether a non-rectangular inter-frame prediction mode is allowed to be used for a block group; S1202: obtaining one or more inter-frame prediction mode parameters and weighted prediction parameters of the block group; S1203: obtaining a prediction value of a current block according to the one or more inter-frame prediction mode parameters and weighted prediction parameters, wherein one inter-frame prediction mode parameter of the one or more inter-frame prediction mode parameters represents reference image information of the current block, and the block group includes the current block.
[1306] In a feasible implementation, the reference image information includes whether weighted prediction is enabled for the reference image index; if weighted prediction is enabled, the non-rectangular inter-frame prediction mode is disabled.
[1307] In a possible implementation, if weighted prediction is disabled, the non-rectangular inter prediction mode is enabled.
[1308] In a feasible implementation, the determining whether to allow the non-rectangular inter prediction mode includes: indicating that the maximum number of triangle merging candidates (MaxNumTriangleMergeCand) is greater than 1.
[1309] In a feasible implementation manner, the block group includes a picture, and the weighted prediction parameter and the indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a picture header of the picture.
[1310] In a feasible implementation manner, the block group includes a slice, and the weighted prediction parameter and the indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a slice header of the slice.
[1311] In a feasible implementation, the non-rectangular inter-frame prediction mode is a triangular partitioning mode.
[1312] In a feasible implementation, the non-rectangular inter-frame prediction mode is a geometric (GEO) partitioning mode.
[1313] In a feasible implementation, the weighted prediction parameter is used for slice-level brightness compensation.
[1314] In a feasible implementation, the weighted prediction parameter is used for block-level brightness compensation.
[1315] In a feasible implementation, the weighted prediction parameters include: a flag indicating whether the weighted prediction is applied to the luminance component and / or chrominance component of the prediction block; and a linear model parameter specifying a linear transformation of the value of the prediction block.
[1316] In a second aspect of the present application, a device 1300 for inter-frame prediction is disclosed, such as Fig.13 As shown, it includes: a non-volatile memory 1301 storing processor-executable instructions; a processor 1302 coupled to the memory 1301 and used to execute the processor-executable instructions to implement any feasible implementation method of the first aspect of the present application.
[1317] In a third aspect of the present application, a code stream for inter-frame prediction includes: indication information for determining whether a non-rectangular inter-frame prediction mode is allowed to be used for a block group; one or more inter-frame prediction mode parameters and weighted prediction parameters of the block group, wherein a prediction value of a current block is obtained based on the one or more inter-frame prediction mode parameters and weighted prediction parameters, one of the one or more inter-frame prediction mode parameters represents reference image information of the current block, and the block group includes the current block.
[1318] In a feasible implementation, the reference image information includes whether weighted prediction is enabled for the reference image index; if weighted prediction is enabled, the non-rectangular inter-frame prediction mode is disabled.
[1319] In a possible implementation, if weighted prediction is disabled, the non-rectangular inter prediction mode is enabled.
[1320] In a feasible implementation, the indication information includes that the maximum number of triangle fusion candidates (MaxNumTriangleMergeCand) is greater than 1.
[1321] In a feasible implementation manner, the block group includes an image, and the weighted prediction parameter and the indication information are in an image header of the image.
[1322] In a feasible implementation manner, the block group is composed of slices, and the weighted prediction parameter and the indication information are in a slice header of the slice.
[1323] In a feasible implementation, the non-rectangular inter-frame prediction mode is a triangular partitioning mode.
[1324] In a feasible implementation, the non-rectangular inter-frame prediction mode is a geometric (GEO) partitioning mode.
[1325] In a feasible implementation, the weighted prediction parameter is used for slice-level brightness compensation.
[1326] In a feasible implementation, the weighted prediction parameter is used for block-level brightness compensation.
[1327] In a feasible implementation, the weighted prediction parameters include: a flag indicating whether the weighted prediction is applied to the luminance component and / or chrominance component of the prediction block; and a linear model parameter specifying a linear transformation of the value of the prediction block.
[1328] In the fourth aspect of the present application, Fig.14 As shown, an inter-frame prediction device 1400 is disclosed, including: a determination module 1401, used to determine whether a non-rectangular inter-frame prediction mode is allowed to be used for a block group; an acquisition module 1402, used to acquire one or more inter-frame prediction mode parameters and weighted prediction parameters of the block group; a prediction module 1403, used to acquire a prediction value of a current block according to the one or more inter-frame prediction mode parameters and weighted prediction parameters, wherein one inter-frame prediction mode parameter of the one or more inter-frame prediction mode parameters represents reference image information of the current block, and the block group includes the current block.
[1329] In a feasible implementation, the reference image information includes whether weighted prediction is enabled for the reference image index; if weighted prediction is enabled, the non-rectangular inter-frame prediction mode is disabled.
[1330] In a possible implementation, if weighted prediction is disabled, the non-rectangular inter prediction mode is enabled.
[1331] In a feasible implementation, the determination module 1401 is specifically configured to indicate that the maximum number of triangle fusion candidates (MaxNumTriangleMergeCand) is greater than 1.
[1332] In a feasible implementation manner, the block group includes a picture, and the weighted prediction parameter and the indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a picture header of the picture.
[1333] In a feasible implementation manner, the block group includes a slice, and the weighted prediction parameter and the indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a slice header of the slice.
[1334] In a feasible implementation, the non-rectangular inter-frame prediction mode is a triangular partitioning mode.
[1335] In a feasible implementation, the non-rectangular inter-frame prediction mode is a geometric (GEO) partitioning mode.
[1336] In a feasible implementation, the weighted prediction parameter is used for slice-level brightness compensation.
[1337] In a feasible implementation, the weighted prediction parameter is used for block-level brightness compensation.
[1338] In a feasible implementation, the weighted prediction parameters include: a flag indicating whether the weighted prediction is applied to the luminance component and / or chrominance component of the prediction block; and a linear model parameter specifying a linear transformation of the value of the prediction block.
[1339] The methods provided by the prior art can be summarized as follows:
[1340] Aspect 1. An inter-frame prediction method, wherein the method comprises:
[1341] determining whether a non-rectangular inter prediction mode is allowed for a block group;
[1342] Obtaining one or more inter-frame prediction mode parameters and weighted prediction parameters of the block group;
[1343] A prediction value of the current block is obtained according to the one or more inter-frame prediction mode parameters and the weighted prediction parameters, wherein one inter-frame prediction mode parameter among the one or more inter-frame prediction mode parameters represents reference image information of the current block, and the block group includes the current block.
[1344] Aspect 2. The method according to aspect 1, wherein the reference image information includes whether weighted prediction is enabled for the reference image index; if weighted prediction is enabled, the non-rectangular inter prediction mode is disabled.
[1345] Aspect 3. The method according to aspect 1 or 2, wherein if weighted prediction is disabled, the non-rectangular inter prediction mode is enabled.
[1346] Aspect 4. The method according to any one of aspects 1 to 3, wherein the determining whether to allow a non-rectangular inter prediction mode comprises:
[1347] Indicates that the maximum number of triangle merge candidates (MaxNumTriangleMergeCand) is greater than 1.
[1348] Aspect 5. The method according to any one of aspects 1 to 4, wherein the block group includes a picture, and the weighted prediction parameter and indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a picture header of the picture.
[1349] Aspect 6. The method according to any one of aspects 1 to 4, wherein the block group includes a slice, and the weighted prediction parameter and indication information for determining whether the non-rectangular inter-frame prediction mode is allowed are in a slice header of the slice.
[1350] Aspect 7. The method according to any one of aspects 1 to 6, wherein the non-rectangular inter-frame prediction mode is a triangular partitioning mode.
[1351] Aspect 8. The method according to any one of aspects 1 to 6, wherein the non-rectangular inter prediction mode is a geometric (GEO) partitioning mode.
[1352] Aspect 8a. A method according to any one of aspects 1 to 8, wherein a syntax element related to the number of fusion mode candidates (representing information used to determine the non-rectangular inter-frame prediction) is indicated in a sequence parameter set (SPS).
[1353] Aspect 8b. The method according to any one of aspects 1 to 8a, wherein when the picture includes only one slice, the picture header is indicated in the slice header.
[1354] Clause 8c. The method according to any one of clauses 1 to 8b, wherein when the picture includes only one slice, the picture header is indicated in the slice header.
[1355] Clause 8d. A method according to any one of clauses 1 to 8c, wherein the picture parameter set comprises a flag, the value of the flag defining whether the weighted prediction parameters are present in a picture header or in a slice header.
[1356] Aspect 8e. The method according to any one of aspects 1 to 8d, wherein a flag in the picture header indicates whether a slice of non-intra type exists and whether an inter prediction mode parameter is indicated for the slice.
[1357] Aspect 9. The method according to any one of aspects 1 to 8, wherein the weighted prediction parameters are used for slice-level brightness compensation.
[1358] Aspect 10. The method according to any one of aspects 1 to 8, wherein the weighted prediction parameters are used for block-level brightness compensation.
[1359] Aspect 11. The method according to any one of aspects 1 to 10, wherein the weighted prediction parameter comprises:
[1360] A flag indicating whether weighted prediction is applied to the luminance component and / or chrominance component of the prediction block;
[1361] Linear model parameters specify the linear transformation of the values of the prediction block.
[1362] Aspect 12. A device for inter-frame prediction, wherein the device comprises:
[1363] Non-transitory memory storing processor executable instructions;
[1364] A processor, coupled to the memory and used to execute the processor-executable instructions to implement the method described in any one of aspects 1 to 11.
[1365] Aspect 13. A code stream for inter-frame prediction, wherein the code stream comprises:
[1366] Indication information for determining whether a non-rectangular inter prediction mode is allowed for a block group;
[1367] One or more inter-frame prediction mode parameters and weighted prediction parameters of the block group, wherein the prediction value of the current block is obtained based on the one or more inter-frame prediction mode parameters and weighted prediction parameters, one inter-frame prediction mode parameter of the one or more inter-frame prediction mode parameters represents reference image information of the current block, and the block group includes the current block.
[1368] Aspect 14. The code stream according to aspect 13, wherein the reference image information includes whether weighted prediction is enabled for the reference image index; if weighted prediction is enabled, the non-rectangular inter prediction mode is disabled.
[1369] Aspect 15. The code stream according to aspect 13 or 14, wherein if weighted prediction is disabled, the non-rectangular inter-frame prediction mode is enabled.
[1370] Aspect 16. The code stream according to any one of aspects 13 to 15, wherein the indication information includes that the maximum number of triangle merging candidates (MaxNumTriangleMergeCand) is greater than 1.
[1371] Aspect 17. The code stream according to any one of aspects 13 to 16, wherein the block group includes a picture, and the weighted prediction parameter and the indication information are in a picture header of the picture.
[1372] Aspect 18. The code stream according to any one of aspects 13 to 17, wherein the block group includes a slice, and the weighted prediction parameter and the indication information are in a slice header of the slice.
[1373] Aspect 19. The code stream according to any one of aspects 13 to 18, wherein the non-rectangular inter-frame prediction mode is a triangular partitioning mode.
[1374] Aspect 20. The code stream according to any one of aspects 13 to 19, wherein the non-rectangular inter-frame prediction mode is a geometric (GEO) partitioning mode.
[1375] Aspect 21. The code stream according to any one of aspects 13 to 20, wherein the weighted prediction parameters are used for slice-level brightness compensation.
[1376] Aspect 22. The code stream according to any one of Aspects 13 to 20, wherein the weighted prediction parameters are used for block-level brightness compensation.
[1377] Aspect 23. The bitstream according to any one of aspects 13 to 22, wherein the weighted prediction parameter comprises:
[1378] A flag indicating whether weighted prediction is applied to the luminance component and / or chrominance component of the prediction block;
[1379] Linear model parameters specify the linear transformation of the values of the prediction block.
Claims
1. A device for storing a code stream, characterized in that: Includes at least one storage medium and a communication interface: The communication interface is used to receive or send an encoded code stream, where the code stream is obtained by encoding the video data according to the encoding method. The encoding method comprises: Determining a value of a first indicator, wherein the first indicator represents a maximum number of fused motion vector prediction MVP candidates; determining a value of a second indicator, wherein the second indicator indicates whether geometric partitioning-based motion compensation is enabled for the video sequence; When the value of the first indicator is greater than a threshold and the value of the second indicator is equal to a preset value, the value of the third indicator is encoded into the code stream, wherein the third indicator represents the maximum number of geometric partition fusion mode candidates subtracted from the value of the first indicator.
2. The device for storing code stream according to claim 1, characterized in that: The threshold is 2.
3. The device for storing code stream according to claim 1 or 2, characterized in that: The method further comprises: When the value of the first indicator is equal to the threshold and the value of the second indicator is equal to the preset value, the value of the maximum number of the geometric partitioning fusion mode candidates is set to 2.
4. The device for storing code stream according to any one of claims 1 to 3, characterized in that: The method further comprises: When the value of the first indicator is less than the threshold or the value of the second indicator is not equal to the preset value, the value of the maximum number of geometric partitioning fusion mode candidates is set to 0.
5. The device for storing code stream according to any one of claims 1 to 4, characterized in that: The preset value is 1.
6. The device for storing code stream according to any one of claims 1 to 5, characterized in that: The step of obtaining the value of the second indicator is performed after the step of obtaining the value of the first indicator.
7. The device for storing code stream according to claim 6, characterized in that: When the value of the first indicator is greater than or equal to the threshold, the value of the second indicator is parsed from a sequence parameter set SPS of the code stream.
8. The device for storing code streams according to any one of claims 1 to 7, characterized in that: The value of the second indicator is obtained from a sequence parameter set SPS of the code stream.
9. The device for storing code streams according to any one of claims 1 to 8, characterized in that: The value of the third indicator is obtained from a sequence parameter set SPS of the code stream.
10. A method for storing a code stream, characterized in that: include: Receive a code stream of a video sequence; The codestream is stored in one or more storage media, wherein the codestream includes multiple syntax elements, wherein the multiple syntax elements include a first indicator, a second indicator and / or a third indicator, wherein the first indicator indicates a maximum number of fused motion vector prediction MVP candidates, the second indicator indicates whether geometric partition-based motion compensation is enabled for the video sequence, and the third indicator indicates the maximum number of geometric partition fusion mode candidates subtracted from the value of the first indicator.
11. A method for transmitting an encoded bitstream of video data, characterized in that: The method comprises: Acquire a code stream from a storage medium, wherein the code stream is a code stream generated by the encoding method according to claim 1 and is stored in the storage medium; The code stream is sent.
12. A method for storing an encoded bitstream of video data, characterized in that: The method comprises: Receiving the code stream generated by the encoding method according to claim 1; The code stream is stored in a storage medium.
