Method, apparatus, readable medium and computer system for video coding

Abstract

Aspects of the disclosure provide methods, apparatuses, readable medium and computer systems for video decoding. An apparatus for video decoding includes processing circuitry. The processing circuitry can decode a first syntax element from a coded video bitstream, the first syntax element being signaled in a first coding level, the first syntax element indicating a maximum number of merge motion vector predictor (MVP) candidates, the first coding level being higher than a picture parameter set (PPS) level. The processing circuitry can determine the maximum number of the merge MVP candidates based on the first syntax element. The processing circuitry can reconstruct a coding block associated with the first coding level based on at least the maximum number of the merge MVP candidates. The first coding level is a sequence level, and the first syntax element is signaled in a sequence parameter set (SPS).

Description

[0001] Introduction and use

[0002] This application claims priority to U.S. Patent Application No. 17 / 088,073, filed November 3, 2020, entitled “Method and Apparatus for Video Coding,” and U.S. Provisional Application No. 62 / 960,930, filed January 14, 2020, entitled “Signaling of Maximum Number of Merge Candidates for Interpretation.” The entire disclosure of these prior applications is incorporated herein by reference. Technical Field

[0003] This application describes embodiments that generally relate to video encoding and decoding. Background Technology

[0004] The background description provided herein is intended to present the overall context of this application. The extent of the work of the currently named inventors described in the background section and various aspects of this specification does not imply that it was prior art at the time of filing of this application, nor is it expressly or implied that it was acknowledged as prior art to this application.

[0005] Video encoding and decoding can be performed using inter-frame prediction techniques with motion compensation. Uncompressed digital video can comprise a series of images, each with spatial dimensions, for example, 1920×1080 luminance samples and correlated chrominance samples. The series of images has a fixed or variable image rate (also informally referred to as the frame rate), for example, 60 images per second or 60Hz. Uncompressed video has very high bitrate requirements. For example, a 1080p60 4:2:0 video with 8 bits per sample (1920x1080 luminance sample resolution, 60Hz frame rate) requires close to 1.5 Gbit / s of bandwidth. One hour of such video would require over 600 GB of storage space.

[0006] One goal of video encoding and decoding is to reduce redundant information in the input video signal through compression. Video compression can help reduce bandwidth or storage requirements by two or more orders of magnitude in some cases. Lossless and lossy compression, as well as combinations of both, can be used. Lossless compression refers to the technique of reconstructing an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may not be exactly the same as the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used in video. The amount of distortion tolerated depends on the application. For example, users of some consumer streaming applications may tolerate higher distortion than users of television applications. The achievable compression ratio reflects that higher allowable / tolerable distortion results in a higher compression ratio.

[0007] Motion compensation can be a lossy compression technique and can involve using sample data blocks obtained from a previously reconstructed image or a portion thereof (the reference image), spatially shifted in a direction indicated by a motion vector (hereinafter referred to as MV), to predict a newly reconstructed image or image portion. In some cases, the reference image can be the same as the image currently being reconstructed. The MV can have two dimensions: an X dimension and a Y dimension, or three dimensions, with the third dimension indicating the reference image in use (the latter can indirectly be a temporal dimension).

[0008] In some video compression techniques, a motion vector (MV) applicable to a given sample data region can be predicted from other MVs, for example, from another sample data region spatially adjacent to the region being reconstructed, whose decoding order precedes that MV. This substantially reduces the amount of data required to encode and decode the MV, thereby eliminating redundancy and enhancing compression. MV prediction can be performed efficiently, for example, because when encoding an input video signal derived from a camera (called natural video), there is statistical likelihood—that is, multiple regions larger than the area applicable to a single MV move in similar directions—so in some cases, MV prediction can be performed using similar motion vectors derived from multiple MVs of adjacent regions. This results in the MV found for a given region being similar or identical to the MV predicted from the surrounding MVs, and after entropy encoding and decoding, it can be represented with fewer bits than would be needed to directly encode and decode the MV. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself can be lossy, for example, because rounding errors occur when calculating predictors from several surrounding MVs.

[0009] H.265 / HEVC (ITU-T H.265 Recommendation, “High Efficiency Video Coding”, December 2016) describes various MV prediction mechanisms. Among the various MV prediction mechanisms provided by H.265, this application describes the technique hereinafter referred to as “spatial combining”.

[0010] Please refer to Figure 1 The current block (101) includes samples that have been discovered by the encoder during the motion search process, and these samples can be predicted based on previous blocks of the same size that have generated spatial offsets. Alternatively, the MV can be derived from metadata associated with one or more reference images, rather than being directly encoded. For example, using the MV associated with any of the five surrounding samples A0, A1 and B0, B1, B2 (corresponding to 102 to 106 respectively), the MV is derived from the metadata of the nearest reference image (in decoding order). In H.265, MV prediction can use predictions from the same reference image that is also being used in adjacent blocks. Summary of the Invention

[0011] This disclosure provides methods, apparatus, readable media, and computer systems for video decoding. In some examples, the video decoding apparatus includes processing circuitry. The processing circuitry can decode a first syntax element from an encoded video stream. The first syntax element is signaled at a first codec level. The first syntax element indicates the maximum number of merged motion vector prediction MVP candidates. The first codec level is higher than the picture parameter set (PPS) level. The processing circuitry can determine the maximum number of merged MVP candidates based on the first syntax element; the processing circuitry can reconstruct a coded block associated with the first codec level based at least on the maximum number of merged MVP candidates.

[0012] In an embodiment, the processing circuitry can decode a second, third, and fourth syntax element from the encoded video stream. The second, third, and fourth syntax elements are signaled in the first codec level. The second syntax element indicates the maximum number of geometric merge mode merge candidates. The third syntax element indicates the maximum number of sub-block-based merge MVP candidates, and the fourth syntax element indicates the maximum number of intra-block copy (IBC) merge block vector prediction (BVP) candidates. The processing circuitry can (i) determine the maximum number of geometric merge mode merge candidates based on the second syntax element, (ii) determine the maximum number of sub-block-based merge MVP candidates based on the third syntax element, and (iii) determine the maximum number of IBC merge BVP candidates based on the fourth syntax element.

[0013] In an embodiment, the first encoding / decoding level is a sequence level, and the first syntax element is signaled in the sequence parameter set (SPS).

[0014] In an embodiment, the processing circuitry can decode a second syntax element from the encoded video stream based on the condition being met. The second syntax element is signaled at a second codec level. The second syntax element indicates the maximum number of merge candidates for one of the geometric merge mode, the sub-block-based merge mode, and the intra-block copy (IBC) merge mode.

[0015] In the example, the geometric merge mode, the sub-block-based merge mode, and the IBC merge mode are all examples of the geometric merge mode. The maximum number of merge candidates is the maximum number of merge candidates for the geometric merge mode. The second syntax element indicates the maximum number of merge candidates for the geometric merge mode. The condition is that the geometric merge mode is enabled in the second codec level, and the maximum number of merge MVP candidates is greater than or equal to 3. The processing circuitry can determine the maximum number of merge candidates for the geometric merge mode based on the second syntax element.

[0016] In the example, the geometric merge mode, the sub-block-based merge mode, and the IBC merge mode are all sub-block-based merge modes. The maximum number of merge candidates is the maximum number of sub-block-based merge MVP candidates. The second syntax element indicates the maximum number of sub-block-based merge MVP candidates. The condition is that affine mode is enabled in the second codec level. The processing circuitry can determine the maximum number of sub-block-based merge MVP candidates based on the second syntax element.

[0017] In this example, the processing circuitry can modify the second syntax element at a codec level lower than the second codec level. In this example, the first and second codec levels are sequence levels, and the codec level lower than the second codec level is an image level.

[0018] In the example, the geometric merging mode, the sub-block-based merging mode, and the IBC merging mode are all IBC merging modes. The maximum number of merging candidates is the maximum number of IBC merge block vector prediction (BVP) candidates. The second syntax element indicates the maximum number of IBC merge BVP candidates. The condition is that the IBC merging mode is enabled in the second codec level. The processing circuitry can determine the maximum number of IBC merge BVP candidates based on the second syntax element.

[0019] In the example, the first and second codec levels are sequence levels.

[0020] This disclosure also provides a method for video decoding, comprising: decoding a first syntax element from an encoded video stream, the first syntax element being signaled at a first codec level, the first syntax element indicating a maximum number of merged motion vector prediction MVP candidates, the first codec level being higher than a picture parameter set (PPS) level; determining the maximum number of merged MVP candidates based on the first syntax element; and reconstructing a coded block associated with the first codec level based at least on the maximum number of merged MVP candidates.

[0021] This disclosure also provides a non-volatile computer-readable medium storing instructions that, when executed by a computer for video decoding, cause the computer to perform video decoding.

[0022] This disclosure also provides a computer system comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and operate according to the instructions of the program code to perform a method of video decoding. Attached Figure Description

[0023] Other features, properties, and various advantages of the disclosed subject matter will become further apparent from the following detailed description and accompanying drawings, wherein:

[0024] Figure 1 This is a schematic diagram of the current block and its surrounding space as merge candidates in an example.

[0025] Figure 2 This is a simplified block diagram of a communication system according to an embodiment.

[0026] Figure 3 This is a simplified block diagram of a communication system according to an embodiment.

[0027] Figure 4 This is a simplified block diagram of the decoder according to an embodiment.

[0028] Figure 5 This is a schematic diagram of a simplified block diagram of an encoder according to an embodiment.

[0029] Figure 6 A block diagram of an encoder according to another embodiment is shown.

[0030] Figure 7 A block diagram of a decoder according to another embodiment is shown.

[0031] Figure 8An exemplary geometry merging pattern is shown.

[0032] Figure 9 An exemplary syntax for sending signal notifications in the image parameter set PPS is shown.

[0033] Figure 10 An exemplary syntax for sending a signal notification in the image header is shown.

[0034] Figure 11 An exemplary syntax for signaling notifications in the sequence parameter set SPS is shown.

[0035] Figure 12 An exemplary syntax for sending signal notifications in PPS is shown.

[0036] Figure 13 An exemplary syntax for sending a signal notification in the image header is shown.

[0037] Figure 14 An exemplary syntax for sending a signal notification in the image header is shown.

[0038] Figure 15 An example syntax from the image header is shown.

[0039] Figure 16 An example syntax in SPS is shown.

[0040] Figure 17 An example syntax in SPS is shown.

[0041] Figure 18A An example syntax in SPS is shown.

[0042] Figure 18B An example syntax from the image header is shown.

[0043] Figure 19A An example syntax in SPS is shown.

[0044] Figure 19B An example syntax from the image header is shown.

[0045] Figure 20 An example syntax in SPS is shown.

[0046] Figure 21 A flowchart outlining a method according to an embodiment of this disclosure is shown.

[0047] Figure 22 This is a schematic diagram of a computer system according to an embodiment. Detailed Implementation

[0048] Figure 2This is a simplified block diagram of a communication system (200) according to an embodiment disclosed in this application. The communication system (200) includes a plurality of terminal devices that can communicate with each other via, for example, a network (250). For example, the communication system (200) includes a first terminal device (210) and a second terminal device (220) interconnected via a network (250). Figure 2 In this embodiment, the first terminal device (210) and the second terminal device (220) perform unidirectional data transmission. For example, the first terminal device (210) may encode video data (e.g., a video image stream captured by the terminal device (210)) for transmission over a network (250) to the second terminal device (220). The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device (220) may receive the encoded video data from the network (250), decode the encoded video data to recover the video data, and display video images based on the recovered video data. Unidirectional data transmission is common in applications such as media services.

[0049] In another embodiment, the communication system (200) includes a third terminal device (230) and a fourth terminal device (240) that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For bidirectional data transmission, each of the third terminal device (230) and the fourth terminal device (240) may encode video data (e.g., a stream of video images captured by the terminal device) for transmission over a network (250) to the other terminal device. Each of the third terminal device (230) and the fourth terminal device (240) may also receive encoded video data transmitted by the other terminal device and may decode the encoded video data to recover the video data, and may display the video images on an accessible display device based on the recovered video data.

[0050] exist Figure 2In the embodiments disclosed herein, the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240) may be servers, personal computers, and smartphones, but the principles disclosed herein are not limited thereto. The embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (250) refers to any number of networks that transmit encoded video data between the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240), including, for example, wired (connected) and / or wireless communication networks. The communication network (250) may exchange data in circuit-switched and / or packet-switched channels. The network may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of the network (250) may be irrelevant to the operation of this application.

[0051] As an example, Figure 3 The diagram illustrates the placement of a video encoder and a video decoder in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0052] The streaming system may include an acquisition subsystem (313) that may include a video source (301) such as a digital camera, which creates an uncompressed video image stream (302). In an embodiment, the video image stream (302) includes samples captured by a digital camera. The video image stream (302) is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data (304) (or encoded video bitstream). The video image stream (302) may be processed by an electronic device (320) that includes a video encoder (303) coupled to the video source (301). The video encoder (303) may include hardware, software, or a combination of hardware and software to implement or carry out aspects of the disclosed subject matter as described in more detail below. Compared to the video image stream (302), the encoded video data (304) (or the encoded video bitstream (304)) is depicted as a thin line to emphasize the lower data volume of the encoded video data (304) (or the encoded video bitstream (304)), which can be stored on a streaming server (305) for future use. One or more streaming client subsystems, such as Figure 3Client subsystems (306) and (308) can access a streaming server (305) to retrieve copies (307) and (309) of encoded video data (304). Client subsystem (306) may include, for example, a video decoder (310) in an electronic device (330). The video decoder (310) decodes the incoming copy (307) of the encoded video data and produces an output video picture stream (311) that can be displayed on a display (312) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (304), video data (307), and video data (309) (e.g., video streams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T H.265. In embodiments, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and this application can be used in the context of the VVC standard.

[0053] It should be noted that the electronic devices (320) and (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown), and the electronic device (330) may also include a video encoder (not shown).

[0054] Figure 4 This is a block diagram of a video decoder (410) according to an embodiment disclosed in this application. The video decoder (410) may be disposed in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., receiving circuitry). The video decoder (410) may be used in place of... Figure 3 The video decoder (310) in the embodiment.

[0055] The receiver (431) may receive one or more encoded video sequences to be decoded by the video decoder (410); in the same embodiment or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences may be received from a channel (401), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (431) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective user entities (not indicated). The receiver (431) may separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (415) may be coupled between the receiver (431) and the entropy decoder / parser (420) (hereinafter referred to as "parser (420)"). In some applications, the buffer memory (415) is part of the video decoder (410). In other cases, the buffer memory (415) may be located external to the video decoder (410) (not indicated). In other cases, an external buffer (not shown) may be provided for the video decoder (410) to prevent network jitter, for example, and another buffer (415) may be configured internally for, for example, handling broadcast timing. When the receiver (431) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, the buffer (415) may not be necessary, or it may be made smaller. Of course, for use on packet networks such as the Internet, a buffer (415) may be required; this buffer may be relatively large and adaptive in size, and may be at least partially implemented in the operating system or a similar component (not shown) external to the video decoder (410).

[0056] The video decoder (410) may include a parser (420) to reconstruct symbols (421) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (410) and potential information for controlling display devices (412), such as displays that are not part of the electronic device (430) but may be coupled to it. Figure 4As shown in the figure. The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (420) may parse / decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (420) may extract a subgroup parameter set of at least one subgroup of pixels in the subgroup of pixels in the encoded video sequence for use in the video decoder based on at least one parameter corresponding to a group. The subgroup may include Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (420) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0057] The parser (420) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415) to create symbols (421).

[0058] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (421) may involve multiple different units. Which units are involved and how they are involved can be controlled by the subgroup control information parsed by the parser (420) from the encoded video sequence. For brevity, the flow of such subgroup control information between the parser (420) and the various units described below is not described.

[0059] In addition to the functional blocks already mentioned, the video decoder (410) can be conceptually subdivided into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and can be integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0060] The first unit is the scaler / inverse transform unit (451). The scaler / inverse transform unit (451) receives quantization transform coefficients as symbols (421) and control information from the parser (420), including the transform method used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (451) can output a block containing sample values, which can be input into the aggregator (455).

[0061] In some cases, the output samples of the scaler / inverse transform unit (451) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-picture prediction unit (452). In some cases, the intra-picture prediction unit (452) uses reconstructed information extracted from the current picture buffer (458) to generate surrounding blocks of the same size and shape as the block being reconstructed. For example, the current picture buffer (458) buffers partially reconstructed and / or fully reconstructed current images. In some cases, the aggregator (455) adds the predictive information generated by the intra-picture prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451) based on each sample.

[0062] In other cases, the output samples of the scaler / inverse transform unit (451) may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit (453) can access the reference image memory (457) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (421), these samples can be added by the aggregator (455) to the output of the scaler / inverse transform unit (451) (referred to in this case as residual samples or residual signals) to generate output sample information. The motion compensation prediction unit (453) can obtain the prediction samples from the address in the reference image memory (457) under motion vector control, and the motion vector is available to the motion compensation prediction unit (453) in the form of the symbols (421), which, for example, include X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (457) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0063] The output samples of the aggregator (455) can be employed by various loop filtering techniques in the loop filter unit (456). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video stream), and these parameters can be used as symbols (421) from the parser (420) in the loop filter unit (456). However, in other embodiments, the video compression techniques may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0064] The output of the loop filter unit (456) can be a sample stream, which can be output to a display device (412) and stored in a reference image memory (457) for subsequent inter-frame image prediction.

[0065] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded image corresponding to the current image has been fully reconstructed and the encoded image (by, for example, the parser (420)) is identified as the reference image, the current image buffer (458) can become part of the reference image memory (457), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0066] The video decoder (410) can perform decoding operations according to a predetermined video compression technique, such as that specified in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the configuration file recorded in the video compression technique or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technique or standard as the only tools available under said configuration file. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference picture size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0067] In this embodiment, the receiver (431) may receive additional (redundant) data along with the encoded video. The additional data may be a portion of the encoded video sequence. The additional data may be used by the video decoder (410) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0068] Figure 5 This is a block diagram of a video encoder (503) according to an embodiment disclosed in this application. The video encoder (503) is disposed in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., a transmission circuit). The video encoder (503) can be used to replace... Figure 3 The video encoder (303) in the embodiment.

[0069] The video encoder (503) can obtain data from the video source (501) (not) Figure 5 In one embodiment, a portion of the electronic device (520) receives video samples, the video source being capable of capturing video images to be encoded by a video encoder (503). In another embodiment, the video source (501) is a portion of the electronic device (520).

[0070] A video source (501) can provide a sequence of source video samples encoded by a video encoder (503) in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (501) can be a storage device storing previously prepared video. In a video conferencing system, the video source (501) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual pictures, which are given motion when viewed in sequence. The pictures themselves can be constructed as spatial pixel arrays, where each pixel can include one or more samples depending on the sampling structure, color space, etc., used. Those skilled in the art can easily understand the relationship between pixels and samples. The following focuses on describing samples.

[0071] According to an embodiment, the video encoder (503) can encode and compress images of a source video sequence into an encoded video sequence (543) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (550). In some embodiments, the controller (550) controls and is functionally coupled to other functional units described below. For simplicity, coupling is not shown in the figures. Parameters set by the controller (550) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (550) may be used with other suitable functions related to the video encoder (503) optimized for a particular system design.

[0072] In some embodiments, the video encoder (503) operates within an encoding loop. As a simplified description, in an embodiment, the encoding loop may include a source encoder (530) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (533) embedded within the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression techniques considered in this application, any compression between the symbols and the encoded video stream is lossless). The reconstructed sample stream (sample data) is input to a reference image memory (534). Since decoding of the symbol stream produces bit-precise results independent of the decoder's location (local or remote), the contents of the reference image memory (534) are also bit-precisely corresponding between the local encoder and the remote encoder. In other words, the reference image samples "seen" by the encoder's prediction portion are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related technologies.

[0073] The operation of the “local” decoder (533) can be combined with, for example, the above-described method. Figure 4 The video decoder (410) is described in detail as the same as the "remote" decoder. However, a further brief reference is provided. Figure 4 When symbols are available and the entropy encoder (545) and parser (420) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (410), including the buffer (415) and parser (420), may not be fully implemented in the local decoder (533).

[0074] It can be observed that any decoder technique other than parsing / entropy decoding present in the decoder must also exist in the corresponding encoder in essentially the same functional form. For this reason, this application focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are inverses of the fully described decoder techniques. More detailed descriptions are only required in certain areas, and are provided below.

[0075] During operation, in some embodiments, the source encoder (530) may perform motion-compensated predictive coding. The motion-compensated predictive coding predictively encodes the input image, referencing one or more previously encoded images from the video sequence designated as "reference images." In this manner, the encoding engine (532) encodes the differences between pixel blocks of the input image and pixel blocks of the reference image, which may be selected as a predictive reference for the input image.

[0076] The local video decoder (533) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (530). The operation of the encoding engine (532) can be a lossy process. When the encoded video data can be decoded by the video decoder (533), Figure 5 When the source video sequence (not shown) is decoded, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (533) replicates the decoding process, which can be performed by the video decoder on the reference image, and allows the reconstructed reference image to be stored in a reference image cache (534). In this way, the video encoder (503) can locally store a copy of the reconstructed reference image that shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0077] The predictor (535) can perform a prediction search against the encoding engine (532). That is, for a new image to be encoded, the predictor (535) can search in the reference image memory (534) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. The predictor (535) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by the predictor (535), it can be determined that the input image may have prediction references obtained from multiple reference images stored in the reference image memory (534).

[0078] The controller (550) can manage the encoding operations of the source encoder (530), including, for example, setting parameters and subgroup parameters for encoding video data.

[0079] The outputs of all the above-mentioned functional units can be entropy encoded in the entropy encoder (545). The entropy encoder (545) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, and arithmetic coding, thereby converting the symbols into an encoded video sequence.

[0080] The transmitter (540) can buffer the encoded video sequence created by the entropy encoder (545) in preparation for transmission via a communication channel (560), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (540) can combine the encoded video data from the video encoder (503) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0081] The controller (550) manages the operation of the video encoder (503). During encoding, the controller (550) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following image types:

[0082] An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand variations of I-pictures and their corresponding applications and characteristics.

[0083] A predictive image (P-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0084] A bidirectional predictive image (B-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0085] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, determined based on the coding assignments of the corresponding images applied to the blocks. For example, blocks of an I-image can be non-predictively coded, or the blocks can be predictively coded (spatial or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be predictively coded with reference to a previously coded reference image via spatial or temporal prediction. Blocks of a B-image can be predictively coded with reference to one or two previously coded reference images via spatial or temporal prediction.

[0086] The video encoder (503) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (503) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0087] In this embodiment, the transmitter (540) may transmit additional data while transmitting encoded video. The source encoder (530) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0088] The acquired video can serve as multiple source images (video images) presented in a time series. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In an embodiment, a specific image being encoded / decoded is segmented into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when multiple reference images are used, the motion vector may have a third dimension that identifies the reference image.

[0089] In some embodiments, bidirectional prediction techniques can be used in inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly past and future in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. Specifically, the block can be predicted using a combination of the first and second reference blocks.

[0090] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0091] According to some embodiments disclosed in this application, predictions such as inter-frame image prediction and intra-frame image prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video image sequence are segmented into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Furthermore, each CTU can be further subdivided into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be subdivided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In embodiments, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Furthermore, depending on temporal and / or spatial predictability, the CU is divided into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In embodiments, prediction operations in encoding (encoding / decoding) are performed on a per-prediction-block basis. Taking a luma prediction block as an example, a prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0092] Figure 6 This is a diagram of a video encoder (603) according to another embodiment disclosed in this application. The video encoder (603) is used to receive processing blocks (e.g., prediction blocks) of sample values ​​within a current video image in a video image sequence, and to encode the processing blocks into an encoded image that is part of an encoded video sequence. In this embodiment, the video encoder (603) is used instead of Figure 3The video encoder (303) in the embodiment.

[0093] In the HEVC embodiment, the video encoder (603) receives a matrix of sample values ​​for a processing block, such as an 8×8 sample prediction block. The video encoder (603) uses, for example, rate-distortion (RD) optimization to determine whether to use an intra-frame mode, an inter-frame mode, or a bidirectional prediction mode to encode the processing block. When encoding the processing block in intra-frame mode, the video encoder (603) can use intra-frame prediction techniques to encode the processing block into an already encoded picture; and when encoding the processing block in inter-frame mode or bidirectional prediction mode, the video encoder (603) can use inter-frame prediction or bidirectional prediction techniques to encode the processing block into an already encoded picture, respectively. In some video coding techniques, the merging mode can be an inter-frame picture prediction sub-mode, in which motion vectors are derived from one or more motion vector prediction values ​​without relying on already encoded motion vector components outside the prediction values. In some other video coding techniques, motion vector components applicable to the subject block may exist. In the embodiment, the video encoder (603) includes other components, such as a mode decision module (not shown) for determining the processing block mode.

[0094] exist Figure 6 In one embodiment, the video encoder (603) includes, as shown below: Figure 6 The inter-frame encoder (630), intra-frame encoder (622), residual calculator (623), switch (626), residual encoder (624), general controller (621) and entropy encoder (625) are shown coupled together.

[0095] An inter-frame encoder (630) is configured to receive samples of the current block (e.g., the processing block), compare the block with one or more reference blocks in a reference image (e.g., blocks in previous and later images), generate inter-frame prediction information (e.g., redundancy information description, motion vectors, merging mode information based on inter-frame coding techniques), and calculate inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some embodiments, the reference image is a decoded reference image based on encoded video information.

[0096] The intra encoder (622) is used to receive samples of the current block (e.g., the processing block), in some cases compare the block with previously encoded blocks in the same image, generate quantization coefficients after transformation, and in some cases also (e.g., based on intra prediction direction information of one or more intra coding techniques) generate intra prediction information. In an embodiment, the intra encoder (622) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and reference blocks in the same image.

[0097] A general-purpose controller (621) determines general-purpose control data and controls other components of the video encoder (603) based on the general-purpose control data. In an embodiment, the general-purpose controller (621) determines the mode of a block and provides control signals to a switch (626) based on the mode. For example, when the mode is an intra-frame mode, the general-purpose controller (621) controls the switch (626) to select an intra-frame mode result for use by the residual calculator (623) and controls the entropy encoder (625) to select intra-frame prediction information and add the intra-frame prediction information to the bitstream; and when the mode is an inter-frame mode, the general-purpose controller (621) controls the switch (626) to select an inter-frame prediction result for use by the residual calculator (623) and controls the entropy encoder (625) to select inter-frame prediction information and add the inter-frame prediction information to the bitstream.

[0098] A residual calculator (623) is used to calculate the difference (residual data) between the received block and the prediction result selected from the intra encoder (622) or the inter encoder (630). A residual encoder (624) is used to operate on the residual data to encode the residual data to generate transform coefficients. In an embodiment, the residual encoder (624) is used to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is used to perform an inverse transform and generate decoded residual data. The decoded residual data can be used appropriately by the intra encoder (622) and the inter encoder (630). For example, the inter encoder (630) can generate a decoded block based on the decoded residual data and inter-frame prediction information, and the intra encoder (622) can generate a decoded block based on the decoded residual data and intra-frame prediction information. The decoded blocks are processed appropriately to generate a decoded image, and in some embodiments, the decoded image may be buffered in a memory circuit (not shown) and used as a reference image.

[0099] An entropy encoder (625) is used to format the bitstream to produce encoded blocks. The entropy encoder (625) generates various information according to a suitable standard such as the HEVC standard. In an embodiment, the entropy encoder (625) is used to obtain general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. It should be noted that, according to the disclosed subject matter, residual information is not present when blocks are encoded in a merged sub-mode of inter-frame mode or bidirectional prediction mode.

[0100] Figure 7This is a diagram of a video decoder (710) according to another embodiment disclosed in this application. The video decoder (710) is used to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed picture. In an embodiment, the video decoder (710) is used instead of Figure 3 The video decoder (310) in the embodiment.

[0101] exist Figure 7 In this embodiment, the video decoder (710) includes, as follows: Figure 7 The entropy decoder (771), inter-frame decoder (780), residual decoder (773), reconstruction module (774), and intra-frame decoder (772) are shown coupled together.

[0102] An entropy decoder (771) can be used to reconstruct certain symbols from an encoded image, these symbols representing the syntax elements constituting the encoded image. Such symbols may include, for example, a mode for encoding the block (e.g., intra-frame mode, inter-frame mode, bidirectional prediction mode, a merged sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra-frame prediction information or inter-frame prediction information) that can respectively identify certain samples or metadata used by the intra-frame decoder (772) or the inter-frame decoder (780) for prediction, residual information in the form of, for example, quantized transform coefficients, and so on. In an embodiment, when the prediction mode is inter-frame or bidirectional prediction mode, inter-frame prediction information is provided to the inter-frame decoder (780); and when the prediction type is intra-frame prediction type, intra-frame prediction information is provided to the intra-frame decoder (772). Residual information may be provided to the residual decoder (773) via inverse quantization.

[0103] The inter-frame decoder (780) is used to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0104] The intra-frame decoder (772) is used to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0105] The residual decoder (773) performs inverse quantization to extract the dequantized transform coefficients and processes the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require some control information (to obtain the quantizer parameter, QP), which can be provided by the entropy decoder (771) (the data path is not indicated because this is only low-level control information).

[0106] The reconstruction module (774) is used to combine the residual output by the residual decoder (773) with the prediction result (which may be output by the inter-frame prediction module or the intra-frame prediction module) in the spatial domain to form a reconstructed block, which may be a part of a reconstructed image, which in turn may be a part of a reconstructed video. It should be noted that other suitable operations, such as deblocking, may be performed to improve visual quality.

[0107] It should be noted that any suitable technology can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In one embodiment, one or more integrated circuits can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In another embodiment, one or more processors executing software instructions can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710).

[0108] This disclosure relates to various aspects of video coding and decoding techniques, such as signaling for the length of various merge candidate lists for inter-frame predictive coding and decoding tools. This signaling is used for tools such as regular merge modes, sub-block-based merge modes, intra-block copy (IBC) merge modes, and geometric merge modes (GEO), which are tools used in VVC, for example, outside of HEVC.

[0109] Geometric merge mode (also known as geometric partition mode) can support a variety of different partitioning methods. Figure 8 An exemplary geometric merging pattern is shown. In the geometric merging pattern, CU (800) can be partitioned into two partitions, partitions 1 and 2, which are divided by lines or edges (810). Each of the two partitions can have any suitable shape, such as a triangle, trapezoid, pentagon, etc.

[0110] Merging candidates for inter-frame prediction are effective in video codecs, such as in HEVC and VVC. In embodiments, such as in VVC, four distinct merge candidate lists (or merge lists) can exist and be constructed on both the encoder and decoder sides. The four merge candidate lists may include a merge candidate list (or first merge candidate) for a regular merge mode, a merge candidate list (or second merge candidate) for a geometric merge mode, a merge candidate list (or third merge candidate) for a sub-block-based merge mode, and a merge candidate list (or fourth merge candidate) for an IBC merge mode.

[0111] Compared to related techniques (such as some techniques in HEVC), the first merge candidate list for a regular merge mode can be expanded to include more types of merge candidates. Additionally, the maximum number of merge candidates (or merge motion vector prediction (MVP) candidates) in the first merge candidate list can be extended, for example, from 5 to 6. The maximum number of merge MVP candidates in the first merge candidate list can be indicated by a signal in the picture parameterset (PPS) at the PPS level or in the picture header at the picture level.

[0112] Figure 9 The first PPS level parameter (e.g., pps_six_minus_max_num_merge_cand_plus1) is shown in the PPS notification. The first PPS level parameter can indicate the maximum number of MVP candidates to be merged. Figure 10 The first picture-level parameter (e.g., pic_six_minus_max_num_merge_cand) is shown in the picture header. The first picture-level parameter can indicate the maximum number of MVP candidates to be merged (MaxNumMergeCand).

[0113] The first syntax element can indicate the maximum number of merged MVP candidates that are signaled at the corresponding codec level. For example, the first syntax element refers to the first PPS level parameter that is signaled at the PPS level or the first picture level parameter that is signaled at the picture level.

[0114] If the first PPS level parameter (e.g., pps_six_minus_max_num_merge_cand_plus1) is equal to 0, it indicates that the first picture level parameter (e.g., pic_six_minus_max_num_merge_cand) is present in the picture header of the reference PPS (or is signaled). If the first PPS level parameter (e.g., pps_six_minus_max_num_merge_cand_plus1) is greater than 0, it indicates that the first picture level parameter (e.g., pic_six_minus_max_num_merge_cand) is not present in the picture header of the reference PPS (or is not signaled). The value of the first PPS level parameter (e.g., pps_six_minus_max_num_merge_cand_plus1) can be in the range of 0 to 6 (inclusive). When the first PPS level parameter (e.g., pps_six_minus_max_num_merge_cand_plus1) does not exist, the first PPS level parameter (e.g., pps_six_minus_max_num_merge_cand_plus1) can be inferred to be equal to 0.

[0115] The first image-level parameter subtracted from 6 (e.g., pic_six_minus_max_num_merge_cand) specifies the maximum number (MaxNumMergeCand) of merge MVP candidates supported in a shard. This shard is associated with the image header. The maximum number of merge MVP candidates (MaxNumMergeCand) can be determined using Equation 1.

[0116] MaxNumMergeCand=6-pic_six_minus_max_num_merge_cand (Equation 1)

[0117] The value of MaxNumMergeCand can be in the range of 1 to 6 (inclusive). When the first picture level parameter is not present, the value of the first picture level parameter (e.g., pic_six_minus_max_num_merge_cand) can be inferred to be equal to the first PPS level parameter (e.g., pps_six_minus_max_num_merge_cand_plus1) - 1.

[0118] The geometry merge mode can be enabled or disabled at the sequence level using SPS syntax elements in the sequence parameter set (SPS). In the example, the SPS syntax element is the first sequence level flag (e.g., sps_geo_enabled_flag). Figure 11 The first sequence-level flag (e.g., sps_geo_enabled_flag) for signaling notifications in SPS is shown.

[0119] The first sequence-level flag (e.g., `sps_geo_enabled_flag`) specifies whether geometry-merging-based motion compensation is used for inter-frame prediction. A first sequence-level flag (e.g., `sps_geo_enabled_flag`) equal to 0 specifies that the syntax should be constrained so that geometry-merging-based motion compensation is not used in the coded layer video sequence (CLVS), and that there are no syntax elements related to geometry-merging in the CLVS codec unit syntax. A first sequence-level flag (e.g., `sps_geo_enabled_flag`) equal to 1 specifies that geometry-merging-based motion compensation is used in the CLVS.

[0120] One or more parameters can indicate the maximum number of merge candidates for the geometric merge pattern (MaxNumGeoMergeCand). These parameters are explicitly signaled at the PPS or image level.

[0121] Figure 12 An exemplary syntax for signaling notifications in PPS is shown. This syntax includes a second PPS-level parameter (e.g., pps_max_num_merge_cand_minus_max_num_geo_cand_plus1) indicating the maximum number of merge candidates for the geometric merge pattern. Figure 13 An exemplary syntax for signaling notifications in the image header is shown. This syntax includes a second image-level parameter (e.g., pic_max_num_merge_cand_minus_max_num_geo_cand) indicating the maximum number of merge candidates for the geometry merging pattern.

[0122] The second syntax element can indicate the maximum number of geometric merge mode merge candidates that are signaled at the corresponding codec level. For example, the second syntax element refers to a second PPS level parameter that is signaled at the PPS level, or a second picture level parameter that is signaled at the picture level.

[0123] A second PPS level parameter (e.g., pps_max_num_merge_cand_minus_max_num_geo_cand_plus1) equal to 0 specifies that the second picture level parameter (e.g., pic_max_num_merge_cand_minus_max_num_geo_cand) exists in the picture header of the reference PPS fragment or is signaled in the picture header of the reference PPS fragment. A second PPS level parameter (e.g., pps_max_num_merge_cand_minus_max_num_geo_cand_plus1) greater than 0 specifies that the second picture level parameter (e.g., pic_max_num_merge_cand_minus_max_num_geo_cand) does not exist in the picture header of the reference PPS. The value of the second PPS level parameter (e.g., pps_max_num_merge_cand_minus_max_num_geo_cand_plus1) can be in the range of 0 to MaxNumMergeCand-1. When the second PPS level parameter (e.g., pps_max_num_merge_cand_minus_max_num_geo_cand_plus1) does not exist, the second PPS level parameter can be inferred to be equal to 0.

[0124] The second image-level parameter (e.g., pic_max_num_merge_cand_minus_max_num_geo_cand) subtracted from the maximum number of merge MVP candidates (MaxNumMergeCand) specifies the maximum number of geometric merge pattern merge candidates supported in the slice associated with the image header.

[0125] When the second picture level parameter is not present, the first sequence level flag (e.g., sps_geo_enabled_flag) is equal to 1, and the maximum number of merged MVP candidates (MaxNumMergeCand) is greater than or equal to 2, the second picture level parameter (e.g., pic_max_num_merge_cand_minus_max_num_geo_cand) can be inferred to be equal to (second PPS level parameter - 1) (e.g., pps_max_num_merge_cand_minus_max_num_geo_cand_plus1 - 1).

[0126] The maximum number of merge candidates (MaxNumGeoMergeCand) for the geometric merge pattern can be determined based on the maximum number of merge MVP candidates (MaxNumMergeCand) and a second picture-level parameter (e.g., pic_max_num_merge_cand_minus_max_num_geo_cand), for example, using Equation 2.

[0127] MaxNumGeoMergeCand=MaxNumMergeCand-

[0128] pic_max_num_merge_cand_minus_max_num_geo_cand (Equation 2)

[0129] When the second picture level parameter (e.g., pic_max_num_merge_cand_minus_max_num_geo_cand) is present (i.e., signaled), the value of MaxNumGeoMergeCand can be in the range of 2 to MaxNumMergeCand (inclusive of end values).

[0130] The maximum number of merge candidates in the geometric merge mode (MaxNumGeoMergeCand) can be set to 0 when the second picture level parameter (e.g., pic_max_num_merge_cand_minus_max_num_geo_cand) is not present (i.e., not signaled), and one of the following conditions is met: (i) the first sequence level flag (e.g., sps_geo_enabled_flag) is equal to 0 and (ii) the maximum number of merge MVP candidates (MaxNumMergeCand) is less than 2.

[0131] When the maximum number of merge candidates for the geometric merge mode (MaxNumGeoMergeCand) is equal to 0, the geometric merge mode is not allowed to be used for fragments associated with the image header.

[0132] The second PPS level parameter (e.g., pps_max_num_merge_cand_minus_max_num_geo_cand_plus1), the second image level parameter (e.g., pic_max_num_merge_cand_minus_max_num_geo_cand), and the maximum number of merge MVP candidates (MaxNumMergeCand) can be used to determine the maximum number of merge candidates (MaxNumGeoMergeCand) for the geometric merge pattern. The maximum number of merge candidates (MaxNumGeoMergeCand) for the geometric merge pattern can be specified to not exceed the value of MaxNumMergeCand. For example, the value of MaxNumGeoMergeCand can vary depending on the specific application. When the value of MaxNumGeoMergeCand does not change frequently, PPS signaling can be effective and therefore does not need to be signaled to each image. On the other hand, when MaxNumGeoMergeCand changes frequently, for example, from one image to another, image header signaling can be effective.

[0133] Figure 14 An exemplary syntax for signaling notifications in the image header is shown. This syntax includes a third image-level parameter (e.g., pic_five_minus_max_num_subblock_merge_cand) in the image header that indicates the maximum number of subblock-based merge MVP candidates (MaxNumSubblockMergeCand) (also known as the maximum number of merge candidates for the subblock-based merge mode).

[0134] The third syntax element can indicate the maximum number of sub-block-based merged MVP candidates that are signaled at the corresponding codec level. For example, the third syntax element refers to the third picture-level parameter that is signaled at the picture level.

[0135] The third picture-level parameter (e.g., `pic_five_minus_max_num_subblock_merge_cand`) specifies the maximum number of subblock-based merge MVP candidates supported in a slice, subtracted from the value 5. When the third picture-level parameter is absent (or not signaled), its value (e.g., `pic_five_minus_max_num_subblock_merge_cand`) can be inferred to be equal to (5 - (sps_sbtmvp_enabled_flag && pic_temporal_mvp_enabled_flag)). In the example, `sps_sbtmvp_enabled_flag` is an SPS-level flag. This SPS-level flag indicates whether SBTMVP mode is enabled at the SPS level. In the example, `pic_temporal_mvp_enabled_flag` is a picture-level flag. This picture-level flag indicates whether temporal MVP mode is enabled at the picture level.

[0136] The maximum number of MVP candidates based on subblocks (MaxNumSubblockMergeCand) can be determined based on a third picture-level parameter (e.g., pic_five_minus_max_num_subblock_merge_cand) using Equation 3, for example.

[0137] MaxNumSubblockMergeCand=5-pic_five_minus_max_num_subblock_merge_cand

[0138] (Equation 3)

[0139] The maximum number of MVP candidates based on subblocks (MaxNumSubblockMergeCand) can be in the range of 0 to 5 (inclusive).

[0140] Figure 15 An exemplary syntax for signaling notifications in the image header is shown. This syntax includes a fourth image-level parameter (e.g., pic_six_minus_max_num_ibc_merge_cand), which indicates the maximum number (MaxNumIbcMergeCand) of IBC merge block vector prediction (BVP) candidates for the IBC merge mode.

[0141] The fourth syntax element can indicate the maximum number of IBC merge BVP candidates that are signaled at the corresponding codec level. For example, the fourth syntax element refers to the fourth picture-level parameter that is signaled at the picture level.

[0142] The fourth picture-level parameter (e.g., pic_six_minus_max_num_ibc_merge_cand) subtracted from the value 6 specifies the maximum number of IBC merge BVP candidates supported in the slice associated with the picture header. The maximum number of IBC merge BVP candidates (MaxNumIbcMergeCand) can be determined based on the fourth picture-level parameter (e.g., pic_six_minus_max_num_ibc_merge_cand), for example, using Equation 4.

[0143] MaxNumIbcMergeCand=6-pic_six_minus_max_num_ibc_merge_cand (Equation 4)

[0144] The maximum number of BVP candidates for IBC merging (MaxNumIbcMergeCand) can range from 1 to 6 (inclusive of end values).

[0145] Syntax elements can include a first syntax element, a second syntax element, a third syntax element, a fourth syntax element, and so on. As mentioned above, the first syntax element can indicate the maximum number of merge MVP candidates signaled at the corresponding codec level (e.g., a codec level higher than the slice level or picture level), and therefore the first syntax element can indicate the length of the merge MVP candidate list used in the regular merge mode. The merge MVP candidate list can include merge MVP candidates used in the regular merge mode. For example, the first syntax element refers to a first PPS level parameter signaled at the PPS level, or a first picture level parameter signaled at the picture level.

[0146] The second syntax element can indicate the maximum number of geometric merge mode merge candidates signaled at the corresponding codec level (e.g., a codec level higher than the slice level or picture level), and therefore can specify the length of the geometric merge mode merge candidate list used for the geometric merge mode. The geometric merge mode merge candidate list can include geometric merge mode merge candidates for the geometric merge mode. For example, the second syntax element refers to a second PPS level parameter signaled at the PPS level or a second picture level parameter signaled at the picture level.

[0147] The third syntax element can indicate the maximum number of sub-block-based merge MVP candidates signaled at the corresponding codec level (e.g., a codec level higher than the fragment level or picture level), and therefore can indicate the length of the sub-block-based merge MVP candidate list for the sub-block-based merge mode. The sub-block-based merge MVP candidate list can include sub-block-based merge MVP candidates for the sub-block-based merge mode. For example, the third syntax element refers to a third picture-level parameter signaled at the picture level.

[0148] The fourth syntax element can indicate the maximum number of IBC merge BVP candidates signaling at the corresponding codec level (e.g., a codec level higher than the fragment level or picture level), and therefore can indicate the length of the IBC merge BVP candidate list (or IBC merge candidate list) used for the IBC merge mode. The IBC merge BVP candidate list can include IBC merge BVP candidates for the IBC merge mode. For example, the fourth syntax element refers to a fourth picture level parameter signaling at the picture level.

[0149] According to various aspects of this disclosure, a first syntax element signaled at a first codec level is decoded from the encoded video stream. The first syntax element may indicate the maximum number of MVP candidates to be merged. The first codec level may be any suitable codec level, for example, higher than the fragment level. In the example, the first codec level is higher than the PPS level. The maximum number of MVP candidates to be merged may be determined based on the first syntax element, for example, based on Equation 5. Subsequently, the codec block associated with the first codec level may be reconstructed, at least based on the maximum number of MVP candidates to be merged. In the example, the first codec level is the sequence level, and the first syntax element is signaled at the SPS level. When the first codec level increases from a lower codec level (e.g., the PPS level) to a higher codec level (e.g., the sequence level), codec efficiency may increase and signaling overhead may decrease.

[0150] According to various aspects of this disclosure, one or more syntax elements related to the length of the corresponding merge candidate list (e.g., merge MVP candidate list, geometric merge pattern merge candidate list, sub-block-based merge MVP candidate list, IBC merge BVP candidate list, etc.) may be signaled at a first codec level. The first codec level can be any suitable codec level. In the example, the first codec level is higher than the fragment level or the image level. In the example, the first codec level is higher than the PPS level. One or more of the syntax elements may include a first syntax element.

[0151] In an embodiment, one or more of the syntax elements include a first syntax element, a second syntax element, a third syntax element, and a fourth syntax element, and are signaled at the sequence level in SPS. In the example, the first, second, third, and fourth syntax elements remain unchanged at the first codec level. In the example, the first codec level is the sequence level. When certain syntax elements (such as one or more of the syntax elements including the first, second, third, and fourth syntax elements) remain unchanged at the sequence level, signaling certain syntax elements at the sequence level rather than at a lower codec level (e.g., PPS level, picture level, or fragment level) can increase codec efficiency and / or reduce signaling overhead.

[0152] In this embodiment, the second, third, and fourth syntax elements can be signaled in the first codec level and can be decoded from the encoded video stream. The second syntax element may indicate the maximum number of merge candidates for geometric merge mode, the third syntax element may indicate the maximum number of MVP candidates for sub-block merge, and the fourth syntax element may indicate the maximum number of BVP candidates for IBC merge. The maximum number of merge candidates for geometric merge mode can be determined based on the second syntax element, the maximum number of MVP candidates for sub-block merge can be determined based on the third syntax element, and the maximum number of BVP candidates for IBC merge can be determined based on the fourth syntax element.

[0153] Figure 16 Exemplary syntax elements in SPS are shown. Syntax elements include a first syntax element (e.g., six_minus_max_num_merge_cand), a second syntax element (e.g., max_num_merge_cand_minus_max_num_geo_cand), a third syntax element (e.g., five_minus_max_num_subblock_merge_cand), and a fourth syntax element (e.g., six_minus_max_num_ibc_merge_cand).

[0154] The first syntax element subtracted from the value 6 (e.g., six_minus_max_num_merge_cand) can specify the maximum number of merge MVP candidates supported in SPS. The maximum number of merge MVP candidates (MaxNumMergeCand) can be determined based on the first syntax element (e.g., six_minus_max_num_merge_cand), for example, based on Equation 5.

[0155] MaxNumMergeCand=6-six_minus_max_num_merge_cand (Equation 5)

[0156] The value of MaxNumMergeCand can be in the range of 1 to 6 (inclusive).

[0157] The second syntax element subtracted from MaxNumMergeCand (e.g., max_num_merge_cand_minus_max_num_geo_cand) specifies the maximum number of geometric merge pattern merge candidates supported in SPS.

[0158] The maximum number of merge candidates in the geometric merge pattern (MaxNumGeoMergeCand) can be determined based on the second syntax element and the maximum number of merge MVP candidates, for example, using Equation 6.

[0159] MaxNumGeoMergeCand=MaxNumMergeCand

[0160] -max_num_merge_cand_minus_max_num_geo_cand (Equation 6)

[0161] When the second syntax element (e.g., max_num_merge_cand_minus_max_num_geo_cand) is present (or signaled), the maximum number of merge candidates in the geometry merge pattern (MaxNumGeoMergeCand) can be in the range of 2 to the maximum number of merge MVP candidates (MaxNumMergeCand) (inclusive of end values).

[0162] The third syntax element subtracted from the value 5 (e.g., five_minus_max_num_subblock_merge_cand) can specify the maximum number of subblock-based merge MVP candidates supported in SPS.

[0163] The maximum number of MVP candidates based on subblocks (MaxNumSubblockMergeCand) can be determined based on a third syntax element (e.g., five_minus_max_num_subblock_merge_cand) using Equation 7, for example.

[0164] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand

[0165] (Equation 7)

[0166] The value of MaxNumSubblockMergeCand can be in the range of 0 to 5 (inclusive).

[0167] The fourth syntax element (e.g., six_minus_max_num_ibc_merge_cand) subtracted from the value 6 can specify the maximum number of IBC merge BVP candidates supported in the SPS. The maximum number of IBC merge BVP candidates (MaxNumIbcMergeCand) can be determined based on the fourth syntax element (e.g., six_minus_max_num_ibc_merge_cand), for example, using Equation 8.

[0168] MaxNumIbcMergeCand=6-six_minus_max_num_ibc_merge_cand (Equation 8)

[0169] The value of MaxNumIbcMergeCand can be in the range of 0 to 6 (inclusive).

[0170] In the example, the signaling in the PPS includes the first, second, third, and fourth syntax elements. Therefore, the first, second, third, and fourth syntax elements can remain unchanged within the PPS and can be changed from the first PPS to the second PPS.

[0171] According to various aspects of this disclosure, one or more syntax elements related to the length of the corresponding merge candidate list can be conditionally signaled, for example, based on whether the corresponding inter-frame prediction codec (or inter-frame codec) is enabled. In embodiments, a first syntax element indicating the maximum number of merge MVP candidates (or the length of the merge MVP candidate list for a regular merge mode) is signaled, and the remaining syntax elements (e.g., second, third, fourth, etc.) related to the length of other merge candidate lists may or may not be signaled. Therefore, an enable flag for a specific inter-frame codec can be used to control the signaling of corresponding syntax elements related to the length of the merge candidate list of that specific inter-frame codec. Thus, in some examples, when an enable flag for a specific inter-frame codec indicates that the specific inter-frame codec is disabled, the corresponding syntax elements are not signaled, thereby increasing encoding / decoding efficiency and / or reducing signaling overhead.

[0172] In an embodiment, the syntax element signaled at the second codec level can be decoded from the encoded video stream based on the fulfillment of a condition. The syntax element may indicate the maximum number of merge candidates for one of the following: geometric merge mode, sub-block-based merge mode, IBC merge mode, etc.

[0173] In the example, one of the geometric merge mode, the sub-block-based merge mode, and the IBC merge mode is the geometric merge mode. The maximum number of merge candidates is the maximum number of merge candidates for the geometric merge mode. The syntax element is a second syntax element that indicates the maximum number of merge candidates for the geometric merge mode. Reference Figure 17 Box (1712) in the code is provided that geometric merging mode is enabled in the second codec level and the maximum number of merge MVP candidates is greater than or equal to 3. The maximum number of merge candidates in geometric merging mode can be determined based on the second syntax element.

[0174] In the example, one of the geometric merge mode, the sub-block-based merge mode, and the IBC merge mode is a sub-block-based merge mode. The maximum number of merge candidates is the maximum number of sub-block-based merge MVP candidates. The syntax element is a third syntax element that indicates the maximum number of sub-block-based merge MVP candidates. Reference Figure 17 Box (1710) in the code is provided that affine mode is enabled in the second codec level. The maximum number of merged MVP candidates based on sub-blocks can be determined based on the third syntax element.

[0175] In the example, one of the geometric merge mode, the sub-block-based merge mode, and the IBC merge mode is the IBC merge mode. The maximum number of merge candidates is the maximum number of IBC merge BVP candidates. The syntax element is the fourth syntax element indicating the maximum number of IBC merge BVP candidates. Reference Figure 17 Box (1711) in the codec indicates that IBC merging mode is enabled in the second codec level. The maximum number of IBC merging BVP candidates can be determined based on the fourth syntax element.

[0176] In the example, the first syntax element indicating the length of the merge MVP candidate list is signaled in SPS, and the second syntax element indicates the length of the merge candidate list for geometric merge mode, the third syntax element indicates the length of the merge MVP candidate list for sub-block-based merge, and the fourth syntax element indicates the length of the merge candidate list for IBC, for example in... Figure 17 The sequence levels shown are conditionally notified by signals.

[0177] refer to Figure 17In the SPS, a signal is sent to the first syntax element (e.g., six_minus_max_num_merge_cand). As described above, the first syntax element (e.g., six_minus_max_num_merge_cand) subtracted from the value 6 can specify the maximum number of merge MVP candidates supported in the SPS. The maximum number of merge MVP candidates (MaxNumMergeCand) can be determined based on the first syntax element (e.g., six_minus_max_num_merge_cand), for example, based on Equation 5. The value of MaxNumMergeCand can be in the range of 1 to 6 (inclusive).

[0178] refer to Figure 17 Box (1712) in the code allows signaling to the second syntax element (e.g., max_num_merge_cand_minus_max_num_geo_cand) when the first sequence-level flag (e.g., sps_geo_enabled_flag) is equal to 1 and the maximum number of merged MVP candidates (MaxNumMergeCand) is greater than or equal to 3. In the example, the first sequence-level flag (e.g., sps_geo_enabled_flag) being equal to 1 indicates that geometric merge mode is enabled at the sequence level.

[0179] The second syntax element subtracted from MaxNumMergeCand (e.g., max_num_merge_cand_minus_max_num_geo_cand) specifies the maximum number of geometric merge pattern merge candidates supported in SPS.

[0180] The maximum number of GEO merge pattern candidates (MaxNumGeoMergeCand) can be determined using Equation 6.

[0181] When the second syntax element (e.g., max_num_merge_cand_minus_max_num_geo_cand) is present, the maximum number of GEO merge pattern candidates (MaxNumGeoMergeCand) can be in the range of 2 to the maximum number of merge MVP candidates (MaxNumMergeCand) (inclusive of end values).

[0182] When the second syntax element (e.g., max_num_merge_cand_minus_max_num_geo_cand) is not present, the first sequence level flag (e.g., sps_geo_enabled_flag) is equal to 1, and the maximum number of merge MVP candidates (MaxNumMergeCand) is equal to 2. The maximum number of GEO merge mode candidates (MaxNumGeoMergeCand) can be set to 2.

[0183] The maximum number of merge candidates in the geometric merge mode (MaxNumGeoMergeCand) can be set to 0 when one of the following conditions is met: (i) the first sequence level flag (e.g., sps_geo_enabled_flag) is equal to 0 and (ii) the maximum number of merge MVP candidates (MaxNumMergeCand) is less than 2.

[0184] refer to Figure 17 Box (1710) in the code can signal a third syntax element (e.g., five_minus_max_num_subblock_merge_cand) when the second sequence-level flag (e.g., sps_affine_enabled_flag) is equal to 1. In the example, the second sequence-level flag (e.g., sps_affine_enabled_flag) being equal to 1 indicates that affine mode is enabled at the sequence level.

[0185] The third syntax element subtracted from the value 5 (e.g., five_minus_max_num_subblock_merge_cand) specifies the maximum number of subblock-based merge MVP candidates supported in SPS. When the third syntax element is not present, its value (e.g., five_minus_max_num_subblock_merge_cand) can be inferred to be equal to (5 - sps_sbtmvp_enabled_flag).

[0186] The maximum number of MVP candidates based on subblocks (MaxNumSubblockMergeCand) can be determined based on Equation 7.

[0187] The value of MaxNumSubblockMergeCand can be in the range of 0 to 5 (inclusive).

[0188] refer to Figure 17Box (1711) in the code allows signaling to a fourth syntax element (e.g., six_minus_max_num_ibc_merge_cand) when a third sequence-level flag (e.g., sps_ibc_enabled_flag) is equal to 1. In the example, a third sequence-level flag (e.g., sps_ibc_enabled_flag) equal to 1 indicates that IBC merge mode is enabled at the sequence level.

[0189] The fourth syntax element (e.g., six_minus_max_num_ibc_merge_cand) subtracted from the value 6 can specify the maximum number of IBC merge BVP candidates supported in the SPS. The maximum number of IBC merge BVP candidates (MaxNumIbcMergeCand) can be determined based on the fourth syntax element (e.g., six_minus_max_num_ibc_merge_cand), for example, using Equation 8.

[0190] MaxNumIbcMergeCand can be equal to 0 when the fourth syntax element (e.g., six_minus_max_num_ibc_merge_cand) is not present.

[0191] The value of MaxNumIbcMergeCand can be in the range of 0 to 6 (inclusive).

[0192] In the example, such as Figure 17 As shown, a first syntax element is signaled at a first codec level (e.g., a sequence level), and a second, third, and fourth syntax element are conditionally signaled at a second codec level (e.g., a sequence level). Alternatively, a first syntax element is signaled at a first codec level (e.g., a sequence level), and one or more of the following are conditionally signaled at a second codec level (e.g., a sequence level): (i) a second syntax element, (ii) a third syntax element, (iii) a fourth syntax element, and / or other syntax elements. The first codec level can be equal to or higher than the second codec level. In the example, the first and second codec levels are sequence levels.

[0193] In this embodiment, a first syntax element indicating the length of the merged MVP candidate list is signaled at the first codec level, and a second syntax element indicating the length of the geometric merge mode merge candidate list, a third syntax element indicating the length of the sub-block-based merged MVP candidate list, and a fourth syntax element indicating the length of the IBC merge candidate list are conditionally signaled at the first codec level. In this example, the first codec level is higher than the fragment level (or fragment header), picture level (or picture header), etc. In this example, the first codec level is the sequence level.

[0194] Some syntax elements (e.g., the first syntax element) remain unchanged at the first codec level (e.g., the sequence level) and are invariant for pictures, fragments, etc., within the sequence. Therefore, for example, certain syntax elements can be signaled at the first codec level (e.g., a higher codec level) instead of at a codec level lower than the first codec level to improve codec efficiency and / or reduce signaling overhead. Other syntax elements (e.g., the third syntax element) can change at the first codec level and therefore can be signaled at a second codec level lower than the first codec level (e.g., a lower codec level) to, for example, increase flexibility. For example, if the first codec level is the sequence level, and the third syntax element changes within the sequence but not within a picture, it can be signaled at the picture level (e.g., the second codec level).

[0195] In an embodiment, a first syntax element indicating the length of the merge MVP candidate list is signaled at a first codec level, and a second syntax element indicating the length of the geometric merge mode merge candidate list and a fourth syntax element indicating the length of the IBC merge candidate list are conditionally signaled at the first codec level. Other syntax elements indicating the length of other merge candidate lists (e.g., a third syntax element indicating the length of the sub-block-based merge MVP candidate list) may be signaled at a second codec level lower than the first codec level. In the example, the first codec level is a sequence level, and the second codec level is a PPS level, a picture level, or a fragment level.

[0196] In the example, only the first, second, and fourth syntax elements at the sequence level are signaled in the SPS. Other syntax elements indicating the length of other merge candidate lists are signaled at a level lower than the sequence level (e.g., the picture level). These other syntax elements may include a third syntax element. In the example, the third syntax element changes within the SPS, for example, from the first picture to the second picture, and is therefore signaled at the picture level for each picture to provide greater flexibility.

[0197] Figure 18AThis illustrates signaling the first, second, and fourth syntax elements (e.g., six_minus_max_num_ibc_merge_cand) at the sequence level in SPS. Figure 18B This demonstrates how to signal a third syntax element at the image level within the image header. Figures 18A to 18B The syntax elements shown can be applied in the following case: the first syntax element (e.g., six_minus_max_num_merge_cand), the second syntax element (e.g., max_num_merge_cand_minus_max_num_geo_cand), and the fourth syntax element do not change at the sequence level, and the third syntax element (e.g., five_minus_max_num_subblock_merge_cand) can change within the sequence.

[0198] In an embodiment, the first syntax element indicating the length of the merge MVP candidate list remains unchanged (e.g., not updated, overwritten, or modified) at codec levels below the first codec level. The second and fourth syntax elements, conditionally signaled at the second codec level, remain unchanged (e.g., not updated, overwritten, or modified) at codec levels below the second codec level. In the example, only the following syntax elements may be updated, overwritten, or modified at codec levels below the second codec level: syntax elements indicating the length of other merge candidate lists excluding any of the merge MVP candidate list, the geometric merge pattern merge candidate list, and the IBC merge candidate list. In the example, the third syntax element conditionally signaled at the second codec level is modified at codec levels below the second codec level. For example, the third syntax element conditionally signaled at the second codec level (e.g., the sequence level) is modified at the picture level.

[0199] In the example, only the third syntax element (which indicates the length of the merge candidate list based on sub-blocks) can be modified, overwritten, or updated at a codec level (e.g., picture level) lower than the first codec level (e.g., sequence level).

[0200] Figure 19A The first, second, and fourth syntax elements (e.g., six_minus_max_num_ibc_merge_cand) are signaled at the sequence level in SPS and remain unchanged at the sequence level. Figures 19A to 19B The third syntax element is shown to be signaled at the sequence level in SPS, and can be further modified at the image level in the image header. Figures 19A to 19BThe syntax elements shown can be applied in the following situations: the first syntax element (e.g., six_minus_max_num_merge_cand), the second syntax element (e.g., max_num_merge_cand_minus_max_num_geo_cand), and the fourth syntax element do not change at the sequence level, and the third syntax element (e.g., five_minus_max_num_subblock_merge_cand) can be modified from one image to another.

[0201] When the third syntax element (e.g., five_minus_max_num_subblock_merge_cand) does not exist in SPS, and both pic_temporal_mvp_enabled_flag and sps_sbtmvp_enabled_flag are equal to 1, the maximum number of MVP candidates based on subblocks (MaxNumSubblockMergeCand) can be equal to 1 for the slice associated with the current image header.

[0202] According to various aspects of this disclosure, when one or more of the first, second, third, and fourth syntax elements are signaled at a first codec level (e.g., sequence level) higher than the fragment level or picture level, one or more of the first, second, third, and fourth syntax elements cannot be updated, overwritten, or modified at a codec level lower than the first codec level (e.g., sequence level) (e.g., PPS level, picture level, or fragment level).

[0203] In the example, the first, second, third, and fourth syntax elements are signaled at the sequence level in SPS, and they cannot be updated, overwritten, or modified at codec levels below the sequence level (e.g., PPS level, picture level, or slice level).

[0204] In the example, the first, second, third, and fourth syntax elements are signaled at the PPS level, and they cannot be updated, overwritten, or modified at a codec level lower than the PPS level (e.g., at the picture level or slice level).

[0205] According to various aspects of this disclosure, additional syntax elements can be used to further indicate the presence of one or more syntax elements related to the length of various merge candidate lists.

[0206] In the example, such as Figure 20 The box in (2010) indicates the presence of a fourth syntax element or flag (e.g., the six_minus_max_num_ibc_merge_cand_present flag) that indicates the presence of a fourth syntax element related to the length of the IBC merge BVP candidate list. This is used to further indicate the presence of the fourth syntax element, such as... Figure 20 As shown in the diagram. Therefore, in addition to the third sequence level flag (e.g., `sps_ibc_enabled_flag`), the existence of the fourth syntax element is conditionally determined based on the syntax element (e.g., the `six_minus_max_num_ibc_merge_cand_present` flag). The fourth syntax element is conditionally signaled based on both the syntax element (e.g., the `six_minus_max_num_ibc_merge_cand_present` flag) and the third sequence level flag (e.g., `sps_ibc_enabled_flag`). The above description can be appropriately adapted to apply to the signaling of other syntax elements, such as the second syntax element, the third syntax element, etc.

[0207] The first syntax element subtracted from the value 6 (e.g., six_minus_max_num_merge_cand) specifies the maximum number of merge MVP candidates supported in the SPS. The maximum number of merge MVP candidates (MaxNumMergeCand) can be determined based on Equation 5 as described above. The value of MaxNumMergeCand can be in the range of 1 to 6 (inclusive).

[0208] The second syntax element subtracted from MaxNumMergeCand (e.g., max_num_merge_cand_minus_max_num_geo_cand) specifies the maximum number of geometric merge pattern merge candidates supported in SPS.

[0209] The maximum number of merger candidates in the geometric merge pattern (MaxNumGeoMergeCand) can be determined based on Equation 6 as described above.

[0210] When max_num_merge_cand_minus_max_num_geo_cand exists, the value of MaxNumGeoMergeCand can be in the range of 2 to MaxNumMergeCand (inclusive).

[0211] When max_num_merge_cand_minus_max_num_geo_cand does not exist, and sps_geo_enabled_flag equals 1 and MaxNumMergeCand equals 2, MaxNumGeoMergeCand is set to equal to 2.

[0212] MaxNumGeoMergeCand is set to 0 when max_num_merge_cand_minus_max_num_geo_cand does not exist and (sps_geo_enabled_flag is equal to 0 or MaxNumMergeCand is less than 2).

[0213] The third syntax element subtracted from the value 5 (e.g., five_minus_max_num_subblock_merge_cand) specifies the maximum number of subblock-based merge MVP candidates supported in SPS. When the third syntax element is not present, the value of five_minus_max_num_subblock_merge_cand can be inferred to be equal to (5 - sps_sbtmvp_enabled_flag).

[0214] The maximum number of MVP candidates based on subblocks (MaxNumSubblockMergeCand) can be determined based on Equation 7.

[0215] The value of MaxNumSubblockMergeCand can be in the range of 0 to 5 (inclusive).

[0216] The syntax element `six_minus_max_num_ibc_merge_cand_present` specifies the maximum number of IBC merge BVP candidates that can be present. When the `six_minus_max_num_ibc_merge_cand_present` flag is absent, it can be inferred to be 0. When the `six_minus_max_num_ibc_merge_cand_present` flag is present, a fourth syntax element (e.g., `six_minus_max_num_ibc_merge_cand`) is present in the bitstream.

[0217] The fourth syntax element subtracted from value 6 (e.g., six_minus_max_num_ibc_merge_cand) specifies the maximum number of IBC merge BVP candidates supported in SPS. The maximum number of IBC merge BVP candidates (MaxNumIbcMergeCand) can be determined using Equation 8.

[0218] MaxNumIbcMergeCand can be equal to 0 when the fourth syntax element (e.g., six_minus_max_num_ibc_merge_cand) is not present.

[0219] The value of MaxNumIbcMergeCand can be in the range of 1 to 6 (inclusive).

[0220] Figure 21 A flowchart outlining a method (2100) according to an embodiment of the present disclosure is shown. The method (2100) can be used to signal various syntax elements indicating the maximum number of merge candidates, such as the maximum number of merge MVP candidates, the maximum number of merge candidates for geometric merge mode, the maximum number of merge MVP candidates based on sub-blocks, the maximum number of merge BVP candidates for IBC, etc. In various embodiments, the method (2100) is executed by processing circuitry, such as processing circuitry in terminal devices (210), (220), (230), and (240), processing circuitry performing the functions of a video encoder (303), processing circuitry performing the functions of a video decoder (310), processing circuitry performing the functions of a video decoder (410), processing circuitry performing the functions of a video encoder (503), etc. In some embodiments, the method (2100) is implemented as software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry executes the method (2100). The method begins at (S2101) and proceeds to (S2110).

[0221] In (S2110), a first syntax element is decoded from the encoded video stream. This first syntax element is signaled at a first codec level. The first syntax element indicates the maximum number of MVP candidates for merging motion vector predictions. The first codec level is higher than the picture parameter set (PPS) level. In this example, the first codec level is the sequence level.

[0222] In (S2120), the maximum number of merged MVP candidates is determined based on the first syntax element.

[0223] At (S2130), the coded block associated with the first codec level is reconstructed, at least based on the maximum number of merged MVP candidates. Method (2100) proceeds to (S2199) and terminates.

[0224] Method (2100) can be modified appropriately. Steps in method (2100) can be modified and / or omitted. Additional steps can be added. Any suitable implementation order can be used.

[0225] In some examples, additional syntax elements, such as the second, third, and fourth syntax elements, can be signaled or conditionally signaled. Therefore, the maximum number of geometric merge pattern merge candidates can be determined based on the second syntax element, the maximum number of sub-block-based merge MVP candidates can be determined based on the third syntax element, and the maximum number of IBC merge BVP candidates can be determined based on the fourth syntax element.

[0226] The embodiments in this disclosure can be used individually or in any combination in any order. Furthermore, each of the method (or embodiment), encoder, and decoder can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-volatile computer-readable medium. As described above, the method (or embodiment), encoder, and decoder in this disclosure can increase encoding / decoding efficiency and / or reduce signaling overhead.

[0227] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 22 A computer system (2200) is shown, which is adapted to implement certain embodiments of the disclosed subject matter.

[0228] The computer software can be encoded using any suitable machine code or computer language, and code including instructions can be created through mechanisms such as assembly, compilation, and linking. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed through decoding, microcode, etc.

[0229] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0230] Figure 22 The components shown for the computer system (2200) are exemplary in nature and are not intended to limit the scope or functionality of the computer software used to implement the embodiments of this application. Nor should the configuration of the components be construed as having any dependency or requirement on any component or combination thereof shown in the exemplary embodiments of the computer system (2200).

[0231] The computer system (2200) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, swiping, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-machine interface device may also be used to capture certain media, which need not be directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from still cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0232] Human-machine interface input devices may include one or more of the following (only one is shown): keyboard (2201), mouse (2202), touchpad (2203), touch screen (2210), data glove (not shown), joystick (2205), microphone (2206), scanner (2207), camera (2208).

[0233] The computer system (2200) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (2210), data gloves (not shown), or joystick (2205), but may also include tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (2209), headphones (not shown)), visual output devices (e.g., screens (2210) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light-emitting diode screens, each of which may or may not have touchscreen input functionality, each of which may or may not have tactile feedback functionality—some of which may output two-dimensional or more three-dimensional visual outputs by means such as stereoscopic image output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)), and printers (not shown).

[0234] The computer system (2200) may also include human-accessible storage devices and related media, such as optical media including high-density read-only / rewritable optical discs (CD / DVD ROM / RW) (2220) or similar media (2221), thumb drives (2222), removable hard disk drives or solid-state drives (2223), conventional magnetic media such as magnetic tapes and floppy disks (not shown), dedicated devices based on ROM / ASIC / PLD such as security software protectors (not shown), and so on.

[0235] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0236] The computer system (2200) may also include interfaces to one or more communication networks. For example, the network may be wireless, wired, or optical. The network may also be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a vehicle-mounted network and an industrial network, a real-time network, a latency-tolerant network, and so on. Networks also include Ethernet, wireless LANs, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), cable or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), vehicle-mounted and industrial networks (including CANBus), and so on. Some networks typically require external network interface adapters for connection to certain general-purpose data ports or peripheral buses (2249) (e.g., the USB port of the computer system (2200); other systems are typically integrated into the core of the computer system (2200) via a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). By using any of these networks, the computer system (2200) can communicate with other entities. The communication can be unidirectional, used only for receiving (e.g., wireless television), unidirectional, used only for sending (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as through a local area or wide area digital network to other computer systems. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0237] The aforementioned human-computer interface device, human-accessible storage device, and network interface can be connected to the core (2240) of the computer system (2200).

[0238] The core (2240) may include one or more central processing units (CPU) (2241), graphics processing units (GPUs) (2242), dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) (2243), task-specific hardware accelerators (2244), etc. These devices, along with read-only memory (ROM) (2245), random access memory (2246), and internal mass storage (e.g., internal non-user-accessible hard disk drives, solid-state drives, etc.) (2247), can be connected via a system bus (2248). In some computer systems, the system bus (2248) can be accessed via one or more physical connectors to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (2248) or connected via a peripheral bus (2249). Peripheral bus architectures include external controller interfaces (PCI), universal serial buses (USB), etc.

[0239] The CPU (2241), GPU (2242), FPGA (2243), and accelerator (2244) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (2245) or RAM (2246). Transient data can also be stored in RAM (2246), while permanent data can be stored, for example, in internal mass storage (2247). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (2241), GPUs (2242), mass storage (2247), ROM (2245), RAM (2246), etc.

[0240] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be specifically designed and constructed for the purposes of this application, or they may be media and code well-known and usable by those skilled in the art of computer software.

[0241] By way of example and not limitation, a computer system having an architecture (2200), particularly a core (2240), can provide functionality as a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with the aforementioned user-accessible mass storage, as well as specific memory of the non-volatile core (2240), such as internal mass storage (2247) or ROM (2245). Software implementing various embodiments of this application can be stored in such a device and executed by the core (2240). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the core (2240), particularly the processor therein (including a CPU, GPU, FPGA, etc.), to execute a specific process or a specific portion of a specific process described herein, including defining data structures stored in RAM (2246) and modifying such data structures according to a software-defined process. Alternatively or as an alternative, the computer system may provide logic hardwired or otherwise incorporated into circuitry (e.g., an accelerator (2244)) that may replace or operate with the software to perform the specific process or a specific portion of the specific process described herein. References to software may include logic, and vice versa, where appropriate. References to computer-readable media may include, where appropriate, circuitry storing the execution of software (such as an integrated circuit (IC)), circuitry containing the execution logic, or both. This application includes any suitable combination of hardware and software.

[0242] Appendix A: Acronyms

[0243] JEM: Joint Exploration Model

[0244] VVC: Versatile Video Coding

[0245] BMS: Benchmark set

[0246] MV: Motion Vector

[0247] HEVC: High Efficiency Video Coding

[0248] SEI: Supplementary Enhancement Information

[0249] VUI: Video Usability Information

[0250] GOPs: Groups of Pictures

[0251] TUs: Transform Units

[0252] PUs: Prediction Units

[0253] CTUs: Coding Tree Units

[0254] CTBs: Coding Tree Blocks

[0255] PBs: Prediction Blocks

[0256] HRD: Hypothetical Reference Decoder

[0257] SNR: Signal-to-Noise Ratio

[0258] CPUs: Central Processing Units

[0259] GPUs: Graphics Processing Units

[0260] CRT: Cathode Ray Tube

[0261] LCD: Liquid Crystal Display

[0262] OLED: Organic Light-Emitting Diode

[0263] CD: Compact Disc

[0264] DVD: Digital Video Disc

[0265] ROM: Read-Only Memory

[0266] RAM: Random Access Memory

[0267] ASIC: Application-Specific Integrated Circuit

[0268] PLD: Programmable Logic Device

[0269] LAN: Local Area Network

[0270] GSM: Global System for Mobile Communications

[0271] LTE: Long-Term Evolution

[0272] CANBus: Controller Area Network Bus

[0273] USB: Universal Serial Bus

[0274] PCI: Peripheral Component Interconnect

[0275] FPGA: Field Programmable Gate Array

[0276] SSD: Solid-state drive

[0277] IC: Integrated Circuit

[0278] CU: Coding Unit

[0279] While this application has described several exemplary embodiments, various modifications, arrangements, and equivalent substitutions of the embodiments are all within the scope of this application. Therefore, it should be understood that those skilled in the art can design various systems and methods that, although not explicitly shown or described herein, embody the principles of this application and are thus within its spirit and scope.

Claims

1. A video encoding method, characterized in that, include: Generate a first syntax element, which indicates the maximum number of MVP candidates for merging motion vector predictions; The first syntax element is encoded in the encoded video stream at a first codec level, which is higher than the picture parameter set (PPS) level. as well as Send the encoded video stream containing the first syntax element; The maximum number of merged MVP candidates indicated by the first syntax element is applicable to reconstructing the encoded blocks of the encoded video stream associated with the first codec level; The method further includes: Based on the conditions met, the encoded video stream further includes: a second syntax element of a second codec level, the second syntax element indicating the maximum number of merge candidates for one of the geometric merge mode, the sub-block-based merge mode, and the intra-block copy (IBC) merge mode. Wherein, when one of the geometric merge mode, the sub-block-based merge mode, and the IBC merge mode is a geometric merge mode: The maximum number of merge candidates is the maximum number of merge candidates in the geometric merge pattern; The second syntax element indicates the maximum number of merge candidates for the geometric merge pattern; and The conditions are: the geometric merging mode is enabled in the second codec level, and the maximum number of merged MVP candidates is greater than or equal to 3. The first codec level and the second codec level are sequence levels.

2. The method according to claim 1, wherein, The encoded video stream further includes: a second syntax element, a third syntax element, and a fourth syntax element signaled in the first codec level, wherein the second syntax element indicates the maximum number of geometric merge mode merge candidates, the third syntax element indicates the maximum number of sub-block merge MVP candidates, and the fourth syntax element indicates the maximum number of intra-block copy (IBC) merge block vector prediction (BVP) candidates.

3. The method according to claim 1, wherein, The first encoding / decoding level is the sequence level, and the first syntax element is encoded using the Sequence Parameter Set (SPS).

4. The method according to claim 1, wherein, When one of the geometric merge mode, the sub-block-based merge mode, and the IBC merge mode is a sub-block-based merge mode: The maximum number of merge candidates is based on the maximum number of merge MVP candidates for sub-blocks; The second syntax element indicates the maximum number of sub-block-based merge MVP candidates; and The condition is: affine mode is enabled in the second codec level.

5. The method according to claim 1, wherein, When one of the geometric merge mode, the sub-block-based merge mode, and the IBC merge mode is the IBC merge mode: The maximum number of merge candidates is the maximum number of IBC merge BVP candidates; The second syntax element indicates the maximum number of BVP candidates that IBC can merge; and The condition is that the IBC merging mode is enabled in the second codec level.

6. The method according to claim 4, wherein, Modify the second syntax element at a codec level lower than the second codec level.

7. The method according to claim 6, wherein, The first and second codec levels are sequence levels, and the codec level below the second codec level is the image level.

8. A method for video decoding, characterized in that, include: Decode the first syntax element from the encoded video stream. The first syntax element is signaled at the first encoding / decoding level. The first syntax element indicates the maximum number of MVP candidates for merging motion vector predictions. The maximum number of merged MVP candidates is determined based on the first syntax element; as well as At least based on the maximum number of merged MVP candidates, reconstruct the encoded blocks of the encoded video stream associated with the first codec level; The method further includes: Based on the conditions met, the encoded video stream further includes: a second syntax element of a second codec level, the second syntax element indicating the maximum number of merge candidates for one of the geometric merge mode, the sub-block-based merge mode, and the intra-block copy (IBC) merge mode. Wherein, when one of the geometric merge mode, the sub-block-based merge mode, and the IBC merge mode is a geometric merge mode: The maximum number of merge candidates is the maximum number of merge candidates in the geometric merge pattern; The second syntax element indicates the maximum number of merge candidates for the geometric merge pattern; and The conditions are: the geometric merging mode is enabled in the second codec level, and the maximum number of merged MVP candidates is greater than or equal to 3. The first codec level and the second codec level are sequence levels.

9. A video encoding apparatus, characterized in that, The method includes a processing circuit configured to perform the method according to any one of claims 1-7.

10. An apparatus for video decoding, characterized in that, It includes processing circuitry configured to perform the method of claim 8.

11. A method for storing video streams, characterized in that, The video encoding method according to any one of claims 1-7 is used to generate a video stream and store the video stream.

12. A method for transmitting video bitstreams, characterized in that, The video encoding method according to any one of claims 1-7 is used to generate a video stream and transmit the video stream.

13. A computer-readable storage medium storing a computer program / instructions and a video stream thereon, characterized in that, When executed by a processor, the computer program / instructions implement the steps of the video encoding method according to any one of claims 1-7 to generate the video stream.

14. A non-volatile computer-readable medium storing instructions, characterized in that, When executed by a computer for video encoding, the instructions cause the computer to perform the method as described in any one of claims 1-7, and when executed by a computer for video decoding, the instructions cause the computer to perform the method as described in claim 8.

15. A computer system, characterized in that, The computer system includes: At least one memory is configured to store program code; and At least one processor is configured to read the program code and operate according to the instructions of the program code to perform the method as described in any one of claims 1-8.

Images