Method and apparatus for palette based coding mode under local dual tree structure

Abstract

To provide methods and apparatuses for video encoding / decoding.SOLUTION: An apparatus for video decoding includes receiving circuitry and processing circuitry. The processing circuitry decodes prediction information of a block from a coded video bitstream, determines whether to allow a palette based coding mode for the block based on a comparison of a size of the block with a threshold, determines whether the block is under a local dual tree structure based on the prediction information, and decodes the block based on the determination of whether to allow the palette based coding mode for the block and the determination of whether the block is under the local dual tree structure.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] Incorporation by Reference This application claims benefit of priority to U.S. Provisional Application No. 62 / 963,216 "SIMPLIFIED PALETTE MODE CODING WITH LOCAL DUAL TREE CODING STRUCTURE," filed on January 20, 2020, which claims benefit of priority to U.S. Provisional Application No. 17 / 097,415 "METHOD AND APPARATUS FOR PALETTE BASED CODING MODE UNDER LOCAL DUAL TREE STRUCTURE," filed on November 13, 2020. The entire disclosure of the prior application is incorporated by reference in its entirety.

[0002] This disclosure describes embodiments generally related to video encoding. [Background technology]

[0003] The background art description provided herein is intended to generally present the context of the present disclosure. To the extent that it is described in this Background Art section, the work of the currently named inventors, and aspects of the description that may not qualify as prior art at the time of filing, are not expressly or impliedly admitted as prior art to the present disclosure.

[0004] Video encoding and decoding may be performed using inter-picture prediction with motion compensation. Uncompressed digital video may include a sequence of images, each having spatial dimensions of, for example, 1920x1080 luma samples and associated chroma samples. The sequence of images may have a fixed or variable picture rate (also informally known as frame rate), for example, 60 images per second or 60 Hz. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luma sample resolution at a frame rate of 60 Hz) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 GByte of storage space.

[0005] One of the goals of video encoding and decoding may be the reduction of redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage space requirements, in some cases by more than two orders of magnitude. Both lossless and lossy compression, as well as combinations thereof, may be employed. Lossless compression refers to techniques where an exact copy of the original signal can be reconstructed from a compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough to make the reconstructed signal useful for the intended application. For video, lossy compression is widely adopted. The amount of acceptable distortion varies by application; for example, users of a particular consumer streaming application may tolerate higher distortion than users of a television distribution application. The achievable compression ratio may reflect that a higher acceptable distortion can result in a higher compression ratio.

[0006] Video encoders and decoders can utilize techniques from a number of broad categories, including, for example, motion compensation, transform, quantization, and entropy coding.

[0007] Video codec techniques can include a technique known as intra-coding. In intra-coding, sample values ​​are represented without reference to the samples or other data from a previously reconstructed reference picture. In some video codecs, an image is spatially subdivided into blocks of samples. If all blocks of samples are coded in intra mode, the image may be an intra-image. Intra-images and their derivatives, such as independent decoder refresh images, can be used to reset the decoder state and thus can be used as the first image in the coded video bitstream and video session or as still images. Samples of an intra-block may be subjected to a transform, and the transform coefficients may be quantized before entropy coding. Intra prediction may be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value after the transform and the smaller the AC coefficients, the fewer bits are required for a given quantization step size to represent the block after entropy coding.

[0008] Conventional intra-coding, for example as known from MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that attempt to do so from surrounding sample data and / or metadata obtained during encoding / decoding of blocks of spatially adjacent and preceding data in decoding order. Such techniques are hereafter referred to as "intra-prediction" techniques. It should be noted that in at least some cases, intra-prediction uses only reference data from the current image being reconstructed, and not from reference images.

[0009] Intra prediction can take many different forms. When two or more of such techniques can be used in a given video coding technique, the technique in use can be coded in an intra prediction mode. In some cases, a mode can have sub-modes and / or parameters, which can be coded separately or included in a mode codeword. Which codeword is used for a given mode / sub-mode / parameter combination can affect the coding efficiency gains via intra prediction, and therefore also the entropy coding technique used to convert the codeword into a bitstream.

[0010] Certain modes of intra prediction were introduced in H.264, improved in H.265, and further refined in new coding techniques such as the Joint Search Model (JEM), Generic Video Coding (VVC), and Benchmark Set (BMS). A predictor block may be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of the neighboring samples are copied to the predictor block according to a direction. The reference to the direction in use may be coded in the bitstream or may itself be predicted.

[0011] Referring to FIG. 1, at the bottom right, a subset of 9 known predictor directions from the 33 possible predictor directions of H.265 (corresponding to the 33 angle modes of the 35 intra modes) is shown. The point (101) where the arrows converge represents the sample to be predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from one or more samples to the top right at an angle of 45 degrees from the horizontal. Similarly, arrow (103) indicates that sample (101) is predicted from one or more samples to the bottom left of sample (101) at an angle of 22.5 degrees from the horizontal.

[0012] Still referring to FIG. 1, at the top left is shown a square block (104) of 4×4 samples (shown in dashed bold). The square block (104) includes 16 samples, each labeled with an “S”, its position in the Y dimension (e.g., row index) and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample (from the top) in the Y dimension and the first sample (from the left) in the X dimension. Similarly, sample S44 is the fourth sample in both the Y and X dimensions in the block (104). Since the size of the block is 4×4 samples, S44 is at the bottom right. Further shown are reference samples that follow a similar numbering scheme. The reference samples are labeled R, their Y position (e.g., row index) and X position (column index) relative to the block (104). In both H.264 and H.265, the predicted samples are adjacent to the block being reconstructed; therefore, there is no need to use negative values.

[0013] Intra-image prediction can work by copying reference sample values ​​from adjacent samples as appropriate by the signaled prediction direction. For example, assume that the coded video bitstream includes signaling for this block indicating a prediction direction that coincides with the arrow (102), i.e., the top right sample is predicted from one or more prediction samples at an angle of 45 degrees from the horizontal. In that case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Then, sample S44 is predicted from reference sample R08.

[0014] In certain cases, the values ​​of several reference samples can be combined, for example by interpolation, to calculate the reference sample; this is especially possible when the orientation is not evenly divisible by 45 degrees.

[0015] The number of possible directions is increasing as video coding techniques develop. In H.264 (2003), nine different directions could be represented. This increased to 33 in H.265 (2013), and JEM / VVC / BMS can support up to 65 directions at the time of this disclosure. Experiments have been performed to identify the most likely directions, and certain techniques in entropy coding are used to represent those likely directions with a small number of bits, accepting certain penalties for less likely directions. Furthermore, the direction itself may be predictable from neighboring directions used in neighboring already decoded blocks.

[0016] FIG. 2 shows a schematic diagram (201) of 65 intra prediction directions with JEM to illustrate the increasing number of prediction directions over time.

[0017] The mapping of intra-prediction direction bits in the coded video bitstream representing the directions can vary from video coding technique to video coding technique; for example, it can vary from a simple direct mapping of prediction directions to complex adaptation schemes involving intra-prediction modes, codewords, most likely modes, and similar techniques. In all cases, however, there may be certain directions that are statistically less likely to occur in the video content than certain other directions. Because the goal of video compression is to reduce redundancy, in a well-performing video coding technique, those less likely directions are represented with more bits than the more likely directions. Summary of the Invention [Means for solving the problem]

[0018] Aspects of the present disclosure provide a method and apparatus for video encoding / decoding. In some examples, the apparatus for video decoding includes a receiving circuit and a processing circuit. For example, the processing circuit decodes prediction information of a block from an encoded video bitstream, determines whether to allow a palette-based coding mode of the block based on a comparison between a size of the block and a threshold, and determines whether the block is under a local dual tree structure based on the prediction information. Furthermore, the processing circuit decodes the block based on the determination of whether to allow a palette-based coding mode of the block and the determination of whether the block is under a local dual tree structure.

[0019] In some embodiments, the processing circuitry disallows a palette-based coding mode for the block in response to the size of the block being smaller than a threshold. In some examples, the threshold is defined to disallow the local dual tree structure in response to the block being larger than the threshold. In one example, the processing circuitry disallows a palette-based coding mode for the block in response to the block being a chroma block and the size of the chroma block being smaller than the threshold. In some examples, the processing circuitry disallows a palette-based coding mode for the block based on a combination of the mode type and tree type of the block indicating the block is a chroma block under the local dual tree structure.

[0020] In some embodiments, in response to the palette-based encoding mode being enabled, the processing circuitry decodes from the encoded video bitstream a flag indicating whether the palette-based encoding mode is used for the block, and in response to the flag indicating use of the palette-based encoding mode for the block, decodes the block using the palette-based encoding mode. Additionally, in some examples, in response to the block being under the local dual tree structure, the processing circuitry bypasses updating the palette predictor list. In one example, in response to not triggering the local dual tree structure, the processing circuitry updates the palette predictor list based on the palette used to decode the block.

[0021] In some other embodiments, in response to the palette-based encoding mode being enabled, the processing circuitry decodes from the encoded video bitstream a flag indicating whether the palette-based encoding mode is used for the block, and in response to the flag indicating use of the palette-based encoding mode for the block and for blocks below the local dual tree structure, decodes the block based on a palette with a reduced maximum size. Further, the processing circuitry can update a palette predictor list based on the palette used to decode the block, where the size of the palette predictor list is limited by the reduced maximum size.

[0022] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a method for video decoding.

[0023] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. [Brief description of the drawings]

[0024] [Figure 1] FIG. 2 is a schematic diagram of an example subset of intra-prediction modes. [Diagram 2] FIG. 2 is a diagram of an example intra-prediction direction. [Diagram 3] FIG. 3 is a schematic diagram of a simplified block diagram of a communication system (300) according to one embodiment. [Figure 4] FIG. 4 is a schematic diagram of a simplified block diagram of a communication system (400) according to one embodiment. [Diagram 5] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 6] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Figure 7] FIG. 4 is a block diagram of an encoder according to another embodiment. [Figure 8] FIG. 4 is a block diagram of a decoder according to another embodiment. [Figure 9] A diagram illustrating example syntax for updating a palette prediction list in some examples. [Figure 10] FIG. 11 illustrates example syntax for determining the presence of several example palette mode flags in a bitstream. [Figure 11] 1 is a flowchart outlining an example process according to some embodiments of the present disclosure. [Figure 12] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] FIG. 3 shows a simplified block diagram of a communication system (300) according to an embodiment of the present disclosure. The communication system (300) includes a plurality of terminal devices capable of communicating with each other, for example, via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). In the example of FIG. 3, the first pair of terminal devices (310) and (320) perform unidirectional transmission of data. For example, the terminal device (310) can encode video data (e.g., a stream of video images captured by the terminal device (310)) for transmission to another terminal device (320) via the network (350). The encoded video data can be transmitted in the form of one or more encoded video bitstreams. The terminal device (320) can receive the encoded video data from the network (350), decode the encoded video data to reconstruct the video image, and display the video image according to the reconstructed video data. The unidirectional data transmission can be common in media serving applications, etc.

[0026] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) performing bidirectional transmission of encoded video data, such as may occur during a video conference. For the bidirectional transmission of data, in one example, each of the terminal devices (330) and (340) can encode video data (e.g., a stream of video images captured by the terminal device) for transmission to the other of the terminal devices (330) and (340) over the network (350). Each of the terminal devices (330) and (340) can also receive the encoded video data transmitted by the other of the terminal devices (330) and (340), can decode the encoded video data to recover the video images, and can display the video images on an accessible display device according to the recovered video data.

[0027] In the example of FIG. 3, the terminal devices (310), (320), (330), and (340) may be depicted as a server, a personal computer, and a smartphone, although the principles of the present disclosure may not be so limited. The embodiments of the present disclosure apply to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (350) represents any number of networks that convey encoded video data between the terminal devices (310), (320), (330), and (340), including, for example, wired and / or wireless communication networks. The communication network (350) may exchange data over circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of the network (350) may not be important to the operation of the present disclosure, unless otherwise described herein below.

[0028] 4 illustrates an arrangement of a video encoder and a video decoder in a streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter may be equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0029] The streaming system may include a capture subsystem (413) that may include a video source (401), such as a digital camera, that generates a stream of uncompressed video images (402). In one example, the stream of video images (402) includes samples captured by a digital camera. The stream of video images (402), shown as a thick line to emphasize the high amount of data compared to the encoded video data (404) (or encoded video bitstream), may be processed by an electronic device (420) that includes a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (404) (or encoded video bitstream (404)), shown as a thin line to emphasize the lower amount of data compared to the stream of video images (402), may be stored in a streaming server (405) for future use. One or more streaming client subsystems, such as the client subsystems (406) and (408) of FIG. 4, can access the streaming server (405) to obtain copies (407) and (409) of the encoded video data (404). The client subsystem (406) can include a video decoder (410), for example, in an electronic device (430). The video decoder (410) decodes an input copy (407) of the encoded video data and creates an output stream of video images (411) that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., a video bitstream) can be encoded according to a particular video encoding / compression standard. An example standard is ITU-T Recommendation H.265.In one example, a video coding standard under development is informally known as Universal Video Coding (VVC), and the disclosed subject matter may be used in the context of VVC.

[0030] It should be noted that the electronic devices (420) and (430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[0031] 5 shows a block diagram of a video decoder (510) according to one embodiment of the present disclosure. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuit). The video decoder (510) may be used in place of the video decoder (410) of the example of FIG. 4.

[0032] The receiver (531) may receive one or more coded video sequences to be decoded by the video decoder (510); in the same or another embodiment, one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel (501), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (531) may receive the coded video data with other data, e.g., coded audio data and / or auxiliary data streams, which may be transferred to each using an entity (not shown). The receiver (531) may separate the coded video sequences from the other data. To combat network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter "parser (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In other cases, the buffer memory (515) may be external to the video decoder (510) (not shown). In still other cases, there may be a buffer memory (not shown) external to the video decoder (510), for example to combat network jitter, and another buffer memory (515) internal to the video decoder (510), for example to handle playback timing. When the receiver (531) is receiving data from a store / forward device of sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory (515) may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer memory (515) may be needed and may be relatively large, advantageously of adaptive size, and may be implemented at least in part in an operating system or similar element (not shown) external to the video decoder (510).

[0033] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from the coded video sequence. These categories of symbols include information used to manage the operation of the video decoder (510) and potentially information for controlling a rendering device such as a rendering device (512) (e.g., a display screen) that is not an integral part of the electronic device (530) but may be coupled to the electronic device (530) as shown in FIG. 5. The control information for the rendering device(s) may be in the form of supplemental enhancement information (SEI message) or video usability information (VUI) parameter set fragments (not shown). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) can extract, from the coded video sequence, a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroups can include groups of pictures (GOPs), images, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (520) can also extract information from the coded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0034] The parser (520) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to produce symbols (521).

[0035] The reconstruction of the symbols (521) can involve several different units, depending on the type of coded video picture or part thereof (e.g., inter and intra pictures, inter and intra blocks), and other factors. Which units are involved and how may be controlled by subgroup control information parsed from the coded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the following units is not shown for clarity.

[0036] Beyond the functional blocks already mentioned, the video decoder (510) may be conceptually subdivided into several functional units, as described below. In an actual implementation operating under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

[0037] The first unit is a scalar / inverse transform unit (551), which receives quantized transform coefficients as well as control information from the parser (520) including which transform to use, block size, quantization factor, quantization scaling matrix, etc. as symbol(s) (521). The scalar / inverse transform unit (551) can output blocks containing sample values ​​that can be input to an aggregator (555).

[0038] In some cases, the output samples of the scalar / inverse transform (551) may relate to intra-coded blocks; i.e., intra-coded blocks that do not use prediction information from a previously reconstructed image, but can use prediction information from a previously reconstructed portion of the current image. Such prediction information may be provided by an intra-image prediction unit (552). In some cases, the intra-image prediction unit (552) generates a block of the same size and shape as the block being reconstructed using surrounding already reconstructed information fetched from a current image buffer (558). The current image buffer (558) buffers, for example, a partially reconstructed current image and / or a fully reconstructed current image. The aggregator (555) may add, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (552) to the output sample information provided by the scalar / inverse transform unit (551).

[0039] In other cases, the output samples of the scalar / inverse transform unit (551) may relate to an inter-coded, potentially motion-compensated block. In such cases, the motion compensated prediction unit (553) may access the reference picture memory (557) to fetch samples used for prediction. After motion compensating the fetched samples according to the symbols (521) related to the block, these samples may be added by the aggregator (555) to the output of the scalar / inverse transform unit (551) (in this case referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (557) from which the motion compensated prediction unit (553) fetches the prediction samples may be controlled by motion vectors available to the motion compensated prediction unit (553) in the form of symbols (521), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (557) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0040] The output samples of the aggregator (555) may be subject to various loop filtering techniques in a loop filter unit (556). The video compression techniques may include in-loop filter techniques controlled by parameters contained in the coded video sequence (also called coded video bitstream) and made available to the loop filter unit (556) as symbols (521) from the parser (520), but may also be responsive to meta-information obtained during decoding of previous (decoding order) parts of the coded image or coded video sequence, or to previously reconstructed, loop filtered sample values.

[0041] The output of the loop filter unit (556) may be a sample stream that can be output to a rendering device (512) and also stored in a reference image memory (557) for use in future inter-image prediction.

[0042] Once fully reconstructed, a particular encoded image may be used as a reference image for future predictions. For example, once an encoded image corresponding to a current image is fully reconstructed and the encoded image is identified as a reference image (e.g., by the parser (520)), the current image buffer (558) may become part of the reference image memory (557), and a new current image buffer may be relocated before beginning reconstruction of a subsequent encoded image.

[0043] The video decoder (510) may perform decoding operations according to a given video compression technique in a standard, such as ITU-T Rec. H.265. The encoded video sequence may comply with the syntax specified by the video compression technique or standard being used, in the sense that the encoded video sequence complies with both the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, the profile may select a particular tool from all tools available in the video compression technique or standard as the only tool available under that profile. Also, what is required for compliance may be that the complexity of the encoded video sequence is within the bounds defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may be further limited in some cases by a hypothetical reference decoder (HRD) specification and metadata for HRD buffer management signaled in the encoded video sequence.

[0044] In one embodiment, the receiver (531) can receive additional (redundant) data with the encoded video. The additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (510) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0045] 6 shows a block diagram of a video encoder (603) according to one embodiment of the present disclosure. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmission circuit). The video encoder (603) may be used in place of the example video encoder (403) of FIG.

[0046] The video encoder (603) can receive video samples from a video source (601) (which in the example of FIG. 6 is not part of the electronic device (620)) that can capture the video image(s) to be encoded by the video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0047] The video source (601) may provide a source video sequence to be encoded by the video encoder (603) in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (601) may be a storage device that stores pre-prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as a number of individual images that give motion when viewed in succession. The image itself may be organized as a spatial array of pixels, each pixel may contain one or more samples depending on the sampling structure, color space, etc. in use. Those skilled in the art can easily understand the relationship between pixels and samples. The following description focuses on samples.

[0048] According to one embodiment, the video encoder (603) can encode and compress images of a source video sequence into an encoded video sequence (643) in real-time or under any other time constraint required by the application. Enforcing an appropriate encoding rate is one function of the controller (650). In some embodiments, the controller (650) controls and is operatively coupled to other functional units as described below. Coupling is not shown for clarity. Parameters set by the controller (650) can include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) can be configured with other appropriate functions for the video encoder (603) optimized for a particular system design.

[0049] In some embodiments, the video encoder (603) is configured to operate in an encoding loop. As an oversimplified explanation, in one example, the encoding loop can include a source coder (630) (e.g., responsible for generating symbols, such as a symbol stream, based on an input image to be encoded and a reference image(s)) and a (local) decoder (633) built into the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a similar manner that the (remote) decoder also creates (since in the video compression techniques considered in the disclosed subject matter, any compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream (sample data) is input to a reference image memory (634). Since the decoding of the symbol stream results in bit-exact results independent of the decoder location (local or remote), the content in the reference image memory (634) is also bit-exact between the local and remote encoders. In other words, the predictor of the encoder "sees" exactly the same sample values ​​as the decoder "sees" when using the prediction during decoding. This basic principle of reference image synchrony (and the resulting drift if synchrony cannot be maintained, e.g., due to channel errors) is also used in several related technologies.

[0050] The operation of the "local" decoder (633) may be the same as that of a "remote" decoder, such as the video decoder (510) already described in detail in connection with Figure 5. However, with brief reference also to Figure 5, because symbols are available and the encoding / decoding of the symbols into a coded video sequence by the entropy coder (645) and parser (520) may be lossless, the entropy decoding portion of the video decoder (510), including the buffer memory (515), and the parser (520), may not be fully implemented in the local decoder (633).

[0051] An observation that can be made at this point is that any decoder techniques, except for parsing / entropy decoding, present in the decoder must also be present in substantially identical functional form in the corresponding encoder. For this reason, the disclosed subject matter focuses on the decoder operation. A description of the encoder techniques can be omitted since they are the inverse of the decoder techniques described generically. Only in certain areas are more detailed descriptions required and are provided below.

[0052] In operation, in some examples, the source coder (630) may perform motion-compensated predictive coding, which predictively codes an input image with reference to one or more previously coded images from a video sequence designated as “reference images.” In this manner, the coding engine (632) codes differences between pixel blocks of the input image and pixel blocks of the reference image(s) that may be selected as predictive reference(s) for the input image.

[0053] The local video decoder (633) may decode the encoded video data of the image that may be designated as a reference image based on the symbols generated by the source coder (630). The operation of the encoding engine (632) may preferably be a lossy process. When the encoded video data may be decoded in a video decoder (not shown in FIG. 6), the reconstructed video sequence may be a copy of the source video sequence, usually with some errors. The local video decoder (633) may replicate the decoding process that may be performed on the reference image by the video decoder and store the reconstructed reference image in a reference image cache (634). In this way, the video encoder (603) may locally store copies of reconstructed reference images that have common content as the reconstructed reference images that will be retrieved (without transmission errors) by the far-end video decoder.

[0054] The predictor (635) may perform the prediction search of the coding engine (632). That is, for a new image to be coded, the predictor (635) may search the reference image memory (634) for sample data (as candidate reference pixel blocks) or specific metadata of the reference images, such as motion vectors, block shapes, etc., that may serve as suitable prediction references for the new image. The predictor (635) may operate on a pixel block-by-block basis of samples to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (635), the input image may have prediction references drawn from multiple reference images stored in the reference image memory (634).

[0055] The controller (650) can manage the coding operations of the source coder (630), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0056] The output of all the aforementioned functional units may be subject to entropy coding in an entropy coder (645), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, or arithmetic coding.

[0057] The transmitter (640) may buffer the encoded video sequence(s) produced by the entropy coder (645) and prepare them for transmission over a communication channel (660), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (640) may merge the encoded video data from the video coder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0058] A controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can assign a particular encoded image type to each encoded image, which can affect the encoding technique that can be applied to the respective image. For example, images are often assigned as one of the following image types:

[0059] An intra-picture (I-picture) may be a picture that can be coded and decoded without using other pictures in a sequence as a source of prediction. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of these variations of I-pictures and their respective uses and characteristics.

[0060] A predicted image (P-image) may be an image that can be encoded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block.

[0061] A bidirectionally predicted image (B-image) may be an image that can be coded and decoded using intra- or inter-prediction that uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted images may use more than two reference images and associated metadata for the reconstruction of a single block.

[0062] A source image is generally spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks as determined by a coding assignment applied to the respective image of the block. For example, blocks of an I image may be non-predictively coded or predictively coded with reference to already coded blocks of the same image (spatial or intra prediction). Pixel blocks of a P image may be predictively coded via spatial prediction with reference to one previously coded reference image or via temporal prediction. Blocks of a B image may be predictively coded via spatial prediction with reference to one or two previously coded reference images or via temporal prediction.

[0063] The video encoder (603) may perform encoding operations according to a given video encoding technique or standard, such as ITU-T Rec. H.265. In its operations, the video encoder (603) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to a syntax specified by the video encoding technique or standard being used.

[0064] In one embodiment, the transmitter (640) can transmit additional data along with the encoded video. The source coder (630) can include such data as part of the coded video sequence. The additional data can include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0065] Video may be captured as multiple source images (video images) in a time sequence. Intra-image prediction (often abbreviated as intra-prediction) exploits spatial correlation in a given image, while inter-image prediction exploits correlation (temporal or other) between images. In one example, a particular image being encoded / decoded, called the current image, is divided into blocks. When a block in the current image is similar to a reference block in a previously coded and still buffered reference image in the video, the block in the current image may be coded by a vector called a motion vector. The motion vector points to a reference block in the reference image and may have a third dimension that identifies the reference image if multiple reference images are used.

[0066] In some embodiments, bi-prediction techniques can be used for inter-image prediction. According to bi-prediction techniques, two reference images, such as a first reference image and a second reference image, are used, both of which are prior to the decoding order of the current image in the video (although the display order may be past and future, respectively). A block in the current image may be coded by a first motion vector that points to a first reference block in the first reference image and a second motion vector that points to a second reference block in the second reference image. A block may be predicted by a combination of the first reference block and the second reference block.

[0067] Furthermore, to improve coding efficiency, merge mode techniques can be used for inter-picture prediction.

[0068] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a sequence of video images are divided into coding tree units (CTUs) for compression, and the CTUs in an image have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree partitioned into one or more coding units (CUs). For example, a CTU of 64×64 pixels can be partitioned into one CU of 64×64 pixels, or four CUs of 32×32 pixels, or 16 CUs of 16×16 pixels. In one example, each CU is analyzed to determine the prediction type of the CU, such as an inter prediction type or an intra prediction type. The CU is partitioned into one or more prediction units (PUs) depending on the temporal and / or spatial predictability. Generally, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values) of 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0069] 7 shows a diagram of a video encoder (703) according to another embodiment of the present disclosure. The video encoder (703) is configured to receive a processed block of sample values ​​(e.g., a predictive block) in a current video image in a sequence of video images and to encode the processed block into an encoded image that is part of an encoded video sequence. In one example, the video encoder (703) is used in place of the video encoder (403) of the example of FIG. 4.

[0070] In an HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a predictive block of 8×8 samples. The video encoder (703) determines whether the processing block is best coded using intra mode, inter mode, or bi-predictive mode, for example using rate-distortion optimization. If the processing block is coded in intra mode, the video encoder (703) may use intra prediction techniques to encode the processing block into a coded image; when the processing block is to be coded in inter mode or bi-predictive mode, the video encoder (703) may use inter prediction techniques or bi-predictive techniques, respectively, to encode the processing block into a coded image. In certain video coding techniques, the merge mode may be an inter-image prediction submode in which motion vectors are derived from one or more motion vector predictors without the benefit of coded motion vector components outside the predictors. In certain other video coding techniques, there may be motion vector components applicable to the current block. In one example, the video encoder (703) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0071] In the example of FIG. 7, the video encoder (703) includes an inter-encoder (730), an intra-encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725), coupled together as shown in FIG.

[0072] The inter-encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the block to one or more reference blocks in reference images (e.g., blocks in previous and subsequent images), generate inter-prediction information (e.g., a description of redundant information through inter-encoding techniques, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a prediction block) based on the inter-prediction information using any suitable technique. In some examples, the reference image is a decoded reference image that is decoded based on the encoded video information.

[0073] The intra encoder (722) is configured to receive samples of a current block (e.g., a processing block), possibly compare the block with blocks already encoded in the same image, generate quantized coefficients after transformation, and possibly also generate intra prediction information (e.g., intra prediction direction information according to one or more intra encoding techniques). In one example, the intra encoder (722) calculates an intra prediction result (e.g., a prediction block) based on the intra prediction information and a reference block in the same image.

[0074] The generic controller (721) is configured to determine generic control data and control other components of the video encoder (703) based on the generic control data. In one example, the generic controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, if the mode is an intra mode, the generic controller (721) controls the switch (726) to select an intra mode result to be used by the residual calculator (723) and controls the entropy encoder (725) to select intra prediction information to be included in the bitstream; if the mode is an inter mode, the generic controller (721) controls the switch (726) to select an inter prediction result to be used by the residual calculator (723) and controls the entropy encoder (725) to select inter prediction information to be included in the bitstream.

[0075] The residual calculator (723) is configured to calculate a difference (residual data) between a received block and a prediction result selected from the intra-encoder (722) or the inter-encoder (730). The residual encoder (724) is configured to operate on the residual data to encode the residual data to generate transform coefficients. In one example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain to generate transform coefficients. The transform coefficients are then subjected to a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data may be used by the intra-encoder (722) and the inter-encoder (730) as appropriate. For example, the inter-encoder (730) may generate decoded blocks based on the decoded residual data and the inter-prediction information, and the intra-encoder (722) may generate decoded blocks based on the decoded residual data and the intra-prediction information. In some examples, the decoded blocks may be appropriately processed to generate a decoded image, which may be buffered in a memory circuit (not shown) and used as a reference image.

[0076] The entropy encoder (725) is configured to format the bitstream to include the encoded block. The entropy encoder (725) is configured to include various information in accordance with an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. It should be noted that, according to the disclosed subject matter, when encoding a block in a merged sub-mode of either the inter-mode or the bi-prediction mode, the residual information is not present.

[0077] 8 shows a diagram of a video decoder (810) according to another embodiment of the present disclosure. The video decoder (810) is configured to receive encoded images that are part of an encoded video sequence and decode the encoded images to generate reconstructed images. In one example, the video decoder (810) is used in place of the video decoder (410) of the example of FIG. 4.

[0078] In the example of FIG. 8, the video decoder (810) includes an entropy decoder (871), an inter-decoder (880), a residual decoder (873), a reconstruction module (874), and an intra-decoder (872), coupled together as shown in FIG.

[0079] The entropy decoder (871) may be configured to reconstruct from the coded image certain symbols that represent the syntax elements from which the coded image is constructed. Such symbols may include, for example, prediction information (e.g., intra-mode, inter-mode, bi-predictive mode, the latter two being merged or separate submodes) that may identify the mode in which the block is coded, certain samples or metadata used for prediction by the intra-decoder (872) or the inter-decoder (880), respectively (e.g., intra-predictive information or inter-predictive information), residual information, e.g., in the form of quantized transform coefficients, etc. In one example, if the prediction mode is an inter-predictive mode or a bi-predictive mode, the inter-predictive information is provided to the inter-decoder (880); if the prediction type is an intra-predictive type, the intra-predictive information is provided to the intra-decoder (872). The residual information may be subject to inverse quantization and provided to the residual decoder (873).

[0080] The inter decoder (880) is configured to receive the inter prediction information and to generate inter prediction results based on the inter prediction information.

[0081] The intra decoder (872) is configured to receive the intra prediction information and to generate a prediction result based on the intra prediction information.

[0082] The residual decoder (873) is configured to perform inverse quantization to extract inverse quantized transform coefficients and process the inverse quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information (to include quantizer parameters (QP)), which may be provided by the entropy decoder (871) (the data path not shown in this manner may be only low volume control information).

[0083] The reconstruction module (874) is configured to combine, in the spatial domain, the residual as output by the residual decoder (873) and the prediction result (possibly as output by an inter- or intra-prediction module) to form a reconstructed block, which may be part of a reconstructed image, which may be part of a reconstructed video. It should be noted that other suitable operations, such as a deblocking operation, may be performed to improve visual quality.

[0084] It should be noted that the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using any suitable technology. In one embodiment, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603), and (603) and the video decoders (410), (510), and (810) may be implemented using one or more processors executing software instructions.

[0085] According to some aspects of the present disclosure, a coding mode called a palette-based coding mode may be used in addition to the inter-prediction and intra-prediction modes. The present disclosure provides techniques to simplify the palette-based coding mode using a local dual-tree coding structure.

[0086] In some instances, screen content has specific characteristics and coding tools for screen coding can be developed. The coding tools for screen coding can achieve significant improvements in coding efficiency. Palette-based coding modes can represent blocks of pixels using an index into a palette that contains the primary color of the block. The palette and index can be encoded by exploiting spatial redundancy.

[0087] Specifically, in some embodiments, a palette is defined as a look-up table of entries that can associate an index with a pixel value. In some examples, a pixel value includes one or more values ​​that correspond to color components. In one example, a pixel value can include three values ​​that respectively represent three color components. In another example, for a monochrome palette, a pixel value includes a value that represents a single color component.

[0088] In some embodiments, pixel blocks may be encoded based on a palette, and a coding mode that uses a palette is referred to as a palette-based coding mode. Palette-based coding modes are adopted in some video coding standards, such as VVC.

[0089] According to some embodiments, the palette-based coding mode is a stand-alone coding mode. In one embodiment, the palette-based coding mode is implemented as a coding unit (e.g., palette-based encoder, palette-based decoder, etc.) separate from the intra encoder (722), the inter encoder (730), the intra decoder (872), and the inter decoder (870). In one example, the palette-based coding mode is implemented as a hardware circuit separate from the intra encoder (722), the inter encoder (730), the intra decoder (872), and the inter decoder (870). In another example, the palette-based coding mode is implemented as a software module separate from the software modules for the intra encoder (722), the inter encoder (730), the intra decoder (872), and the inter decoder (870). The software module for the palette-based coding mode and other software modules may be executed by the same processor or different processors.

[0090] According to some aspects of the present disclosure, a palette-based encoding mode generally includes two parts: a first part of an encoding method for the palette and a second part of an encoding method for samples that use the palette. The first part can include primary color selection and palette encoding. The second part can include palette index encoding, run-length encoding, and escape pixel encoding.

[0091] In some examples, a palette-based encoder may encode a block of video data by determining a palette for the block (e.g., explicitly encoding the palette, predicting the palette, a combination of signaling and prediction, etc.), identifying an entry in the palette that represents one or more pixel values, and encoding the block with an index value that indicates the entry in the palette that is used to represent the pixel values ​​of the block. In some examples, the palette-based encoder may signal the palette and / or the index value in the encoded bitstream. The palette-based decoder may then obtain the palette for the block, as well as index values ​​of individual pixels of the block, from the encoded bitstream. In some other examples, the palette may be predicted, and the palette-based encoder may signal index values ​​in the encoded bitstream without signaling the palette, and the palette-based encoder may predict the palette and receive index values ​​in the encoded bitstream. The palette-based decoder may then associate index values ​​for the pixels with entries in the palette that provide the pixel values. The palette-based decoder may reconstruct various pixels of the block based on the pixel values ​​associated with the index values.

[0092] According to some embodiments, the palette may include the most dominant pixel values ​​in a given block. For example, the most dominant pixel values ​​may include one or more pixel values ​​that occur most frequently in a given block. Furthermore, in some examples, a threshold value (e.g., the number of pixels having a pixel value) may be used to determine whether a pixel value should be included as one of the most dominant pixel values ​​in the block.

[0093] In some embodiments, a histogram-based algorithm is used to classify pixel values ​​of pixels in a block to form a palette. In some examples, the most significant L peak values ​​in the histogram may be selected as the dominant colors, where L is a positive integer. In one example, pixel values ​​close to the dominant colors may be quantized to the dominant colors. In some examples, pixels that do not belong to the dominant colors are called escape pixels. The escape pixels may be quantized before encoding. In some embodiments, for lossless encoding, the quantization process is not performed.

[0094] Furthermore, in some embodiments, for each pixel, a color index is assigned to indicate which color the pixel belongs to. In some examples, if L primary colors are used, 0~(L-1) can be used as the color index of the primary colors, and 0~(L-1) can be called the primary color set. For pixels where there is no primary color index, a special index (e.g., index N~(L-1) other than 0) is assigned, and these pixels are called "escape pixels."

[0095] Generally, a palette is implemented as a color lookup table. The color lookup table contains multiple entries. Each entry associates a color index with a color (e.g., a primary color). In some examples, a single palette is used to encode both luma and chroma components. Each entry can represent a particular RGB (YUV) color using three color components. For example, a palette can have entry 1 with (R,G,B)=(0,0,0) representing a pure black color, and entry 0 with (R,G,B)=(2,10,200) representing a bluish color. Note that when the video format is 420, the chroma planes are upsampled to generate the color lookup table for the palette.

[0096] In some embodiments, palette-based encoding is performed on a per-CU basis. In one example, for a current CU, a color lookup table is derived that includes the most dominant pixel values ​​in the current CU. In some examples, the size and elements of the color lookup table may be transmitted. In some other examples, the size and elements of the color lookup table may be predictively encoded using the size and / or elements of the color lookup tables of other CUs.

[0097] According to some aspects of the present disclosure, palette prediction can be performed based on a palette predictor list. In one example, the palette predictor list can include multiple palettes used for other CUs. To encode the current palette using the palette predictor list as a reference, a binary vector is used to indicate whether each entry in the palette predictor list is reused in the current palette.

[0098] In some examples (e.g., HEVC SCC), the palette predictor list stores previously encoded palette entries as references to predict the current palette. The palette predictor list is updated after each palette mode CU. In one example, the palette predictor list behaves similarly to a least recently used cache. The most recent palette can be inserted at the beginning of the palette predictor list, and entries from the furthest CU in the scan order are discarded if the list size exceeds a threshold. In some examples, the maximum allowed palette predictor size and palette size are signaled in a high-level syntax or as a defined value agreed upon by both the encoder and the decoder. In one embodiment, an upper limit on the size of the palette predictor list is signaled (directly or indirectly) in the SPS. For example, the palette predictor list can have a size of about twice the size limit of the palette. In some examples, the palette predictor list is also referred to as a history palette predictor list (HPPL). In one example, the maximum predictor palette size is signaled to be 63 and the maximum palette size is signaled to be 31 in the SPS. In another example, these two values ​​are assumed by the encoder and decoder without signaling.

[0099] In some embodiments, the entries in the palette prediction list may each have a reuse flag. If the reuse flag of an entry is true (e.g., "1"), the entry is called a reuse entry. The color components of a reuse entry in the palette prediction list may be copied to the color lookup table of the current palette. The reuse entries are placed at the beginning of the current palette to maintain their order in the palette predictor list. A reuse entry in the current palette may be followed by a new palette entry that is not in the palette predictor list. A new palette entry may be signaled. Each new palette entry may include, for example, three color components.

[0100] In some embodiments, a palette initializer with predefined entries can be used to initialize the palette predictor list. Using a palette initializer can improve coding efficiency in some scenarios. The palette initializer can be signaled, for example, in a picture parameter set (PPS), a sequence parameter set (SPS), etc.

[0101] The color index may be encoded as follows: After classification, the pixels of the block may be converted to color indexes according to the selected primary color set. In some examples, a predictive coding method may be applied to the color index, and a pixel line may be predicted by a number of different modes (e.g., three different modes), including a horizontal mode (e.g., copy index mode), a vertical mode (e.g., copy above mode), and a normal mode (e.g., escape mode). In some examples, two index scan orders (e.g., horizontal traverse scan and vertical traverse scan) are used when encoding the color index. An index rotation flag may be signaled to indicate which of the two index scan orders is used.

[0102] In copy index mode, starting from the first pixel, one or more consecutive indices may be copied from the first pixel. The color index of the first pixel may be signaled.

[0103] In copy-above mode, one or more consecutive color indices may be copied from a pixel line above, for example a pixel line above the current pixel line.

[0104] In escape mode, when an escape pixel is encountered that is signaled by, for example, a maximum index (e.g., N) in a primary color set, the corresponding pixel value may be coded after the maximum index (e.g., N). There may be multiple escape pixels with different color values ​​in a CU. For different escape pixel locations, the pixel values ​​of the escape pixels may be different.

[0105] For each copy index mode, an index value can be signaled. Index signaling can be grouped forward (or at the beginning) to improve, for example, context-adaptive binary arithmetic coding (CABAC) throughput. Similarly, pixel values ​​of escape pixels can be signaled backwards to improve, for example, CABAC throughput. Copy index and copy above modes can be signaled between index and escape coding.

[0106] In one embodiment, the coding tree scheme supports the ability for luma components and corresponding chroma components(s) to have separate block tree structures. In one example, for P slices and B slices, the luma CTB and chroma CTB in a CTU share the same coding tree structure (e.g., single tree). For I slices, the luma CTB and chroma CTB in a CTU may have separate block tree structures (e.g., dual tree), and the partition case of a CTU using separate block tree structures is called dual tree partition. In one example, when dual tree partition is applied, the luma CTB may be partitioned into luma CUs by the luma coding tree structure, and the chroma CTB may be partitioned into chroma CUs by the chroma coding tree structure.

[0107] In some examples, such as JVET, samples of chroma components may have an independent or separate split tree structure (or coding tree structure) compared to luma components. The separate coding tree structure may start from the CTU level, and in some examples, the dual tree structure from the CTU level is called a global dual tree structure. In one example, a chroma CU (e.g., a CU that includes only two chroma components) is larger than its luma counterpart at the corresponding sample position.

[0108] In some examples of CTU level dual tree (global dual tree structure), the maximum allowed predictor palette size and palette size are reduced, for example, by half, resulting in reduced complexity for each channel (luma or chroma). For example, in the case of single tree coding, the luma and chroma palette coding blocks are coded together, and the maximum allowed predictor palette size is 63 entries and the maximum allowed palette size is 31 entries. In the case of dual tree coding, for the luma palette coding block, the maximum allowed predictor palette size is 31 entries and the maximum allowed palette size is 15 entries; for the chroma palette coding block, the maximum allowed predictor palette size is 31 entries and the maximum allowed palette size is 15 entries.

[0109] According to some aspects of the present disclosure, a technique called local dual tree is used to avoid using small chroma blocks (such as CUs with chroma samples smaller than 4×4). In one example, the local dual tree technique may be triggered when some conditions are met (such as the parent CU size being below some threshold that may cause chroma blocks to be smaller than 4×4 samples). The luma and chroma coding of the CU can then be separated in a manner similar to the dual tree at the CTU level.

[0110] According to one aspect of the present disclosure, when a split tree structure is used, the luma component may be coded in one of an intra prediction mode, an IBC mode, and a palette-based coding mode, and the chroma component may be coded in one of an intra prediction mode and a palette-based coding mode. However, the use of a local dual tree complicates the operation in the palette-based coding mode. For example, a block of the local dual tree structure may have a neighboring block in a single tree structure. The same palette predictor list may be used by a block of the local dual tree structure and a neighboring block in the single tree structure. Updating the palette predictor list may be complicated.

[0111] Aspects of the present disclosure provide techniques for simplifying palette-based coding using a local dual-tree coding structure.

[0112] According to one aspect of the present disclosure, if certain conditions are true, the palette predictor list update process is disabled or bypassed for palette mode encoding. In some examples, a local dual tree may be triggered and the palette predictor list update process may be disabled if a block in the local dual tree structure is encoded in a palette-based encoding mode. Thus, for luma blocks encoded in a palette-based encoding mode and / or chroma blocks encoded in a palette-based encoding mode under the local dual tree structure, the palette entries of those blocks are not used to update the palette entries in the palette predictor list.

[0113] In one embodiment, a trigger of the local dual tree can be detected. In one example, when the local dual tree is triggered, the palette predictor list is not updated based on the chroma blocks coded in the palette-based coding mode under the local dual tree structure, and is not updated based on the luma blocks coded in the palette-based coding mode under the local dual tree structure. In another example, when the local dual tree is triggered, the palette predictor list is not updated based on the luma blocks coded in the palette-based coding mode under the local dual tree structure, and may be updated based on the chroma blocks coded in the palette-based coding mode under the local dual tree structure. In another example, when the local dual tree is triggered, the palette predictor list is not updated based on the chroma blocks coded in the palette-based coding mode under the local dual tree structure, and may be updated based on the luma blocks coded in the palette-based coding mode under the local dual tree structure.

[0114] Specifically, in one example, a variable denoted localDualTree is used to detect whether the local dual tree can be triggered. In one example, the variable localDualTree is derived according to (Equation 1): localDualTree=treeType !=SINGLE_TREE&& (slice_type !=I | |(slice_type==I&&qtbtt_dual_tree_intra_flag==0))?1:0(Formula 1) Here, treeType represents the type of split tree structure (e.g., SINGLE_TREE, DUAL_TREE_LUMA, DUAL_TREE_CHROMA, etc.), slice_type represents the type of the current slice (e.g., I, P or B), and qtbtt_dual_tree_intra_flag represents the flag for the global dual tree. If the variable localDualTree is equal to 1, the local dual tree can be triggered for detection; if the variable localDualTree is equal to 0, the local dual tree is not triggered.

[0115] 9 illustrates an example syntax (900) for updating the palette prediction list in some examples. In some examples, if localDualtree is equal to 0, the palette prediction list may be updated according to the example syntax (900); if localDualtree is equal to 1, the update process according to the example syntax (900) is skipped.

[0116] In one example, for a palette coding block under a single tree structure, the palette prediction list can be updated based on the block. For example, treeType is equal to SINGLE_TREE, so localDualtree is equal to 0. Also, a variable startComp (e.g., the start color component) is set to 0, and a variable numComps (e.g., the number of color components) is set to 1 if only the luma component is coded, and to 3 if both the luma and chroma components are coded. Then, the size of the palette predictor list (e.g., represented by PredictorPaletteSize) and the entries of the palette predictor list (e.g., represented by predictorPaletteEntries) can be updated according to the syntax (900).

[0117] In another example, for a palette-coded luma block under a global dual tree, the palette prediction list can be updated based on the block. For example, for a CTU level dual tree (global dual tree), treeType is equal to DUAL_TREE_LUMA, slice_type is equal to I, and the global dual tree flag qtbtt_dual_tree_intra_flag is equal to 1, so localDualtree is equal to 0. Also, variables startComp is set to 0 and numComps is set to 1. Then, according to syntax (900), the size of the palette predictor list (e.g., represented by PredictorPaletteSize) and the entries of the palette predictor list (e.g., represented by predictorPaletteEntries) can be updated.

[0118] In another example, for a palette-coded chroma block under a global dual tree, the palette prediction list can be updated based on the block. For example, for a CTU level dual tree (global dual tree), treeType is equal to DUAL_TREE_CHROMA, slice_type is equal to I, and the global dual tree flag qtbtt_dual_tree_intra_flag is equal to 1, so localDualtree is equal to 0. Also, variables startComp is set to 1 and numComps is set to 2. Then, according to syntax (900), the size of the palette predictor list (e.g., represented by PredictorPaletteSize) and the entries of the palette predictor list (e.g., represented by predictorPaletteEntries) can be updated.

[0119] In another example, for a palette-coded chroma block under the local dual tree, the update of the palette prediction list can be skipped. For example, treeType is equal to DUAL_TREE_CHROMA, slice_type is equal to I, and the global dual tree flag qtbtt_dual_tree_intra_flag is equal to 0, so localDualtree is equal to 1. Then, the update process according to the example syntax (900) is skipped.

[0120] According to another aspect of the present disclosure, palette-based coding modes are not allowed for small blocks when a coding configuration that can trigger the use of a local dual tree is in use, thus avoiding the complication of palette predictor list updates in the case of a local dual tree.

[0121] In one example, a local dual tree can be triggered (e.g., localDualTree is equal to 1) if slice_type is not equal to I, or if slice_type is equal to I but a slice level flag (e.g., qtbtt_dual_tree_intra_flag) that enables a (global) dual tree is equal to 0. When a local dual tree is triggered, small blocks, such as coding units whose width x height < threshold, are not allowed to be coded by palette-based coding modes. In one embodiment, a threshold is defined such that if the block size of the current coding tree exceeds this threshold, the local dual tree is not used. In some examples, palette-based coding modes are not allowed for small luma blocks and small chroma blocks under the local dual tree structure. In some examples, palette-based coding modes are not allowed only for small chroma blocks under the local dual tree structure.

[0122] Figure 10 shows an example syntax for determining the presence of a flag (pred_mode_plt_flag) in the bitstream that indicates whether a palette-based coding mode is used for the current block. In the example of Figure 10, part (1001) is used to apply size constraints, and part (1002) is used to apply chroma block and local dual tree constraints. For example, if the block is small (e.g., width x height is smaller than 16 for luma blocks, or width x height is smaller than 16 x SubWidthC x SubHeightC for chroma blocks), part (1001) can be 0, and thus the pre_mode_plt_flag of the block is not present in the bitstream, and palette-based coding mode is not allowed for the block.

[0123] In one example, for a chroma block under a local dual tree structure, the portion (1002) may be 0 since the modeType is equal to MODE_TYPE_INTRA and the treeType is equal to DUAL_TREE_CHROMA. Thus, the pred_mode_plt_flag for the chroma block is not present in the bitstream, and thus, a palette-based coding mode is not allowed for the chroma block. In another example, for a chroma block under a global dual tree, the portion (1002) may be 1 since the modeType is equal to MODE_TYPE_ALL and the treeType is equal to DUAL_TREE_CHROMA. Thus, the chroma block under the global dual tree structure may be coded in a palette-based coding mode. In another example, for a luma block under a local dual tree structure, the portion (1002) may be 1 since the modeType is equal to MODE_TYPE_INTRA and the treeType is equal to DUAL_TREE_LUMA. Thus, the luma block under the local dual tree structure may be coded in a palette-based coding mode.

[0124] According to another aspect of the present disclosure, when a local dual tree is used, the maximum allowed predictor palette size and palette size are reduced, for example by half. Reducing the predictor palette size and palette size can reduce the complexity of the update. In some examples, for each local region (e.g., a local dual tree) that allows luma and chroma blocks to be coded separately (smaller than a CTU), the maximum allowed predictor palette size and palette size are set to half of the joint palette case (using a palette with both luma and chroma components).

[0125] In one embodiment, the maximum allowed predictor palette size and palette size are 63 and 31, respectively, in the joint palette case. Then, under the local dual tree structure, the maximum allowed predictor palette size and palette size for the palette-based coding mode of the luma block are set to 31 and 15, respectively; the maximum allowed predictor palette size and palette size for the palette-based coding mode of the chroma block are set to 31 and 15, respectively.

[0126] FIG. 11 shows a flow chart outlining a process (1100) according to an embodiment of the present disclosure. The process (1100) may be used for block reconstruction. In various embodiments, the process (1100) is performed by a processing circuit such as the processing circuit of the terminal devices (310), (320), (330) and (340), a processing circuit performing the function of the video encoder (403), a processing circuit performing the function of the video decoder (410), a processing circuit performing the function of the video decoder (510), a processing circuit performing the function of the video encoder (603), etc. In some embodiments, the process (1100) is implemented in software instructions, and thus the processing circuit performs the process (1100) when the processing circuit executes the software instructions. The process starts at (S1101) and proceeds to (S1110).

[0127] At (S1110), prediction information for the block is decoded from the coded video bitstream.

[0128] At (S1120), whether to allow a palette-based coding mode for the block is determined based on a comparison of the size of the block with a threshold.

[0129] At (S1130), it is determined whether the block is under the local dual tree structure.

[0130] At (S1140), the block is decoded based on a determination of whether a palette-based coding mode of the block is allowed and a determination of whether the block is under a local dual tree structure.

[0131] In some embodiments, in response to the size of the block being smaller than a threshold, the palette-based coding mode of the block is not allowed. For example, if condition (1001) is not satisfied for the block, the palette-based coding mode is not allowed. In some examples, a threshold is defined such that the local dual tree structure is not used if the block is larger than the threshold.

[0132] In one embodiment, in response to the block being a chroma block and the size of the chroma block being less than a threshold, a palette-based coding mode for the block is not allowed, In one example, a palette-based coding mode may be used for small luma blocks but not for small chroma blocks.

[0133] In another embodiment, palette-based coding mode is not allowed for a block based on a combination of the block's mode type and tree type that indicates the block is a chroma block under a local dual tree structure. In one example, to decode a block, if the mode type is equal to MODE_TYPE_INTRA, the tree type is equal to DUAL_TREE_CHROMA, and the value of the condition (1002) is equal to 0, palette-based coding mode of the block is not allowed.

[0134] In some embodiments, when the palette-based coding mode is allowed, a flag (e.g., pred_mode_plt_flag) is decoded indicating whether the palette-based coding mode is used for the block. In response to the flag indicating the use of the palette-based coding mode for the block, the block is decoded using the palette-based coding mode. In some embodiments, in response to the block being under a local dual tree structure, updating the palette predictor list may be skipped or bypassed.

[0135] In some examples, the palette predictor list is updated based on the palette used in decoding the block update in response to no triggering of the local dual tree structure. In one example, the variable localDualTree is used to detect whether the local dual tree is triggered. For example, if the variable localDualTree is 0, updating of the palette predictor list based on the current palette of the block can be performed. In one example, if the variable localDualTree is 1, updating of the palette predictor list based on the current palette of the block is skipped. In another example, if the variable localDualTree is 1, the palette predictor list can be updated when the block is a luma block and is skipped when the block is a chroma block.

[0136] In another embodiment, the blocks under the local dual tree structure, the maximum size of the palette, and the maximum size of the palette predictor list are reduced, for example, by half. In some examples, if the palette-based coding mode is allowed for the block, a flag indicating whether the palette-based coding mode is used for the block is decoded. Then, if the flag indicates the use of the palette-based coding mode for the block and the block is under the local dual tree structure, the block is decoded based on a palette with the maximum size of the palette reduced. For example, the normal maximum size of the palette is 31, and the reduced maximum size of the palette is 15. Furthermore, the palette predictor list is updated based on the palette used to decode the block, and the size of the palette predictor list is limited by the reduced maximum size of the palette predictor list. For example, the normal maximum size of the palette predictor list is 63, and the reduced maximum size of the palette predictor list is 31.

[0137] The process then proceeds to (S1199) and ends.

[0138] The techniques described above may be implemented using computer readable instructions and as computer software physically stored on one or more computer readable media. For example, Figure 12 illustrates a computer system (1200) suitable for implementing certain embodiments of the disclosed subject matter.

[0139] Computer software may be encoded using any suitable machine code or computer language that may be subject to mechanisms such as assembly, compilation, linking, etc. to create code including instructions that can be executed directly or indirectly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., via microcode execution, etc.

[0140] The instructions may be executed on various types of computers or components thereof including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, and the like.

[0141] 12 for computer system (1200) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. The arrangement of components should not be interpreted as having a dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of computer system (1200).

[0142] The computer system (1200) may include certain human interface input devices. Such human interface input devices may be responsive to input by one or more human users via, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), olfactory input (not shown). Human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), video (e.g., two-dimensional video, three-dimensional video including stereoscopic video), etc.

[0143] The input human interface devices may include one or more (only one of each) of a keyboard (1201), a mouse (1202), a trackpad (1203), a touch screen (1210), a data glove (not shown), a joystick (1205), a microphone (1206), a scanner (1207), and a camera (1208).

[0144] The computer system (1200) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, by haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touch screen (1210), data gloves (not shown), or joystick (1205), although there may also be haptic feedback devices that do not function as input devices), audio output devices (e.g., speakers (1209), headphones (not shown)), visual output devices (e.g., screens (1210), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capability, each with or without haptic feedback capability, some of which may be capable of outputting two-dimensional visual output or three-dimensional or higher output via means such as stereo output; virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0145] The computer system (1200) may also include human-accessible storage devices and their associated media, such as optical media (1220) including CD / DVD ROM / RW with media (1221) such as CDs / DVDs, thumb drives (1222), removable hard drives or solid state drives (1223), legacy magnetic media such as tapes and floppy disks (not shown), dedicated ROM / ASIC / PLD based devices such as security dongles (not shown), etc.

[0146] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0147] The computer system (1200) may also include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial including CANBus, etc. Certain networks generally require an external network interface adapter attached to a particular general-purpose data port or peripheral bus (1249) (e.g., a USB port of the computer system (1200)); others are generally integrated into the core of the computer system (1200) by attachment to a system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1200) may communicate with other entities. Such communications may be, for example, one-way receive only (e.g., broadcast TV), one-way transmit only (e.g., CANbus to a particular CANbus device), or two-way, to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks may be used in each of these networks and network interfaces, as described above.

[0148] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be attached to the core (1240) of the computer system (1200).

[0149] The cores (1240) may include one or more central processing units (CPUs) (1241), graphics processing units (GPUs) (1242), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (1243), hardware accelerators for specific tasks (1244), and the like. These devices may be connected via a system bus (1248), along with read-only memory (ROM) (1245), random access memory (1246), and internal mass storage (1247), such as an internal non-user accessible hard drive or SSD. In some computer systems, the system bus (1248) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, and the like. Peripheral devices may be attached directly to the core's system bus (1248) or via a peripheral bus (1249). Peripheral bus architectures include PCI, USB, and the like.

[0150] The CPU (1241), GPU (1242), FPGA (1243), and accelerator (1244) can execute certain instructions that, in combination, may constitute the computer code described above. That computer code may be stored in ROM (1245) or RAM (1246). Transient data may also be stored in RAM (1246), while persistent data may be stored, for example, in internal mass storage (1247). Rapid storage and retrieval in any of the memory devices may be made possible by the use of cache memories that may be closely associated with one or more of the CPU (1241), GPU (1242), mass storage (1247), ROM (1245), RAM (1246), etc.

[0151] The computer-readable medium can bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0152] By way of example and not limitation, a computer system having the architecture (1200), and in particular the cores (1240), can provide functionality as a result of one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be user-accessible mass storage as described above, as well as media associated with specific storage of the cores (1240) of a non-transitory nature, such as the core internal mass storage (1247) or ROM (1245). Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the cores (1240). The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can cause the cores (1240), and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.) to perform certain processes or certain portions of certain processes described herein, including defining data structures stored in RAM (1246) and modifying such data structures according to the processes defined by the software. Additionally, or alternatively, the computer system may provide functionality as a result of logic implemented in hardwired or otherwise circuitry (e.g., accelerator (1244)), operating in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software may encompass logic, and vice versa, where appropriate. References to computer-readable media may encompass circuitry (such as integrated circuits (ICs)) that stores software for execution, circuitry that implements logic for execution, or both, as appropriate. The present disclosure encompasses any appropriate combination of hardware and software. Appendix A: Acronyms JEM: Joint Exploration Model VVC: General Purpose Video Coding BMS: Benchmark Set MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplemental Enhancement Information VUI: Video Usability Information GOP: Group of Pictures TU: conversion unit PU: Prediction unit CTU: Coding Tree Unit CTB: coding tree block PB: Predicted block HRD: Virtual Reference Decoder SNR: Signal to Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: cathode ray tube LCD: Liquid crystal display OLED: Organic Light Emitting Diode CD:Compact Disc DVD: Digital Video Disc ROM: Read-Only Memory RAM: Random Access Memory ASIC: Application Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile Communications LTE: Long Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Area SSD: Solid State Drive IC: Integrated Circuit CU: Coding Unit

[0153] While this disclosure has described several exemplary embodiments, there are alterations, substitutions, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope. [Explanation of symbols]

[0154] 101 Samples 201 Schematic diagram 300 Communication Systems 310 Terminal Devices 320 Terminal Devices 330 Terminal Devices 340 Terminal Devices 350 Network 400 Communication Systems 401 Video Source 402 Video Images 403 Video Encoder 404 Encoded Video Data 405 Streaming Server 406 Client Subsystem 407 Copy of encoded video data 408 Client Subsystem 409 Copy of encoded video data 410 Video Decoder 411 Video Images 412 Display 413 Capture Subsystem 420 Electronic Devices 430 Electronic Devices 501 Channel 510 Video Decoder 512 Rendering Device 515 Buffer Memory 520 Parser 521 Symbols 530 Electronic Devices 531 Receiver 551 Scaler / Descaler Unit 552 Intra Image Prediction Unit 553 Motion Compensation Prediction Unit 555 Aggregator 556 Loop Filter Unit 557 Reference Image Memory 558 Current Image Buffer 601 Video Sources 603 Video Encoder 620 Electronic Devices 630 Source Coder 632 encoding engine 633 Decoder 634 Reference Image Memory 635 Predictor 640 Transmitter 643 coded video sequence 645 Entropy Coder 650 Controller 660 Channels 703 Video Encoder 721 General-purpose controller 722 Intra Encoder 723 Residual Calculator 724 Residual Encoder 725 Entropy Encoder 726 Switch 728 Residual Decoder 730 InterEncoder 810 Video Decoder 871 Entropy Decoder 872 Intra Decoder 873 Residual Decoder 874 Reconstruction Module 880 Interdecoder 900 Syntax 1001 Part, condition 1002 Part, condition 1200 Computer Systems 1201 Keyboard 1202 Mouse 1203 Trackpad 1205 Joystick 1206 Microphone 1207 Scanner 1208 Camera 1209 Speaker 1210 Touch Screen 1220 Optical media 1221 CD / DVD and other media 1222 Thumb Drive 1223 Removable Hard Drive or Solid State Drive 1240 cores 1241 CPU 1242 GPU 1243 FPGA 1244 Accelerator 1245 ROM 1246 Random Access Memory 1247 Mass Storage 1248 System Bus 1249 Surrounding bus

Claims

1. A method of video encoding performed by an encoder, comprising: generating prediction information for a block and determining a size of said block; a first step of determining whether the size of the block is less than or equal to a threshold; a second step of determining whether the block is under a local dual tree structure based on the prediction information; a third step of determining whether the block is a chroma block; signaling a first flag indicating whether a palette mode is used for the block based on a result of at least one of the first step, the second step, and the third step; A method comprising:

2. The step of signaling the first flag comprises: signaling the first flag if the block is not the chroma block and the product of the width and height of the block is greater than 16; 2. The method of claim 1, further comprising: signaling the first flag on the condition that, if the block is the chroma block, a value obtained by multiplying the width and height of the block is greater than 16 multiplied by a predetermined constant.

3. The method described in claim 1, wherein the step of signaling the first flag signals the first flag under the condition that the block is not the chroma block under the local dual tree structure.

4. A method of video encoding performed by an encoder, comprising: generating and transmitting an encoded video bitstream; The generating and transmitting of the encoded video bitstream comprises: generating prediction information for a block and determining a size of said block; a first step of determining whether the size of the block is less than or equal to a threshold; a second step of determining whether the block is under a local dual tree structure based on the prediction information; a third step of determining whether the block is a chroma block; including, in the coded video bitstream, a first flag indicating whether a palette mode is used for the block based on a result of at least one of the first step, the second step, and the third step; A method comprising:

5. A method of video decoding performed by a decoder, comprising: decoding prediction information for the block from the encoded video bitstream; a first step of determining whether the size of the block is less than or equal to a threshold; a second step of determining whether the block is under a local dual tree structure based on the prediction information; a third step of determining whether the block is a chroma block; decoding, from the coded video bitstream, a first flag indicating whether a palette mode is used for the block based on a result of at least one of the first step, the second step, and the third step; A method comprising:

6. The step of decoding the first flag includes: if the block is not the chroma block, decoding the first flag if a product of the width and height of the block is greater than 16; The method of claim 5 , further comprising: decoding the first flag on condition that, if the block is the chroma block, a value obtained by multiplying the width and height of the block is greater than a value obtained by multiplying 16 by a predetermined constant.

7. The method described in claim 5, wherein the step of decoding the first flag decodes the first flag under the condition that the block is not the chroma block under the local dual tree structure.

8. An apparatus configured to perform a method according to any one of claims 1 to 7.

9. A program for causing a computer to execute a method according to any one of claims 1 to 7.

Images