Method and apparatus for encoding video data in palette mode

Abstract

To provide apparatuses and methods for performing palette coding of video data.SOLUTION: Methods include: determining whether a luma component of a coding unit (CU) and chroma components of the CU are coded jointly or separately in a palette mode; and in response to the luma component and the chroma components being coded jointly in the palette mode, determining a first maximum palette-table size for the CU; determining a first maximum palette-predictor size for the CU; and predicting the CU based on the first maximum palette-table size and the first maximum palette-predictor size.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] Cross - reference to Related Applications

[0001] This disclosure claims priority to U.S. Provisional Patent Application No. 62 / 954,843, filed on December 30, 2019, which is incorporated herein by reference in its entirety.

[0002] Technical Field

[0002] This disclosure generally relates to video processing, and more particularly, to methods and apparatuses for signaling and determining a maximum palette table size and a maximum palette predictor size based on an encoding tree structure for luma and chroma components in a palette mode.

Background Art

[0003] Background

[0003] Video is a series of still pictures (or "frames") that capture visual information. To reduce memory storage and transmission bandwidth, video can be compressed before storage or transmission and restored before display. The compression process is usually called encoding, and the restoration process is usually called decoding. Most commonly, there are various video encoding formats that use standardized video encoding techniques based on prediction, transformation, quantization, entropy encoding, and in - loop filtering. Video encoding standard specifications such as the HEVC (High Efficiency Video Coding) / H.265 standard specification, the VVC (Versatile Video Coding) / H.266 standard specification, and the AVS standard specification, which specify a particular video encoding format, have been developed by standardization organizations. As more advanced video encoding techniques are adopted in video standard specifications, the encoding efficiency of new video standard specifications becomes higher and higher.

Summary of the Invention

Means for Solving the Problems

[0004] Summary of the Disclosure

[0004] In some embodiments, the exemplary palette coding method includes determining whether the luma component of a coding unit (CU) and the chroma component of the CU are coded jointly or individually in palette mode; determining a first maximum palette table size for the CU, determining a first maximum palette predictor size for the CU, and predicting the CU based on the first maximum palette table size and the first maximum palette predictor size, in response to the luma component and the chroma component being coded jointly in palette mode.

[0005]

[0005] In some embodiments, the exemplary image processing device includes at least one memory for storing instructions and at least one processor. The at least one processor is configured to execute instructions to cause the device to determine whether the lumen component and the chromen component of the CU are encoded jointly or individually in palette mode, and, in response to the lumen component and the chromen component being encoded jointly in palette mode, to determine a first maximum palette table size for the CU, to determine a first maximum palette predictor size for the CU, and to predict the CU based on the first maximum palette table size and the first maximum palette predictor size.

[0006]

[0006] In some embodiments, an exemplary non-temporary computer-readable storage medium stores an instruction set. The instruction set can be performed by one or more processing devices to cause an image processing device to determine whether the lumen component and chromen component of a CU are encoded jointly or individually in palette mode, and, in response to the lumen component and chromen component being encoded jointly in palette mode, to determine a first maximum palette table size for the CU, to determine a first maximum palette predictor size for the CU, and to predict the CU based on the first maximum palette table size and the first maximum palette predictor size.

[0007] Brief explanation of the drawing

[0007] Embodiments and various aspects of the present disclosure are shown in the following detailed description and accompanying drawings. The various features shown in the drawings are not drawn to a constant scale. [Brief explanation of the drawing]

[0008] [Figure 1]

[0008] This is a schematic diagram showing the structure of an example video sequence according to some embodiments of the present disclosure. [Figure 2A]

[0009] This is a schematic diagram illustrating an exemplary encoding process of a hybrid video coding system consistent with the embodiments of the present disclosure. [Figure 2B]

[0010] This schematic diagram illustrates another exemplary encoding process for a hybrid video coding system consistent with the embodiments of the present disclosure. [Figure 3A]

[0011] This is a schematic diagram illustrating an exemplary decoding process of a hybrid video coding system consistent with the embodiments of the present disclosure. [Figure 3B]

[0012] This schematic diagram shows another exemplary decoding process for a hybrid video coding system, consistent with the embodiments of this disclosure. [Figure 4]

[0013] This is a block diagram of an exemplary apparatus for encoding or decoding video, according to some embodiments of the present disclosure. [Figure 5]

[0014] Schematic diagrams of exemplary blocks encoded in palette mode according to several embodiments of the present disclosure are shown. [Figure 6]

[0015] A schematic diagram illustrates an exemplary process of updating the palette predictor after encoding of the encoding unit, according to several embodiments of this disclosure. [Figure 7]

[0016] Table 1 illustrates exemplary uniform maximum predictor size and maximum pallet size according to several embodiments of the present disclosure. [Figure 8]

[0017] Table 2 shows an exemplary table showing exemplary maximum predictor sizes and maximum palette sizes according to some embodiments of the present disclosure. [Figure 9]

[0018] Table 3 shows an exemplary table showing an exemplary decoding process using predefined maximum palette predictor sizes and maximum palette sizes according to some embodiments of the present disclosure. [Figure 10]

[0019] Table 4 shows an exemplary table showing an exemplary palette encoding syntax table for using predefined maximum palette predictor sizes and maximum palette sizes according to some embodiments of the present disclosure. [Figure 11]

[0020] Table 5 shows an exemplary table showing an exemplary derivation of the maximum palette size and maximum palette predictor size of an individual palette according to some embodiments of the present disclosure. [Figure 12]

[0021] Table 6 shows an exemplary table showing another exemplary derivation of the maximum palette size and maximum palette predictor size of an individual palette according to some embodiments of the present disclosure. [Figure 13]

[0022] Table 7 shows an exemplary table showing an exemplary sequence parameter set (SPS) syntax table according to some embodiments of the present disclosure. [Figure 14]

[0023] Table 8 shows an exemplary table showing another exemplary derivation of the maximum palette size and maximum palette predictor size of an individual palette according to some embodiments of the present disclosure. [Figure 15]

[0024] Table 9 shows an exemplary table showing another exemplary derivation of the maximum palette size and maximum palette predictor size of an individual palette according to some embodiments of the present disclosure. [Figure 16]

[0025] Table 10 shows an exemplary table showing an exemplary picture header (PH) syntax according to some embodiments of the present disclosure. [Figure 17]

[0026] Exemplary Table 11 showing an exemplary derivation of the maximum palette size and the maximum palette predictor size for I slices, P slices, and B slices according to some embodiments of the present disclosure is shown. [Figure 18]

[0027] Exemplary Table 12 showing an exemplary slice header (SH) syntax according to some embodiments of the present disclosure is shown. [Figure 19]

[0028] A flowchart of an exemplary palette coding method according to some embodiments of the present disclosure is shown. [[ID=io]]

Mode for Carrying Out the Invention

[0009] Detailed Description

[0029] Hereinafter, reference will be made in detail to exemplary embodiments shown in the accompanying drawings. The following description refers to the accompanying drawings, in which, unless otherwise specified, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following description of the exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of apparatuses and methods consistent with aspects related to the present invention described in the appended claims. Specific aspects of the present disclosure will be described in more detail below. In case of conflict with the terms and / or definitions incorporated, the terms and definitions provided in this specification shall prevail.

[0010]

[0030] ITU-T VCEG (ITU-T Video Coding Expert Group) and ISO / IEC MPEG (ISO / IEC Moving Picture Expert Group)'s JVET (Joint Video Experts Team) are currently developing the VVC (Versatile Video Coding) / H.266 standard. The goal of the VVC standard is to double the compression efficiency of its predecessor, the HEVC (High Efficiency Video Coding) / H.265 standard. That is, the goal of VVC is to achieve the same subjective quality as HEVC / H.265 with half the bandwidth.

[0011]

[0031] To achieve the same subjective quality as HEVC / H.265 with half the bandwidth, JVET has developed techniques that surpass HEVC using the JEM (Joint Exploration Model) reference software. Because the encoding techniques were incorporated into JEM, JEM achieved significantly higher encoding performance than HEVC.

[0012]

[0032] The VVC standard is a relatively recent development and continues to incorporate more encoding techniques to provide better compression performance. VVC is based on the same hybrid video encoding system used in modern video compression standards such as HEVC, H.264 / AVC, MPEG2, and H.263.

[0013]

[0033] Video is a series of still pictures (or "frames") arranged in chronological order to preserve visual information. These pictures can be captured and saved chronologically using a video capture device (e.g., a camera), and such pictures can be displayed chronologically using a video playback device (e.g., a television, computer, smartphone, tablet computer, video player, or any end-user terminal with display capabilities). Furthermore, depending on the application, such as for surveillance, conferences, or live broadcasts, the video capture device can transmit the captured video in real time to a video playback device (e.g., a computer with a monitor).

[0014]

[0034] To reduce the memory space and transmission bandwidth required for such applications, video can be compressed before storage and transmission and decompressed before display. Compression and decompression can be performed by software executed by a processor (e.g., a general-purpose computer processor) or by dedicated hardware. The module for compression is generally called an “encoder,” and the module for decompression is generally called a “decoder.” Encoders and decoders are sometimes collectively called a “codec.” Encoders and decoders can be implemented as any of various appropriate hardware, software, or a combination thereof. For example, a hardware implementation of an encoder and decoder may include a network of one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, or any combination thereof. A software implementation of an encoder and decoder may include any appropriate computer-executable algorithm or process fixed in program code, computer-executable instructions, firmware, or computer-readable media. Video compression and decompression can be performed using various algorithms or standards such as MPEG-1, MPEG-2, MPEG-4, and H.26x. Depending on the application, a codec can decompress video from a first encoding standard and recompress the decompressed video using a second encoding standard; in this case, the codec is sometimes called a "transcoder."

[0015]

[0035] A video encoding process can identify and retain useful information that can be used to reconstruct a picture, while ignoring information that is not important for reconstruction. If the ignored, non-essential information cannot be fully reconstructed, such an encoding process is sometimes called "reversible." Otherwise, it is sometimes called "reversible." Most encoding processes are irreversible, which is a trade-off to reduce the required memory space and transmission bandwidth.

[0016]

[0036] Useful information about the encoded picture (referred to as the "current picture") includes changes relative to the reference picture (e.g., previously encoded and reconstructed pictures). Such changes may include changes in pixel position, brightness, or color, with position being the most important. Changes in the position of a group of pixels representing an object can reflect the movement of that object between the reference picture and the current picture.

[0017]

[0037] A picture encoded without referencing another picture (i.e., it is its own reference picture) is called an "I picture". A picture encoded using the previous picture as the reference picture is called a "P picture". A picture encoded using both the previous and future pictures as reference pictures (i.e., the reference is "bidirectional") is called a "B picture".

[0018]

[0038] Figure 1 shows the structure of an example video sequence 100 according to several embodiments of the present disclosure. The video sequence 100 may be live video or captured and archived video. The video 100 may be real video, computer-generated video (e.g., computer game video), or a combination thereof (e.g., real video with augmented reality effects). The video sequence 100 may be input from a video capture device (e.g., a camera), a video archive containing previously captured video (e.g., video files stored on a storage device), or a video feed interface for receiving video from a video content provider (e.g., a video broadcast transceiver).

[0019]

[0039] As shown in Figure 1, the video sequence 100 may include a series of pictures arranged in time along a timeline, including pictures 102, 104, 106, and 108. Pictures 102-106 are consecutive, with more pictures between pictures 106 and 108. In Figure 1, picture 102 is an I picture, and its reference picture is picture 102 itself. Picture 104 is a P picture, and its reference picture is picture 102, as indicated by the arrows. Picture 106 is a B picture, and its reference pictures are pictures 104 and 108, as indicated by the arrows. In some embodiments, the reference picture of a certain picture (e.g., picture 104) does not have to be immediately before or after that picture. For example, the reference picture of picture 104 may be a picture preceding picture 102. Please note that the reference pictures 102-106 are merely examples, and this disclosure does not limit the embodiments of the reference pictures to the examples shown in Figure 1.

[0020]

[0040] Generally, video codecs do not encode or decode an entire picture at once due to the computational complexity of such a task. More precisely, they divide the picture into basic segments and encode or decode each segment. Such basic segments are referred to in this disclosure as basic processing units ("BPUs"). For example, structure 110 in Figure 1 shows an example structure of a picture (e.g., any of pictures 102-108) in video sequence 100. In structure 110, the picture is divided into 4x4 basic processing units, their boundaries indicated by dashed lines. In some embodiments, the basic processing unit may be called a “macroblock” in some video encoding standards (e.g., MPEG family, H.261, H.263, or H.264 / AVC), or a “coding tree unit” ("CTU (coding tree unit)") in some other video encoding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing unit may have a variable picture size, such as 128×128, 64×64, 32×32, 16×16, 4×8, 16×32, or any shape and size of pixels. The size and shape of the basic processing unit can be selected for each picture based on a balance between encoding efficiency and the level of detail to be maintained in the basic processing unit.

[0021]

[0041] A basic processing unit may be a logical unit that contains a set of different types of video data stored in computer memory (e.g., in a video frame buffer). For example, a basic processing unit for a color picture may contain a luminance component (Y) representing achromatic lightness information, one or more chroma components (e.g., Cb and Cr) representing color information, and associated syntax elements (here, the luminance and chroma components may have basic processing units of the same size). The luminance and chroma components are sometimes called a "coding tree block" (CTB) in some video coding standards (e.g., H.265 / HEVC or H.266 / VVC). Any operation performed on a basic processing unit can be repeated on its luminance and chroma components, respectively.

[0022]

[0042] Video coding has multiple processing stages, examples of which are shown in Figures 2A-2B and 3A-3B. At each stage, the size of the basic processing unit may still be too large to process, and therefore it may be further divided into segments referred to in this disclosure as “basic processing subunits.” In some embodiments, a basic processing subunit may be referred to as a “block” in some video coding standards (e.g., MPEG family, H.261, H.263, or H.264 / AVC), or as a “coding unit” ("CU (coding unit)") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). A basic processing subunit may be the same size as or smaller than a basic processing unit. Like a basic processing unit, a basic processing subunit is a logical unit that may contain a set of different types of video data (e.g., Y, Cb, Cr, and associated syntax elements) stored in computer memory (e.g., in a video frame buffer). Any operation performed on a basic processing subunit can be repeated on its luma and chroma components, respectively. Note that such divisions may be performed at further levels depending on the processing needs. Also note that different stages may divide the basic processing unit using different schemes.

[0023]

[0043] For example, in the mode determination stage (an example of which is shown in Figure 2B), the encoder can determine which prediction mode (e.g., intra-picture prediction or inter-picture prediction) should be used for the basic processing unit, which may be too large to make such a decision. The encoder can divide the basic processing unit into multiple basic processing subunits (e.g., CUs in the case of H.265 / HEVC or H.266 / VVC) and determine the prediction type for each individual basic processing subunit.

[0024]

[0044] As another example, in the prediction stage (an example of which is shown in Figures 2A-2B), the encoder can perform prediction calculations at the level of the basic processing subunit (e.g., CU). However, in some cases, the basic processing subunit may still be too large to process. The encoder can further divide the basic processing subunit into smaller segments (for example, called "prediction blocks" or "PB (prediction blocks)" in H.265 / HEVC or H.266 / VVC), and perform prediction calculations at the level of these segments.

[0025]

[0045] As another example, in the transformation stage (an example of which is shown in Figures 2A-2B), the encoder can perform transformation operations on residual basic processing subunits (e.g., CUs). However, in some cases, the basic processing subunit may still be too large to process. The encoder can further divide the basic processing subunit into smaller segments (e.g., called "transform blocks" or "TBs (transform blocks)" in H.265 / HEVC or H.266 / VVC), and perform transformation operations at the segment level. Note that the division scheme for the same basic processing subunit may differ between the prediction stage and the transformation stage. For example, in H.265 / HEVC or H.266 / VVC, the prediction blocks and transform blocks of the same CU may have different sizes and numbers.

[0026]

[0046] In the structure 110 of Figure 1, the basic processing unit 112 is further divided into 3x3 basic processing subunits, their boundaries indicated by dotted lines. Different basic processing units of the same picture may be divided into basic processing subunits in different schemes.

[0027]

[0047] In some embodiments, a picture can be divided into multiple regions for processing to provide parallel processing capability and error tolerance for video encoding and decoding, so that the encoding or decoding process does not depend on information from any other region of the picture with respect to that region. In other words, each region of the picture can be processed independently. In this way, the codec can process multiple different regions of the picture in parallel, thus improving encoding efficiency. Furthermore, if data in one region is corrupted during processing or lost during network transmission, the codec can accurately encode or decode other regions of the same picture without relying on the corrupted or lost data, thus providing error tolerance. In some video encoding standards, a picture can be divided into multiple different types of regions. For example, H.265 / HEVC and H.266 / VVC offer two region types: "slice" and "tile". It should also be noted that multiple different pictures in video sequence 100 may have different partition schemes for dividing the picture into regions.

[0028]

[0048] For example, in Figure 1, structure 110 is divided into three regions 114, 116, and 118, their boundaries shown as solid lines within structure 110. Region 114 contains four basic processing units. Regions 116 and 118 each contain six basic processing units. Note that the basic processing units, basic processing subunits, and regions of structure 110 in Figure 1 are merely examples, and this disclosure does not limit their embodiments.

[0029]

[0049] Figure 2A shows a schematic diagram of an example encoding process 200A consistent with embodiments of the present disclosure. For example, the encoding process 200A can be performed by an encoder. As shown in Figure 2A, the encoder can encode a video sequence 202 into a video bitstream 228 according to process 200A. Similar to video sequence 100 in Figure 1, video sequence 202 may contain a set of pictures arranged in chronological order (referred to as "original pictures"). Similar to structure 110 in Figure 1, each original picture in video sequence 202 can be divided by the encoder into a basic processing unit, a basic processing subunit, or a processing area. In some embodiments, the encoder can perform process 200A at the level of a basic processing unit for each original picture in video sequence 202. For example, the encoder can perform process 200A in an iterative manner, in which case the encoder can encode one basic processing unit in one iteration of process 200A. In some embodiments, the encoder can perform process 200A in parallel for each region of the original picture in the video sequence 202 (e.g., regions 114-118).

[0030]

[0050] In Figure 2A, the encoder can send the basic processing unit of the original picture of the video sequence 202 (called the "original BPU") to the prediction stage 204 to generate the prediction data 206 and the prediction BPU 208. The encoder can generate the residual BPU 210 by subtracting the prediction BPU 208 from the original BPU. The encoder can send the residual BPU 210 to the conversion stage 212 and the quantization stage 214 to generate the quantization conversion coefficients 216. The encoder can send the prediction data 206 and the quantization conversion coefficients 216 to the binary coding stage 226 to generate the video bitstream 228. Components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 are sometimes referred to as the "forward path". During process 200A, after the quantization stage 214, the encoder can send the quantization conversion coefficients 216 to the inverse quantization stage 218 and the inverse conversion stage 220 to generate the reconstructed residual BPU 222. The encoder can generate a prediction criterion 224 to be used in the prediction stage 204 for the next iteration of process 200A by adding the reconstructed residual BPU 222 to the prediction BPU 208. Components 218, 220, 222, and 224 of process 200A are sometimes referred to as the “reconstruction path”. The reconstruction path can be used to ensure that the encoder and decoder both use the same reference data for prediction.

[0031]

[0051] The encoder can iteratively perform process 200A to encode each original BPU of the original picture (in the forward path) and to generate a predictive criterion 224 for encoding the next original BPU of the original picture (in the reconstruction path). After encoding all original BPUs of the original picture, the encoder can proceed to encoding the next picture in the video sequence 202.

[0032]

[0052] Referring to process 200A, the encoder can receive a video sequence 202 generated by a video capture device (e.g., a camera). As used herein, the term “receive” can mean any action of any means of receiving, inputting, acquiring, retrieving, obtaining, reading, accessing, or inputting data.

[0033]

[0053] In prediction stage 204, in the current iteration, the encoder can receive the original BPU and prediction criterion 224, and can perform prediction calculations to generate prediction data 206 and prediction BPU 208. The prediction criterion 224 can be generated from the reconfiguration path of the previous iteration of process 200A. The objective of prediction stage 204 is to reduce information redundancy by extracting prediction data 206, which can be used to reconfigure the original BPU as prediction BPU 208 from prediction data 206 and prediction criterion 224.

[0034]

[0054] Ideally, the predicted BPU 208 can be identical to the original BPU. However, due to non-ideal prediction and reconstruction operations, the predicted BPU 208 is generally slightly different from the original BPU. To record such differences, after generating the predicted BPU 208, the encoder can generate the residual BPU 210 by subtracting it from the original BPU. For example, the encoder can subtract the pixel values ​​(e.g., grayscale values ​​or RGB values) of the predicted BPU 208 from the corresponding pixel values ​​of the original BPU. Each pixel in the residual BPU 210 may have a residual value as a result of such subtraction between the original BPU and the corresponding pixels of the predicted BPU 208. Compared to the original BPU, the predicted data 206 and residual BPU 210 may have fewer bits, but they can be used to reconstruct the original BPU without significant quality degradation. Thus, the original BPU is compressed.

[0035]

[0055] To further compress the residual BPU210, in the transformation stage 212, the encoder can reduce the spatial redundancy of the residual BPU210 by decomposing it into a set of two-dimensional "basis patterns" (each basis pattern associated with a "transformation coefficient"). The basis patterns may have the same size (e.g., the size of the residual BPU210). Each basis pattern may represent a variation frequency component (e.g., the frequency of brightness variation) of the residual BPU210. No basis pattern can be reconstructed from any combination (e.g., a linear combination) of other basis patterns. In other words, this decomposition allows the variation of the residual BPU210 to be decomposed into the frequency domain. Such a decomposition is analogous to the discrete Fourier transform of a function, where the basis patterns are analogous to the basis functions of the discrete Fourier transform (e.g., trigonometric functions), and the transformation coefficients are analogous to the coefficients associated with the basis functions.

[0036]

[0056] Different transformation algorithms can use different basis patterns. For example, various transformation algorithms such as discrete cosine transform or discrete sine transform can be used in transformation stage 212. The transformation in transformation stage 212 is reversible. That is, the encoder can reconstruct the residual BPU 210 by the inverse operation of the transformation (called the "inverse transform"). For example, to reconstruct the pixels of the residual BPU 210, the inverse transform may be to multiply the values ​​of the corresponding pixels in the basis pattern by their respective associated coefficients and generate a weighted sum by adding their products. For video coding standards, both the encoder and decoder can use the same transformation algorithm (and therefore the same basis pattern). Therefore, the encoder can record only the transformation coefficients, and the decoder can reconstruct the residual BPU 210 from the transformation coefficients without receiving the basis pattern from the encoder. Compared to the residual BPU 210, the transformation coefficients may have fewer bits, but they can be used to reconstruct the residual BPU 210 without significant quality degradation. Therefore, the residual BPU210 is further compressed.

[0037]

[0057] The encoder can further compress the conversion coefficients in the quantization stage 214. In the conversion process, different basis patterns may represent different fluctuation frequencies (e.g., brightness fluctuation frequencies). Since the human eye is generally good at recognizing low-frequency fluctuations, the encoder can ignore high-frequency fluctuation information without causing significant quality degradation in decoding. For example, in the quantization stage 214, the encoder can generate quantization conversion coefficients 216 by dividing each conversion coefficient by an integer value (called a "quantization parameter") and rounding the quotient to the nearest integer. After such an operation, some conversion coefficients of high-frequency basis patterns may be converted to zero, and conversion coefficients of low-frequency basis patterns may be converted to smaller integers. The encoder can ignore the zero-value quantization conversion coefficients 216, thereby further compressing the conversion coefficients. The quantization process is also reversible, where the quantization conversion coefficients 216 can be reconstructed into conversion coefficients in the inverse operation of quantization (called "inverse quantization").

[0038]

[0058] Since the encoder ignores the remainder of such division in rounding operations, the quantization stage 214 can be irreversible. Generally, the quantization stage 214 can contribute the most to information loss in process 200A. The greater the information loss, the fewer bits the quantization conversion coefficient 216 may require. To obtain different levels of information loss, the encoder may use different values ​​for the quantization parameter or other parameters of the quantization process.

[0039]

[0059] In the binary coding stage 226, the encoder can encode the prediction data 206 and quantization conversion coefficients 216 using binary coding techniques such as entropy coding, variable-length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or other reversible or irreversible compression algorithms. In some embodiments, in addition to the prediction data 206 and quantization conversion coefficients 216, the encoder can encode other information in the binary coding stage 226, such as the prediction mode used in the prediction stage 204, the parameters of the prediction operation, the conversion type in the conversion stage 212, the parameters of the quantization process (e.g., quantization parameters), or encoder control parameters (e.g., bitrate control parameters). The encoder can use the output data from the binary coding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 may be further packetized for network transmission.

[0040]

[0060] Referring to the reconstruction path of process 200A, in the inverse quantization stage 218, the encoder can generate reconstruction transformation coefficients by performing inverse quantization on the quantization transformation coefficients 216. In the inverse transformation stage 220, the encoder can generate reconstruction residual BPU 222 based on the reconstruction transformation coefficients. The encoder can generate a prediction criterion 224 to be used in the next iteration of process 200A by adding the reconstruction residual BPU 222 to the prediction BPU 208.

[0041]

[0061] It should be noted that other variations of process 200A may be used to encode video sequence 202. In some embodiments, the stages of process 200A may be performed by the encoder in a different order. In some embodiments, one or more stages of process 200A may be combined into a single stage. In some embodiments, a single stage of process 200A may be divided into multiple stages. For example, the conversion stage 212 and the quantization stage 214 may be combined into a single stage. In some embodiments, process 200A may include further stages. In some embodiments, process 200A may omit one or more stages shown in Figure 2A.

[0042]

[0062] Figure 2B shows a schematic diagram of another encoding process example 200B consistent with embodiments of the present disclosure. Process 200B may be modified from process 200A. For example, process 200B may be used by an encoder compliant with a hybrid video coding standard (e.g., H.26x). Compared with process 200A, the forward path of process 200B further includes a mode determination stage 230 and divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044. The reconfiguration path of process 200B further includes a loop filter stage 232 and a buffer 234.

[0043]

[0063] Generally, prediction techniques can be classified into two types: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-picture prediction or "intra-prediction") can predict the current BPU by using pixels from one or more already encoded neighboring BPUs within the same picture. That is, the prediction criterion 224 in spatial prediction may include neighboring BPUs. Spatial prediction can reduce the inherent spatial redundancy of a picture. Temporal prediction (e.g., inter-picture prediction or "inter-prediction") can predict the current BPU by using regions from one or more already encoded pictures. That is, the prediction criterion 224 in temporal prediction may include encoded pictures. Temporal prediction can reduce the inherent temporal redundancy of a picture.

[0044]

[0064] Referring to process 200B, in the forward path, the encoder performs prediction calculations in the spatial prediction stage 2042 and the temporal prediction stage 2044. For example, in the spatial prediction stage 2042, the encoder can perform intra-prediction. With respect to the original BPU of the encoded picture, the prediction criterion 224 may include one or more adjacent BPUs within the same picture that are encoded (in the forward path) and reconstructed (in the reconstruction path). The encoder can generate a prediction BPU 208 by extrapolating adjacent BPUs. Extrapolation techniques may include, for example, linear extrapolation or interpolation, or polynomial extrapolation or interpolation. In some embodiments, the encoder may perform pixel-level extrapolation, for example, by extrapolating the values ​​of the corresponding pixels for each pixel of the prediction BPU 208. The adjacent BPU used for extrapolation may be located relative to the original BPU from various directions, such as vertically (e.g., above the original BPU), horizontally (e.g., to the left of the original BPU), diagonally (e.g., below left, below right, above left, or above right of the original BPU), or in any direction defined in the video encoding standard used. In the case of intra-prediction, the prediction data 206 may include, for example, the location (e.g., coordinates) of the adjacent BPU used, the size of the adjacent BPU used, the extrapolation parameters, or the orientation of the adjacent BPU used relative to the original BPU.

[0045]

[0065] As another example, in the time prediction stage 2044, the encoder can perform interpretation. With respect to the original BPU of the current picture, the prediction criterion 224 may include one or more pictures (referred to as "reference pictures") that have been encoded (in the forward path) and reconstructed (in the reconstruction path). In some embodiments, the reference pictures may be encoded and reconstructed for each BPU. For example, the encoder may generate a reconstructed BPU by adding the reconstructed residual BPU 222 to the prediction BPU 208. Once all the reconstructed BPUs for the same picture have been generated, the encoder can generate a reconstructed picture as a reference picture. The encoder may perform a "motion estimation" operation to search for matching regions within a range (referred to as a "search window") of the reference picture. The location of the search window in the reference picture may be determined based on the location of the original BPU in the current picture. For example, the search window may be centered at a location in the reference picture that has the same coordinates as the original BPU of the current picture, or it may extend outward by a predetermined distance. When the encoder identifies a region in the search window that resembles the original BPU (e.g., using a Pell recursive algorithm or a block matching algorithm), the encoder can determine such a region as a matching region. The matching region may have different dimensions from the original BPU (e.g., smaller, equal to, larger, or different shape). Since the reference picture and the current picture are temporally separated on the timeline (e.g., as shown in Figure 1), the matching region can be considered to "move" to the location of the original BPU over time. The encoder may record the direction and distance of such movement as a "motion vector". If multiple reference pictures are used (e.g., picture 106 in Figure 1), the encoder can search for a matching region and determine the associated motion vector for each reference picture. In some embodiments, the encoder can assign weights to the pixel values ​​of the matching region for each matching reference picture.

[0046]

[0066] Motion estimation can be used to identify various types of motion, such as translation, rotation, or zooming. In the case of interpretation, the prediction data 206 may include, for example, the location of the matching region (e.g., coordinates), the motion vector associated with the matching region, the number of reference pictures, or the weights associated with the reference pictures.

[0047]

[0067] To generate a predicted BPU 208, the encoder may perform a “motion compensation” operation. Using motion compensation, the predicted BPU 208 can be reconstructed based on prediction data 206 (e.g., motion vectors) and prediction criteria 224. For example, the encoder may move the matching region of a reference picture according to the motion vector, in which case the encoder can predict the original BPU of the current picture. If multiple reference pictures are used (e.g., picture 106 in Figure 1), the encoder may move the matching region of each reference picture according to its respective motion vector and average the pixel values ​​of the matching region. In some embodiments, if the encoder assigns weights to the pixel values ​​of the matching region of each matching reference picture, the encoder may add the weighted sum of the pixel values ​​of the moved matching region.

[0048]

[0068] In some embodiments, interpretation may be unidirectional or bidirectional. Unidirectional interpretation may use one or more reference pictures in the same time direction relative to the current picture. For example, picture 104 in Figure 1 is a unidirectional interpretation picture in which the reference picture (e.g., picture 102) precedes picture 104. Bidirectional interpretation may use one or more reference pictures in both time directions relative to the current picture. For example, picture 106 in Figure 1 is a bidirectional interpretation picture in which the reference pictures (i.e., pictures 104 and 108) are in both time directions relative to picture 104.

[0049]

[0069] Further reference to the forward path of process 200B, after the spatial prediction 2042 and time prediction stages 2044, in the mode determination stage 230, the encoder may select a prediction mode (e.g., either intra-prediction or inter-prediction) for the current iteration of process 200B. For example, the encoder may perform a rate-distortion optimization technique in which the encoder can select a prediction mode to minimize the value of the cost function, depending on the bit rate of the candidate prediction mode and the distortion of the reconstructed reference picture under such candidate prediction mode. Depending on the selected prediction mode, the encoder may generate the corresponding prediction BPU 208 and prediction data 206.

[0050]

[0070] In the reconstruction path of process 200B, if intra-prediction mode was selected in the forward path, after generating the prediction criterion 224 (e.g., the current BPU encoded and reconstructed within the current picture), the encoder can send the prediction criterion 224 directly to the spatial prediction stage 2042 for later use (e.g., for extrapolation of the next BPU of the current picture). If inter-prediction mode was selected in the forward path, after generating the prediction criterion 224 (e.g., the current picture with all BPUs encoded and reconstructed), the encoder can send the prediction criterion 224 to the loop filter stage 232, where the encoder can apply loop filters to the prediction criterion 224 to reduce or eliminate distortions (e.g., blocking artifacts) introduced by inter-prediction. In the loop filter stage 232, the encoder can apply various loop filtering techniques, such as deblocking, sample-adaptive offset, or adaptive loop filtering. The loop-filtered reference picture may be stored in buffer 234 (or “decode picture buffer”) for later use (for example, to be used as an inter-prediction reference picture for future pictures in video sequence 202). The encoder may store one or more reference pictures used in the time prediction stage 2044 in buffer 234. In some embodiments, the encoder may encode the loop filter parameters (e.g., loop filter strength) along with the quantization conversion coefficients 216, prediction data 206, and other information in the binary coding stage 226.

[0051]

[0071] Figure 3A shows a schematic diagram of an example decoding process 300A consistent with embodiments of the present disclosure. Process 300A may also be a decompression process corresponding to the compression process 200A in Figure 2A. In some embodiments, process 300A may be similar to the reconstruction path of process 200A. The decoder can decode the video bitstream 228 into a video stream 304 according to process 300A. The video stream 304 may be very similar to the video sequence 202. However, due to information loss in the compression and decompression processes (e.g., the quantization stage 214 in Figures 2A-2B), the video stream 304 is generally not identical to the video sequence 202. Similar to processes 200A and 200B in Figures 2A-2B, the decoder may perform process 300A at the level of the basic processing unit (BPU) for each picture encoded in the video bitstream 228. For example, the decoder can perform process 300A in an iterative manner, in which case the decoder can decode one basic processing unit in one iteration of process 300A. In some embodiments, the decoder can perform process 300A in parallel for each region of the encoded picture (e.g., regions 114-118) in the video bitstream 228.

[0052]

[0072] In Figure 3A, the decoder can send a portion of the video bitstream 228 associated with the basic processing unit of the encoded picture (called the "encode BPU") to the binary decoding stage 302. In the binary decoding stage 302, the decoder can decode this portion into prediction data 206 and quantization conversion coefficients 216. The decoder can send the quantization conversion coefficients 216 to the inverse quantization stage 218 and the inverse conversion stage 220 to generate the reconstructed residual BPU 222. The decoder can send the prediction data 206 to the prediction stage 204 to generate the prediction BPU 208. The decoder can generate a prediction criterion 224 by adding the reconstructed residual BPU 222 to the prediction BPU 208. In some embodiments, the prediction criterion 224 can be stored in a buffer (e.g., a decoded picture buffer in computer memory). The decoder can send the prediction criterion 224 to the prediction stage 204 for performing the prediction calculation in the next iteration of process 300A.

[0053]

[0073] The decoder may iterate through process 300A to decode each encoding BPU of the encoded picture and generate a predictive criterion 224 for encoding the next encoding BPU of the encoded picture. After decoding all encoding BPUs of the encoded picture, the decoder may output the picture to the video stream 304 for display and proceed to decode the next encoded picture in the video bitstream 228.

[0054]

[0074] In the binary decoding stage 302, the decoder can perform the inverse operation of the binary coding technique used by the encoder (e.g., entropy coding, variable-length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or other lossless compression algorithms). In some embodiments, in addition to the prediction data 206 and quantization conversion coefficients 216, the decoder can decode other information in the binary decoding stage 302, such as the prediction mode, prediction operation parameters, conversion type, quantization process parameters (e.g., quantization parameters), or encoder control parameters (e.g., bitrate control parameters). In some embodiments, if the video bitstream 228 is transmitted in packets over a network, the decoder can depacketize the video bitstream 228 before sending it to the binary decoding stage 302.

[0055]

[0075] Figure 3B shows a schematic diagram of another decoding process example 300B consistent with embodiments of the present disclosure. Process 300B may be modified from process 300A. For example, process 300B may be used with a decoder compliant with a hybrid video coding standard (e.g., H.26x). Compared to process 300A, process 300B further divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and further includes a loop filter stage 232 and a buffer 234.

[0056]

[0076] In process 300B, with respect to the encoding base processing unit ("current BPU") of the encoded picture being decoded ("current picture"), the prediction data 206 decoded by the decoder from the binary decoding stage 302 may contain various types of data depending on which prediction mode was used by the encoder to encode the current BPU. For example, if intra-prediction is used by the encoder to encode the current BPU, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating intra-prediction, or parameters of the intra-prediction calculation. Parameters of the intra-prediction calculation may include, for example, the location (e.g., coordinates) of one or more adjacent BPUs used as a reference, the size of the adjacent BPUs, extrapolation parameters, or the orientation of the adjacent BPU relative to the original BPU. As another example, if inter-prediction is used by the encoder to encode the current BPU, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating inter-prediction, or parameters of the inter-prediction calculation. The parameters for the interpretation calculation may include, for example, the number of reference pictures associated with the current BPU, the weights associated with each reference picture, the location (e.g., coordinates) of one or more matching regions in each reference picture, or one or more motion vectors associated with each matching region.

[0057]

[0077] Based on the prediction mode indicator, the decoder can decide whether to perform spatial prediction (e.g., intra-prediction) in the spatial prediction stage 2042 or temporal prediction (e.g., inter-prediction) in the temporal prediction stage 2044. Details of how to perform such spatial or temporal predictions are shown in Figure 2B and will not be repeated below. After performing such spatial or temporal predictions, the decoder can generate a prediction BPU 208. The decoder can generate a prediction criterion 224 by adding the prediction BPU 208 and the reconstructed residual BPU 222, as shown in Figure 3A.

[0058]

[0078] In process 300B, the decoder can send the prediction criterion 224 to the spatial prediction stage 2042 or the temporal prediction stage 2044 for performing prediction calculations in the next iteration of process 300B. For example, if the current BPU is decoded using intra-prediction in the spatial prediction stage 2042, after generating the prediction criterion 224 (e.g., the decoded current BPU), the decoder can send the prediction criterion 224 directly to the spatial prediction stage 2042 for later use (e.g., for extrapolating the next BPU of the current picture). If the current BPU is decoded using inter-prediction in the temporal prediction stage 2044, after generating the prediction criterion 224 (e.g., the reference picture with all BPUs decoded), the encoder can send the prediction criterion 224 to the loop filter stage 232 to reduce or eliminate distortion (e.g., blocking artifacts). The decoder can apply the loop filter to the prediction criterion 224 in the manner shown in Figure 2B. The loop-filtered reference picture may be stored in buffer 234 (e.g., a decode picture buffer in computer memory) for later use (e.g., to be used as an inter-prediction reference picture for a picture to be encoded in the future of the video bitstream 228). The decoder may store one or more reference pictures used in the time prediction stage 2044 in buffer 234. In some embodiments, if the prediction mode indicator of the prediction data 206 indicates that inter-prediction was used to encode the current BPU, the prediction data may further include loop filter parameters (e.g., loop filter strength).

[0059]

[0079] Figure 4 is a block diagram of an example apparatus 400 for encoding or decoding video according to an embodiment of the present disclosure. As shown in Figure 4, the apparatus 400 may include a processor 402. When the processor 402 executes instructions described herein, the apparatus 400 can become a dedicated machine for video encoding or decoding. The processor 402 may be any type of circuit network capable of manipulating or processing information. For example, the processor 402 may include any combination of several central processing units (i.e., "CPUs"), graphics processing units (i.e., "GPUs"), neural processing units ("NPUs"), microcontroller units ("MCUs"), optical processors, programmable logic controllers, microcontrollers, microprocessors, digital signal processors, IP (intellectual property) cores, programmable logic arrays (PLAs), programmable array logic (PALs), general-purpose array logic (GALs), composite programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), systems on a chip (SoCs), or application-specific integrated circuits (ASICs). In some embodiments, the processor 402 may be a set of processors grouped as a single logical component. For example, as shown in Figure 4, the processor 402 may include multiple processors, including processor 402a, processor 402b, and processor 402n.

[0060]

[0080] The device 400 may also include a memory 404 configured to store data (e.g., instruction sets, computer code, or intermediate data). For example, as shown in Figure 4, the stored data may include program instructions (e.g., program instructions for implementing stages of processes 200A, 200B, 300A, or 300B) and processing data (e.g., video sequence 202, video bitstream 228, or video stream 304). The processor 402 can access the program instructions and processing data (e.g., via bus 410) and execute the program instructions to perform calculations or operations on the processing data. The memory 404 may include a high-speed random-access storage device or a non-volatile storage device. In some embodiments, the memory 404 may include any combination of several random-access memories (RAM), read-only memories (ROM), optical disks, magnetic disks, hard drives, solid-state drives, flash drives, SD (security digital) cards, memory sticks, or CompactFlash® (CF) cards. Memory 404 may also be a group of memories (not shown in Figure 4) grouped as a single logical component.

[0061]

[0081] Bus 410 may also be a communication device that transfers data between components within device 400, such as an internal bus (e.g., a CPU memory bus) or an external bus (e.g., a universal serial bus port, a peripheral component interconnection express port).

[0062]

[0082] To simplify the explanation without creating ambiguity, the processor 402 and other data processing circuits are collectively referred to as “data processing circuits” in this disclosure. The data processing circuits may be implemented entirely as hardware, or as a combination of software, hardware, or firmware. Furthermore, the data processing circuits may be a single, independent module, or may be fully or partially integrated with any other component of the device 400.

[0063]

[0083] The device 400 may further include a network interface 406 to provide wired or wireless communication to a network (e.g., the Internet, an intranet, a local area network, or a mobile communication network). In some embodiments, the network interface 406 may include any combination of several network interface controllers (NICs), radio frequency (RF) modules, transponders, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth® adapters, infrared adapters, near-field communication ("NFC") adapters, or cellular network chips.

[0064]

[0084] In some embodiments, the apparatus 400 may optionally further include a peripheral interface 408 to provide connectivity to one or more peripheral devices. As shown in Figure 4, peripheral devices may include (but are not limited to) cursor control devices (e.g., mouse, touchpad, or touchscreen), keyboards, displays (e.g., cathode ray tube displays, liquid crystal displays, or light-emitting diode displays), or video input devices (e.g., cameras, or input interfaces coupled to video archives).

[0065]

[0085] It should be noted that a video codec (for example, a codec that performs processes 200A, 200B, 300A, or 300B) may be implemented as any combination of any software or hardware modules within device 400. For example, some or all stages of processes 200A, 200B, 300A, or 300B may be implemented as one or more software modules of device 400, such as program instructions that can be loaded into memory 404. As another example, some or all stages of processes 200A, 200B, 300A, or 300B may be implemented as one or more hardware modules of device 400, such as dedicated data processing circuits (e.g., FPGA, ASIC, or NPU).

[0066]

[0086] In the quantization and dequantization function blocks (e.g., quantization 214 and dequantization 218 in Figure 2A or 2B, and dequantization 218 in Figure 3A or 3B), quantization parameters (QP) are used to determine the amount of quantization (and dequantization) applied to the prediction residuals. The initial QP values ​​used for encoding the picture or slice may be signaled at a high level, for example, using the init_qp_minus26 syntax element of the picture parameter set (PPS) and the slice_qp_delta syntax element of the slice header. Furthermore, the QP values ​​may be adapted at a local level per CU using delta QP values ​​sent at the granularity of the quantization group.

[0067]

[0087] In VVC, palette mode can be used with the 4:4:4 color format. When palette mode is enabled, a flag indicating whether palette mode is used is sent at the CU level, provided the CU size is 64x64 or less.

[0068]

[0088] Figure 5 shows a schematic diagram of an exemplary block 500 encoded in palette mode according to several embodiments of the present disclosure. When palette mode is used to encode the current CU (e.g., block 500) as shown in Figure 5, the sample value at each position in the CU (e.g., position 501, position 502, position 503, or position 504) is represented by a small set of representative color values. This set is called a “palette” or “palette table” (e.g., palette 510). For sample positions with values ​​close to the palette color, the corresponding palette index (e.g., index 0, index 1, index 2, or index 3) is signaled. According to some embodiments of the disclosure, color values ​​outside the palette table can be specified by signaling an escape index (e.g., index 4). Then, for all positions in the CU using the escape color index, the (quantized) color component value is signaled for each of these positions.

[0069]

[0089] To encode the palette table, a palette predictor is maintained. The palette predictor is initialized to 0 (e.g., empty) at the beginning of each slice in the non-wavefront case, and at the beginning of each CTU row in the wavefront case. In some cases, the palette predictor may also be initialized to 0 at the beginning of a tile. Figure 6 shows a schematic diagram of an exemplary process for updating the palette predictor after encoding and decoding of an encoding unit, according to some embodiments of the present disclosure. As shown in Figure 6, for each entry in the palette predictor, a reuse flag is signaled indicating whether or not it is included in the current palette table of the current CU. The reuse flag is transmitted using zero run-length encoding, and then the number of new palette entries and the component values ​​of the new palette entries are signaled. After encoding and / or decoding of the palette encoding CU, the palette predictor is updated using the current palette table, and entries from previous palette predictors that are not reused in the current palette table are appended to the end of the new palette predictor until the maximum allowable size is reached.

[0070]

[0090] In some embodiments, each CU is signaled with an escape flag to indicate whether an escape character exists within the current CU. If an escape character exists, the palette table is expanded by one (as shown in Figure 5), and the escape character is assigned to the last index.

[0071]

[0091] Referring to Figure 5, the palette index of a sample in the CU forms a palette index map. The index map is encoded using horizontal or vertical transverse scanning. The scan order is explicitly signaled in the bitstream using the syntax element "palette_transpose_flag". The palette index map is encoded using either index execution mode or index copy mode.

[0072]

[0092] According to some embodiments, the tree structure of an I-slice is signaled by the syntax element "qtbtt_dual_tree_intra_flag" in the Sequence Parameter Set (SPS) syntax. A syntax element qtbtt_dual_tree_intra_flag equal to 1 indicates that two separate coding_tree syntax structures are used for the luma and chroma components of an I-slice, respectively. A syntax element qtbtt_dual_tree_intra_flag equal to 0 indicates that no separate coding_tree syntax structures are used for the luma and chroma components of an I-slice. Also, P-slices and B-slices are always encoded as a single tree slice. Consistent with the disclosed embodiments, an I-picture is an intra-encoded picture that does not reference other pictures in the encoding / decoding process. P-pictures and B-pictures are both inter-encoded pictures and are decoded while referencing other pictures. The difference between P-pictures and B-pictures is that each block in a P-picture can reference a maximum of one block in each referenced picture, whereas each block in a B-picture can reference a maximum of two blocks in each referenced picture.

[0073]

[0093] According to some embodiments, in the case of a slice with a dual luma / chroma tree, different palettes (e.g., different palette tables) are applied separately to the luma (Y component) and chroma (Cb and Cr components). In the case of a dual-tree slice (e.g., a dual luma / chroma tree), each entry in the luma palette table contains only Y values, and each entry in the chroma palette table contains both Cb and Cr values. In the case of a single-tree slice, the palette is applied congruently to the Y, Cb, and Cr components (e.g., each entry in the palette table contains Y, Cb, and Cr values). Also, in the case of certain color formats such as 4:2:0 and 4:2:2 color formats, a coding unit (CU) in a single-tree slice may have separate luma and chroma trees due to limitations on the minimum allowable chroma coding block size. Therefore, in the case of these color formats, a CU in a single-tree slice may have a local dual-tree structure (e.g., a single tree at the slice level but a dual tree at the CU level).

[0074]

[0094] Therefore, in the case of a single-tree slice coding unit, if it is not an inter-SCIPU (smallest chroma intra prediction unit), the chroma cannot be further divided, but the luma can, so it may have separate luma trees and chroma trees. In single-tree coding, SCIPU is defined as a coding tree node with a chroma block size of 16 chroma samples or more and has at least one child luma block of less than 64 luma samples. As mentioned above, the separate trees associated with SCIPU are called local dual trees.

[0075]

[0095] Based on the slice's tree type (e.g., single-tree or dual-tree), two types of palette tables ("congruent palette" and "individual palette") can be used for the slice. A single-tree slice can be palette-encoded using a congruent palette table. Each entry in the congruent palette table contains the Y color component, Cb color component, and Cr color component, and all color components of an encoded unit (CU) in a single-tree slice (except for the local dual-tree described above) are coded congruently using the congruent palette table. In contrast, a dual-tree slice is palette-encoded using two individual palettes. The lumens and chroma components of a dual-tree slice require different palette tables and are coded individually. Therefore, in the case of a dual-tree slice, two index maps (one for the lumens and one for the chroma) are signaled in the bitstream.

[0076]

[0096] Figure 7 shows an exemplary Table 1 illustrating exemplary uniform maximum predictor size and maximum pallet size according to several embodiments of the present disclosure. As shown in Table 1, the maximum pallet predictor size for both congruent and individual pallets is uniformly set to 63, and the maximum pallet size for both congruent and individual pallet tables is uniformly set to 31. However, as stated above, two individual pallet tables are required for a dual tree slice / CU with individual luma-chroma trees, while only one congruent pallet table is required for a single tree slice / CU with a congruent luma-chroma tree. Therefore, the complexity of generating an individual pallet table for a dual tree slice / CU is approximately twice the complexity of generating a congruent pallet table for a single tree slice / CU.

[0077]

[0097] In accordance with several disclosed embodiments, to address the computational complexity and time imbalance between dual-tree-slice / CU palette coding and single-tree-slice / CU palette coding, the maximum predictor size of individual luma-trees and chroma-trees can be set smaller than the maximum predictor size of a single (e.g., congruent) luma-chroma-tree. Alternatively or additionally, the maximum palette size (e.g., maximum palette table size) of individual luma-trees and chroma-trees can be set smaller than the maximum palette size of a single (e.g., congruent) luma-chroma-tree.

[0078]

[0098] In some disclosed embodiments, the following six variables are defined representing the maximum predictor size and the maximum palette size. In particular, the variable "max_plt_predictor_size_joint" represents the maximum predictor size of the joint palette. The variable "max_plt_predictor_size_luma" represents the maximum predictor size of the luma component when individual palettes are used. The variable "max_plt_predictor_size_chroma" represents the maximum predictor size of the chroma component when individual palettes are used. The variable "max_plt_size_joint" represents the maximum palette size of the joint palette. The variable "max_plt_size_luma" represents the maximum palette size of the luma component when individual palettes are used. The variable "max_plt_size_chroma" represents the maximum palette size of the chroma component when individual palettes are used.

[0079]

[0099] In some embodiments, the maximum pallet predictor size and maximum pallet size are a predefined set of fixed values ​​and do not need to be signaled to the video decoder. Figure 8 shows exemplary Table 2 illustrating exemplary maximum predictor size and maximum pallet size according to some embodiments of the present disclosure.

[0080]

[0100] In some embodiments, the maximum pallet predictor size and maximum pallet size for a dual tree slice are set to half the maximum pallet predictor size and maximum pallet size for a single tree slice. As shown in Table 2, the maximum pallet predictor size and maximum pallet size for a combined pallet (i.e., for a single tree slice) are defined as 63 and 31, respectively. The maximum pallet predictor size and maximum pallet size for separate pallets for both the luma and chroma components (i.e., for a dual tree slice) are defined as 31 and 15, respectively.

[0081]

[0101] Figure 9 shows an exemplary Table 3 illustrating exemplary decoding processes using predefined maximum palette predictor sizes and maximum palette sizes according to several embodiments of the present disclosure. As shown in Table 3, changes to the palette mode decoding process currently proposed in VVC Draft 7 are highlighted in boxes 901-906 and in italics, and content removed from the palette mode decoding process currently proposed in VVC Draft 7 is shown in boxes 905-906, struck through and in italics. In this embodiment, when a CU is encoded as a local dual tree (e.g., a separate luma / chroma local tree of a single tree slice), the maximum predictor size for encoding the local dual tree is set to the maximum predictor size for the congruent palette.

[0082]

[0102] Figure 10 shows an exemplary Table 4 illustrating an exemplary palette coding syntax table for using predefined maximum palette predictor sizes and maximum palette sizes according to several embodiments of the present disclosure. Compared to the syntax used to implement the uniform maximum predictor sizes and maximum palette sizes shown in Table 1, changes to that syntax are highlighted in boxes 1001-1003 and in italics in Table 4, and syntax elements removed from that syntax are shown in boxes 1002-1003, struck through and in italics in Table 4.

[0083]

[0103] In some embodiments, the maximum palette size of the joint palette, and the difference between the maximum palette size and the maximum predictor size of the joint palette, are signaled to the decoder by SPS syntax. Exemplary semantics consistent with this embodiment are described below. The syntax element "sps_max_plt_size_joint_minus1" specifies the maximum allowable palette size of the joint palette table minus 1. The value of the syntax element sps_max_plt_size_joint_minus1 is in the range of 0 to 63 (including 0 and 63). If the syntax element sps_max_plt_size_joint_minus1 is not present, its value is inferred to be 0. The syntax element "sps_delta_max_plt_predictor_size_joint" specifies the difference between the maximum allowable palette predictor size and the maximum allowable palette size of the joint palette. The value of the syntax element sps_delta_max_plt_predictor_size_joint is within the range of 0 to 63 (including 0 and 63). If the syntax element sps_delta_max_plt_predictor_size_joint does not exist, its value is inferred to be 0.

[0084]

[0104] The maximum pallet size and maximum pallet predictor size for individual luma / chroma pallets are not signaled. Instead, they are derived from the maximum pallet size and maximum pallet predictor size for the combined pallet. Figure 11 shows an exemplary Table 5 illustrating exemplary derivations of the maximum pallet size and maximum pallet predictor size for individual pallets according to several embodiments of the present disclosure.

[0085]

[0105] In the examples shown in Table 5, when individual luma / chroma palettes are used, the syntax element max_plt_size_joint is equally distributed between the luma and chroma components. Consistent with this disclosure, the maximum palette size of a joint palette can also be unevenly distributed between the luma and chroma components. Figure 12 shows an exemplary Table 6 illustrating another exemplary derivation of the maximum palette size and maximum palette predictor size of individual palettes according to several embodiments of this disclosure. Table 6 shows an example of uneven distribution.

[0086]

[0106] Figure 13 shows an exemplary Table 7 illustrating exemplary Sequence Parameter Set (SPS) syntax tables for several embodiments of the present disclosure. Changes to the syntax compared to the syntax used to implement the uniform maximum predictor size and maximum palette size shown in Table 1 are highlighted in Table 7, within box 1301, and in italics. Although not shown in Table 7, it is intended that the maximum palette size and maximum palette predictor size for individual luma / chroma palettes may also be signaled in the SPS along with the maximum palette size and maximum palette predictor size for combined palettes.

[0087]

[0107] In some embodiments, syntax related to the maximum palette size and maximum palette predictor size is transmitted through the picture header (PH). Exemplary semantics consistent with this embodiment are described below. The syntax element "pic_max_plt_size_joint_minus1" specifies the maximum allowable palette size of the congruent palette table minus 1 for the slice associated with the PH. The value of the syntax element pic_max_plt_size_joint_minus1 is in the range of 0 to 63 (including 0 and 63). The syntax element "pic_delta_max_plt_predictor_size_joint" specifies the difference between the maximum allowable palette predictor size and the maximum allowable palette size of the congruent palette for the slice associated with the PH. The maximum allowable value of the syntax element pic_delta_max_plt_predictor_size_joint is 63. If the syntax element pic_delta_max_plt_predictor_size_joint is not present, its value is inferred to be 0.

[0088]

[0108] The maximum pallet size and maximum pallet predictor size for individual luma / chroma pallets are not signaled. Instead, they are derived from the maximum pallet size and maximum pallet predictor size for the combined pallet. Figure 14 shows an exemplary Table 8 illustrating another exemplary derivation of the maximum pallet size and maximum pallet predictor size for individual pallets according to some embodiments of the present disclosure.

[0089]

[0109] In the example shown in Table 8, the syntax element max_plt_size_joint is equally distributed between the luma and chroma components when individual luma / chroma palettes are used. Consistent with this disclosure, the maximum palette size of the joint table may also be unevenly distributed between the luma and chroma components. Figure 15 shows an exemplary Table 9 illustrating another exemplary derivation of the maximum palette size and maximum palette predictor size of individual palettes according to several embodiments of this disclosure. Table 9 shows an example of uneven distribution.

[0090]

[0110] Figure 16 shows an exemplary Table 10 illustrating exemplary PH syntax according to several embodiments of the present disclosure. Changes to the syntax compared to the syntax used to implement the uniform maximum predictor size and maximum palette size shown in Table 1 are highlighted in box 1601 and in italics in Table 10. Although not shown in Table 10, it is intended that the maximum palette size and maximum palette predictor size for individual luma / chroma palettes may also be signaled in the picture header along with the maximum palette size and maximum palette predictor size for combined palettes.

[0091]

[0111] In some embodiments, syntax related to the maximum palette size and maximum palette predictor size is signaled in each slice by the slice header. An exemplary semantics consistent with this embodiment is described below.

[0092]

[0112] Specifically, the syntax elements "slice_max_plt_size_joint_minus1" and "slice_delta_max_plt_predictor_size_joint" are conditionally signaled when a slice is encoded as a single tree slice. The syntax element slice_max_plt_size_joint_minus1 specifies the maximum allowed palette size of the congruent palette table minus 1 for a single tree slice. A bitstream compatibility requirement is that the maximum value of the syntax element slice_max_plt_size_joint be 63. The syntax element slice_delta_max_plt_predictor_size_joint specifies the difference between the maximum allowed palette predictor size and the maximum allowed palette size for the congruent palette for a single tree slice. The maximum allowed value of the syntax element slice_delta_max_plt_predictor_size_joint is 63. If the syntax element slice_delta_max_plt_predictor_size_joint does not exist, its value is inferred to be 0.

[0093]

[0113] The syntax elements "slice_max_plt_size_luma_minus1" and "slice_delta_max_plt_predictor_size_luma" are conditionally signaled when a slice is encoded as a dual-tree slice. The syntax element slice_max_plt_size_luma_minus1 specifies the maximum allowable palette size of the Luma palette table minus 1 for dual-tree slices. If the syntax element slice_max_plt_size_luma does not exist, its value is inferred to be 0. A maximum value of 63 for the syntax element slice_max_plt_size_luma_minus1 is a requirement for bitstream compatibility. The syntax element slice_delta_max_plt_predictor_size_luma specifies the difference between the maximum allowable palette predictor size and the maximum allowable palette size for the Luma palette for dual-tree slices. The maximum allowed value for the syntax element slice_delta_max_plt_predictor_size_luma is 63. If the syntax element slice_delta_max_plt_predictor_size_luma does not exist, its value is inferred to be 0.

[0094]

[0114] Figure 17 shows an exemplary Table 11 illustrating exemplary derivations of the maximum pallet size and maximum pallet predictor size for I-slice, P-slice, and B-slice according to several embodiments of the present disclosure.

[0095]

[0115] Figure 18 shows exemplary Table 12 illustrating exemplary SH syntax according to several embodiments of the present disclosure. Changes to the syntax compared to the syntax used to implement the uniform maximum predictor size and maximum palette size shown in Table 1 are highlighted in box 1801 and in italics in Table 12. The predictive update procedure corresponding to this embodiment is the same as that shown in Table 3, and the palette coding syntax corresponding to this embodiment is the same as that shown in Table 4.

[0096]

[0116] Figure 19 shows a flowchart of an exemplary palette coding method 1900 according to several embodiments of the present disclosure. Method 1900 can be performed by an encoder (e.g., by process 200A in Figure 2A or process 200B in Figure 2B), a decoder (e.g., by process 300A in Figure 3A or process 300B in Figure 3B), or by one or more software or hardware components of a device (e.g., device 400 in Figure 4). For example, a processor (e.g., processor 402 in Figure 4) can perform Method 1900. In some embodiments, Method 1900 can be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code executed by a computer (e.g., device 400 in Figure 4).

[0097]

[0117] Step 1901 allows for a decision on whether the lumens and chromens of the CU are encoded jointly or individually in palette mode. (For example, as shown in Table 3 of Figure 9 or Table 4 of Figure 10) The variable treeType can be used to indicate whether the lumens and chromens of the CU are encoded jointly or individually in palette mode.

[0098]

[0118] In step 1903, a first maximum palette table size for the CU can be determined in response to the fact that the luma and chroma components are jointly encoded in palette mode. In some embodiments, the first maximum palette table size for the CU can be determined based on the value of a first syntax element signaled in the video bitstream (e.g., the syntax element sps_max_plt_size_joint_minus1 shown in Table 7 of Figure 13, or the syntax element pic_max_plt_size_joint_minus1 shown in Table 10 of Figure 16).

[0099]

[0119] In step 1905, a first maximum palette predictor size for the CU can be determined in response to the fact that the luma and chroma components are jointly encoded in palette mode. In some embodiments, the first maximum palette predictor size for the CU can be determined based on the value of a first syntax element and the value of a second syntax element signaled in the video bitstream (e.g., the syntax element sps_delta_max_plt_predictor_size_joint shown in Table 7 of Figure 13, or the syntax element pic_delta_max_plt_predictor_size_joint shown in Table 10 of Figure 16). For example, the first maximum palette predictor size for the CU can be determined to be the sum of the value of the first syntax element and the value of the second syntax element (e.g., as shown in Table 5 of Figure 11, Table 6 of Figure 12, Table 8 of Figure 14, or Table 9 of Figure 15). In some embodiments, the first and second syntax elements are signaled in an SPS associated with a CU (for example, as shown in Table 7 of Figure 13) or in a PH associated with a CU (for example, as shown in Table 10 of Figure 16).

[0100]

[0120] In step 1907, in response to the luma and chroma components being congruently encoded in palette mode, the CU can be predicted based on the first maximum palette table size and the first maximum palette predictor size. For example, the CU can be predicted as shown in Table 3 of Figure 9.

[0101]

[0121] In some embodiments, Method 1900 may include determining a second maximum palette table size for CUs based on a first maximum palette table size, determining a second maximum palette predictor size for CUs based on a first maximum palette predictor size, and predicting CUs based on the second maximum palette table size and the second maximum palette predictor size, in response to the fact that the luma and chroma components are encoded separately in palette mode. The second maximum palette table size or the second maximum palette predictor size is related to the luma or chroma component. For example, the maximum palette table size or maximum palette predictor size for the luma or chroma component can be determined based on Table 5 in Figure 11, Table 6 in Figure 12, Table 8 in Figure 14, or Table 9 in Figure 15.

[0102]

[0122] In some embodiments, method 1900 may include determining a first maximum palette table size for CUs such that it is a first predetermined value. Method 1900 may also include determining a first maximum palette predictor size for CUs such that it is a second predetermined value. For example, as shown in Table 2 of Figure 8, the maximum palette table size for a congruent palette may be 31, and the maximum palette predictor size for a congruent palette may be 63. In some embodiments, method 1900 may include determining a third maximum palette table size for CUs such that it is a third predetermined value, in response to the fact that the luma and chroma components are encoded separately in palette mode, and predicting CUs based on the third maximum palette table size. The third predetermined value is smaller than the first predetermined value. For example, as shown in Table 2 of Figure 8, the maximum palette table size for a luma or chroma palette may be 15.

[0103]

[0123] In some embodiments, method 1900 may include predicting the CU based on a first maximum palette predictor size (as shown in Table 3 of Figure 9), in response to the luma and chroma components being encoded separately in palette mode and the CU being part of a single tree slice.

[0104]

[0124] In some embodiments, method 1900 may include determining a third maximum palette predictor size for a CU such that it is a fourth predetermined value, in response to the fact that the luma and chroma components are encoded separately in palette mode and the CU is not part of a single tree slice, and predicting the CU based on the third maximum palette predictor size. The fourth predetermined value is smaller than the second predetermined value. For example, the maximum palette predictor size for individual palettes may be 31, as shown in Table 2 of Figure 8. The CU can be predicted as shown in Table 3 of Figure 9.

[0105]

[0125] In some embodiments, Method 1900 may include determining whether a picture slice containing a CU is a single tree slice or a dual tree slice; determining a first maximum palette table size for the CU in the picture slice based on the value of a third syntax element signaled in the slice header of the picture slice, in response to the picture slice being a single tree slice; and determining a first maximum palette predictor size for the CU based on the value of the third syntax element and the value of a fourth syntax element signaled in the slice header. The first maximum palette predictor size for the CU can be determined to be the sum of the values ​​of the third and fourth syntax elements. For example, as shown in Table 11 of Figure 17, in response to the picture slice being a single tree slice (e.g., slice_type != I | | qtbtt_dual_tree_intra_flag == 0), the maximum palette table size for a congruent palette can be determined based on the value of the syntax element slice_max_plt_size_joint_minus1 signaled in the slice header (e.g., SH as shown in Table 12 of Figure 18), and the maximum palette predictor size for a congruent palette can be determined to be the sum of the value of the syntax element slice_max_plt_size_joint_minus1 and the value of the syntax element slice_delta_max_plt_predictor_size_joint signaled in the slice header (e.g., SH as shown in Table 12 of Figure 18).

[0106]

[0126] In some embodiments, Method 1900 may include, in response to the picture slice being a dual-tree slice, determining a fourth maximum palette table size for the CU based on the value of a fifth syntax element signaled in the slice header; determining a fourth maximum palette predictor size for the CU based on the value of the fifth syntax element and the value of a sixth syntax element signaled in the slice header; and predicting the CU based on the fourth maximum palette table size and the fourth maximum palette predictor size. The fourth maximum palette predictor size for the CU can be determined to be the sum of the value of the fifth syntax element and the value of the sixth syntax element. For example, as shown in Table 11 of Figure 17, in response to the picture slice being a dual-tree slice, the maximum palette table size for a luma or chroma palette can be determined based on the value of the syntax element slice_max_plt_size_luma_minus1 signaled in the slice header (e.g., SH as shown in Table 12 of Figure 18), and the maximum palette predictor size for a luma or chroma palette can be determined to be the sum of the value of the syntax element slice_max_plt_size_luma_minus1 and the value of the syntax element slice_delta_max_plt_predictor_size_luma signaled in the slice header (e.g., SH as shown in Table 12 of Figure 18).

[0107]

[0127] Embodiments can be further described using the following clauses. 1. Determine whether the luma component and chroma component of the coding unit (CU) are coded jointly or individually in palette mode. In response to the luma and chroma components being jointly encoded in palette mode, To determine the first maximum pallet table size for CU, To determine the first maximum pallet predictor size for CU, and Predicting CU based on the first maximum pallet table size and the first maximum pallet predictor size, A palette encoding method that includes this. 2. Determine the first maximum pallet table size for CU. The method according to Clause 1, comprising determining a first maximum palette table size for a CU based on the value of a first syntax element signaled in the video bitstream. 3. Determine the first maximum pallet predictor size for CU. The method according to Clause 2, comprising determining a first maximum palette predictor size for a CU based on the value of a first syntax element and the value of a second syntax element signaled in the video bitstream. 4. Determine the first maximum pallet predictor size for CU. The method according to Clause 3, comprising determining a first maximum palette predictor size for a CU such that it is the sum of the values ​​of the first syntax element and the second syntax element. 5. The method according to clause 3 or 4, wherein the first syntax element and the second syntax element are signaled in the sequence parameter set (SPS) associated with the CU. 6. The method according to Clause 3 or 4, wherein the first syntax element and the second syntax element are signaled in the picture header (PH) associated with the CU. 7. In response to the fact that the luma and chroma components are encoded separately in palette mode, Based on the first maximum pallet table size, determine the second maximum pallet table size for the CU. Based on the first maximum pallet predictor size, a second maximum pallet predictor size for CU is determined, and Predicting CU based on the second maximum pallet table size and the second maximum pallet predictor size, The method described in any one of the clauses 1 to 6, further including the method described in any one of the clauses 1 to 6. 8. The method according to Clause 7, wherein the second maximum pallet table size or the second maximum pallet predictor size relates to the luma component or the chroma component. 9. Determine the first maximum pallet table size for CU. The method according to Clause 1, comprising determining a first maximum pallet table size for a CU such that it is a first predetermined value. 10. Determine the first maximum pallet predictor size for CU. The method according to clause 1 or 9, comprising determining a first maximum pallet predictor size for CU such that it becomes a second predetermined value. 11. In response to the fact that the luma and chroma components are encoded separately in palette mode, Determine the third maximum pallet table size for CU so that it becomes a third predetermined value, and Predicting CU based on the third maximum palette table size, It further includes, The method according to clause 9 or 10, wherein the third predetermined value is less than the first predetermined value. 12. The method according to clause 11, further comprising predicting the CU based on a first maximum palette predictor size in response that the lumar and chromar components are encoded separately in palette mode and that the CU is part of a single tree slice. 13. In response to the fact that the luma and chroma components are encoded separately in palette mode, and that the CU is not part of a single tree slice, Determine the third maximum pallet predictor size for CU so that it becomes a fourth predetermined value, and Predicting CU based on the third maximum palette predictor size, It further includes, The method according to any one of the clauses 10 to 12, wherein the fourth predetermined value is less than the second predetermined value. 14. Determine whether the picture slice containing CU is a single tree slice or a dual tree slice, In response to the fact that the picture slice is a single tree slice, Based on the value of the third syntax element signaled in the slice header of the picture slice, the first maximum palette table size for CU in the picture slice is determined, and Based on the value of the third syntax element and the value of the fourth syntax element signaled in the slice header, the first maximum palette predictor size for the CU is determined, The method described in Clause 1, further including the method described in Clause 1. 15. Determining the first maximum pallet predictor size for CU, The method according to Clause 14, comprising determining a first maximum palette predictor size for CU such that it is the sum of the values ​​of the third syntax element and the fourth syntax element. 16. In response to the picture slice being a dual tree slice, Based on the value of the fifth syntax element signaled in the slice header, determine the fourth maximum palette table size for the CU. Based on the value of the fifth syntax element and the value of the sixth syntax element signaled in the slice header, the fourth maximum palette predictor size for the CU is determined, and Based on the fourth maximum pallet table size and the fourth maximum pallet predictor size, predict the CU. The method described in Clause 14 or 15, further including the method described in Clause 14 or 15. 17. Determining the fourth maximum pallet predictor size for CU is The method according to Clause 16, comprising determining a fourth maximum palette predictor size for CU such that it is the sum of the values ​​of the fifth syntax element and the sixth syntax element. 18. Image processing device, At least one memory for storing instructions, It includes at least one processor, and at least one processor is The process involves determining whether the luma component and chroma component of an encoding unit (CU) are encoded jointly or individually in palette mode, In response to the luma and chroma components being jointly encoded in palette mode, To determine the first maximum pallet table size for CU, To determine the first maximum pallet predictor size for CU, and Predicting CU based on the first maximum pallet table size and the first maximum pallet predictor size, A video processing device configured to execute commands to cause a device to perform an action. 19. At least one processor, To determine a first maximum palette table size for a CU based on the value of a first syntax element signaled in the video bitstream, The apparatus described in Clause 18, configured to execute instructions in order to cause the apparatus to perform the following. 20. At least one processor, Based on the value of the first syntax element and the value of the second syntax element signaled in the video bitstream, the first maximum palette predictor size for the CU is determined. The apparatus described in Clause 19, configured to execute instructions in order to cause the apparatus to perform the following. 21. At least one processor, Determine the first maximum palette predictor size for CU such that it is the sum of the values ​​of the first syntax element and the second syntax element. The apparatus described in Clause 20, configured to execute instructions in order to cause the apparatus to perform the following. 22. The apparatus according to Clause 20 or 21, wherein the first syntax element and the second syntax element are signaled in a sequence parameter set (SPS) associated with the CU. 23. The apparatus according to Clause 20 or 21, wherein the first syntax element and the second syntax element are signaled in a picture header (PH) associated with the CU. 24. At least one processor, In response to the fact that the luma and chroma components are encoded separately in palette mode, Based on the first maximum pallet table size, determine the second maximum pallet table size for the CU. Based on the first maximum pallet predictor size, a second maximum pallet predictor size for CU is determined, and Predicting CU based on the second maximum pallet table size and the second maximum pallet predictor size, A device as described in any one of clauses 18 to 23, configured to execute instructions in order to cause the device to perform a certain action. 25. The apparatus according to Clause 24, wherein the second maximum pallet table size or the second maximum pallet predictor size relates to the luma component or the chroma component. 26. At least one processor, Determine the first maximum pallet table size for CU so that it becomes a first predetermined value. The apparatus described in Clause 18, configured to execute instructions in order to cause the apparatus to perform the following. 27. At least one processor, Determine the first maximum pallet predictor size for CU so that it becomes a second predetermined value. The apparatus described in Clause 18 or 26, configured to execute instructions in order to cause the apparatus to perform the following. 28. At least one processor, In response to the fact that the luma and chroma components are encoded separately in palette mode, Determine the third maximum pallet table size for CU so that it becomes a third predetermined value, and Predicting CU based on the third maximum palette table size, It is configured to execute commands in order to cause the device to do so. The apparatus according to clause 26 or 27, wherein the third predetermined value is smaller than the first predetermined value. 29. At least one processor, In response to the fact that the luma and chroma components are encoded separately in palette mode, and that the CU is part of a single tree slice, predict the CU based on a first maximum palette predictor size. The apparatus described in Clause 28, configured to execute instructions in order to cause the apparatus to perform the following. 30. At least one processor, In response to the fact that the luma and chroma components are encoded separately in palette mode, and that the CU is not part of a single tree slice, Determine the third maximum pallet predictor size for CU so that it becomes a fourth predetermined value, and Predicting CU based on the third maximum palette predictor size, It is configured to execute commands in order to cause the device to do so. The apparatus described in any one of clauses 27 to 29, wherein the fourth predetermined value is smaller than the second predetermined value. 31. At least one processor, Determining whether a picture slice containing a CU is a single-tree slice or a dual-tree slice, In response to the fact that the picture slice is a single tree slice, Based on the value of the third syntax element signaled in the slice header of the picture slice, the first maximum palette table size for CU in the picture slice is determined, and Based on the value of the third syntax element and the value of the fourth syntax element signaled in the slice header, the first maximum palette predictor size for the CU is determined, The apparatus described in Clause 18, configured to execute instructions in order to cause the apparatus to perform the following. 32. At least one processor, Determine the first maximum palette predictor size for CU such that it is the sum of the values ​​of the third syntax element and the fourth syntax element. The apparatus described in Clause 31, configured to execute instructions in order to cause the apparatus to perform the following. 33. At least one processor, In response to the picture slice being a dual tree slice, Based on the value of the fifth syntax element signaled in the slice header, determine the fourth maximum palette table size for the CU. Based on the value of the fifth syntax element and the value of the sixth syntax element signaled in the slice header, the fourth maximum palette predictor size for the CU is determined, and Based on the fourth maximum pallet table size and the fourth maximum pallet predictor size, predict the CU. The apparatus according to clause 31 or 32, configured to execute instructions in order to cause the apparatus to perform the following. 34. At least one processor, Determine the fourth maximum palette predictor size for CU such that it is the sum of the values ​​of the fifth syntax element and the sixth syntax element. The apparatus described in Clause 33, configured to execute instructions in order to cause the apparatus to perform the following. 35. A non-temporary computer-readable storage medium storing an instruction set, wherein the instruction set is The process involves determining whether the luma component and chroma component of an encoding unit (CU) are encoded jointly or individually in palette mode, In response to the luma and chroma components being jointly encoded in palette mode, To determine the first maximum pallet table size for CU, To determine the first maximum pallet predictor size for CU, and Predicting CU based on the first maximum pallet table size and the first maximum pallet predictor size, A non-temporary computer-readable storage medium that can be executed by one or more processing devices to cause a video processing device to perform a method including the above. 36. The instruction set is To determine a first maximum palette table size for a CU based on the value of a first syntax element signaled in the video bitstream, A non-temporary computer-readable storage medium as described in Clause 35, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 37. The instruction set is Based on the value of the first syntax element and the value of the second syntax element signaled in the video bitstream, the first maximum palette predictor size for the CU is determined. A non-temporary computer-readable storage medium as described in Clause 36, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 38. The instruction set is Determine the first maximum palette predictor size for CU such that it is the sum of the values ​​of the first syntax element and the second syntax element. A non-temporary computer-readable storage medium as described in Clause 37, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 39. A non-temporary computer-readable storage medium as described in Clause 37 or 38, wherein the first syntax element and the second syntax element are signaled in a sequence parameter set (SPS) associated with the CU. 40. A non-temporary computer-readable storage medium as described in Clause 37 or 38, wherein the first syntax element and the second syntax element are signaled in the picture header (PH) associated with the CU. 41. The instruction set is In response to the fact that the luma and chroma components are encoded separately in palette mode, Based on the first maximum pallet table size, determine the second maximum pallet table size for the CU. Based on the first maximum pallet predictor size, a second maximum pallet predictor size for CU is determined, and Predicting CU based on the second maximum pallet table size and the second maximum pallet predictor size, A non-temporary computer-readable storage medium as described in any one of the clauses 35 to 40, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 42. A non-temporary computer-readable storage medium as described in Clause 41, wherein the second maximum pallet table size or the second maximum pallet predictor size relates to a luma component or a chroma component. 43. The instruction set is Determine the first maximum pallet table size for CU so that it becomes a first predetermined value. A non-temporary computer-readable storage medium as described in Clause 35, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 44. The instruction set is Determine the first maximum pallet predictor size for CU so that it becomes a second predetermined value. A non-temporary computer-readable storage medium as described in Clause 35 or 43, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 45. The instruction set is In response to the fact that the luma and chroma components are encoded separately in palette mode, Determine the third maximum pallet table size for CU so that it becomes a third predetermined value, and Predicting CU based on the third maximum palette table size, To have the video processing device perform this, it can be executed by one or more processing devices. A non-temporary computer-readable storage medium as described in Clause 43 or 44, wherein the third predetermined value is less than the first predetermined value. 46. ​​The instruction set is In response to the fact that the luma and chroma components are encoded separately in palette mode, and that the CU is part of a single tree slice, predict the CU based on a first maximum palette predictor size. A non-temporary computer-readable storage medium as described in Clause 45, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 47. The instruction set is In response to the fact that the luma and chroma components are encoded separately in palette mode, and that the CU is not part of a single tree slice, Determine the third maximum pallet predictor size for CU so that it becomes a fourth predetermined value, and Predicting CU based on the third maximum palette predictor size, To have the video processing device perform this, it can be executed by one or more processing devices. A non-temporary computer-readable storage medium as described in any one of clauses 44 to 46, wherein the fourth predetermined value is less than the second predetermined value. 48. The instruction set is Determining whether a picture slice containing a CU is a single-tree slice or a dual-tree slice, In response to the fact that the picture slice is a single tree slice, Based on the value of the third syntax element signaled in the slice header of the picture slice, the first maximum palette table size for CU in the picture slice is determined, and Based on the value of the third syntax element and the value of the fourth syntax element signaled in the slice header, the first maximum palette predictor size for the CU is determined, A non-temporary computer-readable storage medium as described in Clause 35, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 49. The instruction set is Determine the first maximum palette predictor size for CU such that it is the sum of the values ​​of the third syntax element and the fourth syntax element. A non-temporary computer-readable storage medium as described in Clause 48, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 50. The instruction set is In response to the picture slice being a dual tree slice, Based on the value of the fifth syntax element signaled in the slice header, determine the fourth maximum palette table size for the CU. Based on the value of the fifth syntax element and the value of the sixth syntax element signaled in the slice header, the fourth maximum palette predictor size for the CU is determined, and Based on the fourth maximum pallet table size and the fourth maximum pallet predictor size, predict the CU. A non-temporary computer-readable storage medium as described in Clause 48 or 49, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action. 51. The instruction set is Determine the fourth maximum palette predictor size for CU such that it is the sum of the values ​​of the fifth syntax element and the sixth syntax element. A non-temporary computer-readable storage medium as described in Clause 50, which can be executed by one or more processing devices in order to cause a video processing device to perform the same action.

[0108]

[0128] In some embodiments, non-temporary computer-readable storage media containing instructions are also provided, which may be executed by a device (such as an encoder and decoder as disclosed) to perform the above-described method. Common forms of non-temporary media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tapes, or other magnetic data storage media, CD-ROMs, other optical data storage media, any physical media having a pattern of holes, RAM, PROMs, and EPROMs, FLASH®-EPROMs or other flash memory, NVRAMs, caches, registers, other memory chips or cartridges, and networked versions thereof. The device may include one or more processors (CPUs), input / output interfaces, network interfaces, and / or memory.

[0109]

[0129] It should be noted that relational terms as herein, such as “first” and “second,” are used solely to distinguish one entity or action from another, and do not require or imply any actual relationship or order between these entities or actions. Furthermore, the words “comprising,” “having,” “containing,” and “including,” as well as other similar forms, are intended to be open-ended in that they are equivalent in meaning, and that the one or more items following any one of these words are not exhaustive enumerations of such items, or are not limited to only the enumerated one or more items.

[0110]

[0130] In this specification, unless otherwise specified, the term “or” encompasses all possible combinations, unless impractical. For example, if it is stated that a database may contain A or B, then unless otherwise specifically stated or impractical, the database may contain A or B, or A and B. As a second example, if it is stated that a database may contain A, B, or C, then unless otherwise specified or impractical, the database may contain A or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0111]

[0131] It is understood that the embodiments described above may be implemented by hardware, software (program code), or a combination of hardware and software. If implemented by software, it may be stored on the computer-readable medium described above. The software may perform the methods disclosed when executed by a processor. The computing units and other functional units described in this disclosure may be implemented by hardware, software, or a combination of hardware and software. Those skilled in the art will understand that several of the above modules / units may be integrated into a single module / unit, and that each of the above modules / units may be further divided into several submodules / subunits.

[0112]

[0132] The above specification describes embodiments with respect to numerous specific details that may vary depending on the embodiment. Specific adaptations and modifications of the described embodiments may be made. Other embodiments may become apparent to those skilled in the art by considering the specification and practice of the invention disclosed herein. The above specification and examples are intended to be merely illustrative, and the true scope and spirit of the invention are shown by the following claims. Furthermore, the order of steps shown in the drawings is intended to be for illustrative purposes only and is not intended to limit the steps to any particular order. Therefore, those skilled in the art will understand that these steps may be performed in different orders while carrying out the same method.

[0113]

[0133] The drawings and specification disclose exemplary embodiments. However, many variations and modifications can be made to these embodiments. Therefore, while specific terms are used, they are used merely in a general and descriptive sense and are not intended to be limiting.

Claims

1. A palette encoding method implemented in a decoder, The process involves determining whether the luma component and the chroma component of an encoding unit (CU) are encoded jointly or individually in palette mode, In response to the fact that the luma component and the chroma component are encoded individually in the palette mode, The maximum individual pallet table size for the aforementioned CU is determined to be smaller than the maximum combined pallet table size. To determine the maximum individual pallet predictor size for the CU, and Predicting the CU based on the maximum individual pallet table size and the maximum individual pallet predictor size, A palette encoding method that includes this.

2. The method according to claim 1, wherein the maximum individual pallet table size or the maximum individual pallet predictor size relates to the luma component or the chroma component.

3. Determining the maximum individual pallet table size for the CU is This includes determining the maximum individual palette table size for the CU based on the value of a first syntax element signaled in the video bitstream, Determining the maximum individual pallet predictor size for the CU is The method according to claim 1, comprising determining the maximum individual palette predictor size for the CU based on the value of the first syntax element and the value of the second syntax element signaled in the video bitstream.

4. Determining the maximum individual pallet predictor size for the CU is The method according to claim 3, comprising determining the maximum individual pallet predictor size for the CU such that it is the sum of the value of the first syntax element and the value of the second syntax element.

5. The method according to claim 3, wherein the first syntax element and the second syntax element are signaled in a sequence parameter set (SPS) or picture header (PH) associated with the CU.

6. Determining the maximum individual pallet table size for the CU is The method according to claim 1, comprising determining the maximum individual pallet table size for the CU based on the maximum combined pallet table size.

7. Determining the maximum individual pallet predictor size for the CU is The method according to claim 6, comprising determining the maximum individual pallet predictor size for the CU based on the maximum combined pallet predictor size.

8. In response to the CU being part of a single tree slice, determining the maximum individual pallet table size for the CU is: This includes determining the maximum individual pallet table size such that it is smaller than the maximum combined pallet table size. The method according to claim 7, wherein predicting the CU includes predicting the CU based on the maximum combined pallet predictor size.

9. In response to the fact that the CU is not part of a single tree slice, determining the maximum individual pallet predictor size for the CU is: This includes determining the maximum individual pallet predictor size such that it is smaller than the maximum combined pallet predictor size. The method according to claim 7, wherein predicting the CU includes predicting the CU based on the maximum individual pallet predictor size.

10. Determining whether the picture slice containing the CU is a single tree slice or a dual tree slice, In response to the fact that the picture slice is a single tree slice, Based on the value of the first syntax element signaled in the slice header of the picture slice, the maximum congruent palette table size for the CU in the picture slice is determined, and Based on the value of the first syntax element and the value of the second syntax element signaled in the slice header, the maximum congruent pallet predictor size for the CU is determined, The method according to claim 7, further comprising:

11. Determining the maximum combined pallet predictor size for the CU is The method according to claim 10, comprising determining the maximum combined pallet predictor size for the CU such that it is the sum of the value of the first syntax element and the value of the second syntax element.

12. In response to the fact that the picture slice is a dual tree slice, Based on the value of the third syntax element signaled in the slice header, the maximum individual pallet table size for the CU is determined, and Based on the value of the third syntax element and the value of the fourth syntax element signaled in the slice header, the maximum individual pallet predictor size for the CU is determined, The method according to claim 10, further comprising:

13. Determining the maximum individual pallet predictor size for the CU is The method according to claim 12, comprising determining the maximum individual pallet predictor size for the CU such that it is the sum of the value of the third syntax element and the value of the fourth syntax element.

14. The method according to claim 1, wherein the maximum individual pallet table size is 15.

15. The method according to claim 14, wherein the maximum combined pallet table size is 31.

16. A video processing device implemented in an encoder, At least one memory for storing instructions, It includes at least one processor, and the at least one processor is The process involves determining whether the luma component and the chroma component of an encoding unit (CU) are encoded jointly or individually in palette mode, In response to the fact that the luma component and the chroma component are encoded individually in the palette mode, The maximum individual pallet table size for the aforementioned CU is determined to be smaller than the maximum combined pallet table size. To determine the maximum individual pallet predictor size for the CU, and Predicting the CU based on the maximum individual pallet table size and the maximum individual pallet predictor size, A video processing device configured to execute the command in order to cause the device to perform the above action.

17. A method for storing a video bitstream, wherein the method is Receiving a video sequence, Encoding one or more pictures of the aforementioned video sequence, The process of generating a bitstream based on the aforementioned encoding, The bitstream is stored in a non-temporary computer-readable storage medium, Including the above encoding, The process involves determining whether the luma component and the chroma component of an encoding unit (CU) are encoded jointly or individually in palette mode, In response to the fact that the luma component and the chroma component are encoded individually in the palette mode, The maximum individual pallet table size for the aforementioned CU is determined to be smaller than the maximum combined pallet table size. To determine the maximum individual pallet predictor size for the CU, and Predicting the CU based on the maximum individual pallet table size and the maximum individual pallet predictor size, Methods that include...

Images