Method for pallet prediction

Abstract

To provide a computer-implemented method for encoding video.SOLUTION: The method includes: receiving a video frame for processing; generating one or more coding units of the video frame; and processing one or more coding units using one or more palette predictors having palette entries, where each palette entry of the one or more palette predictors has a corresponding reuse flag, and where the number of reuse flags for each palette predictor is set to a fixed number for a corresponding coding unit.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] Cross - reference to Related Applications

[0001] This disclosure claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 005,305, filed Apr. 4, 2020, and U.S. Provisional Patent Application No. 63 / 002,594, filed Mar. 31, 2020, both of which are hereby incorporated by reference in their entirety.

[0002] Technical Field

[0002] This disclosure generally relates to video processing, and more particularly, to the use of palette modes in video encoding and decoding.

Background Art

[0003] Background

[0003] Video is a set of still pictures (or "frames") that capture visual information. To reduce memory storage and transmission bandwidth, video can be compressed before storage or transmission and decompressed before display. The compression process is usually called encoding, and the decompression process is usually called decoding. There are various video coding formats that use standardized video coding techniques, most commonly techniques based on prediction, transformation, quantization, entropy coding, and in - loop filtering. Video coding standards such as the HEVC (High Efficiency Video Coding) (HEVC / H.265) standard, the VVC (Versatile Video Coding) (VVC / H.266) standard, and the AVS standard, which specify a particular video coding format, have been developed by standardization organizations. As more advanced video coding techniques are adopted in video standards, the coding efficiency of new video coding standards is becoming higher.

Summary of the Invention

Means for Solving the Problems

[0004] Summary of the Disclosure

[0004] Embodiments of the present disclosure provide a computer implementation method for a palette predictor. In some embodiments, the method includes receiving a video frame for processing, generating one or more coded units of the video frame, and processing one or more coded units using one or more palette predictors having palette entries, each palette entry of one or more palette predictors having a corresponding reuse flag, and the number of reuse flags for each palette predictor is set to a fixed number for the corresponding coded unit.

[0005]

[0005] Embodiments of the present disclosure provide a device. In some embodiments, the device includes a memory configured to store instructions and a processor coupled to the memory, the processor being configured to execute instructions causing the device to receive video frames for processing, generate one or more coded units of video frames, and process one or more coded units using one or more palette predictors having palette entries, each palette entry of one or more palette predictors having a corresponding reuse flag, the number of reuse flags for each palette predictor being set to a fixed number for the corresponding coded unit.

[0006]

[0006] Embodiments of the present disclosure provide a non-temporary computer-readable storage medium for storing a set of instructions, the set of instructions being executable by one or more processors of the device to cause the device to initiate a method for performing video data processing. In some embodiments, the method includes receiving video frames for processing, generating one or more coded units of the video frames, and processing one or more coded units using one or more palette predictors having palette entries, each palette entry of one or more palette predictors having a corresponding reuse flag, the number of reuse flags of each palette predictor being set to a fixed number for the corresponding coded unit.

[0007]

[0007] Embodiments of the present disclosure provide a computer implementation method for deblocking a palette mode filter. In some embodiments, the method includes receiving a video frame for processing; generating one or more coding units of the video frame, each coding unit having one or more coding blocks; and setting a boundary filter intensity to 1 in response that at least a first coding block of two adjacent coding blocks is coded in palette mode and a second coding block of two adjacent coding blocks has a coding mode different from palette mode.

[0008]

[0008] Embodiments of the present disclosure provide a device. In some embodiments, the device includes a memory configured to store instructions and a processor coupled to the memory, the processor being configured to execute instructions to cause the device to receive a video frame for processing, generate one or more coded units of the video frame, each coded unit having one or more coded blocks, and set a boundary filter intensity to 1 in response that at least a first coded block of two adjacent coded blocks is coded in palette mode and a second coded block of two adjacent coded blocks has a coding mode different from palette mode.

[0009]

[0009] Embodiments of the present disclosure provide a non-temporary computer-readable storage medium for storing a set of instructions, the set of instructions being executable by one or more processors of the device to cause the device to initiate a method for performing video data processing. In some embodiments, the method includes receiving a video frame for processing; receiving a video frame for processing and generating one or more coded units of the video frame, each coded unit having one or more coded blocks; and setting a boundary filter strength to 1 in response that at least a first coded block of two adjacent coded blocks is coded in palette mode and a second coded block of two adjacent coded blocks has a coding mode different from palette mode.

[0010] Brief explanation of the drawing

[0010] Embodiments and various aspects of the present disclosure are shown in the following detailed description and accompanying drawings. Various features shown in the drawings are not drawn to scale. [Brief explanation of the drawing]

[0011] [Figure 1]

[0011] This is a schematic diagram illustrating the structure of an example of a video sequence according to some embodiments of the present disclosure. [Figure 2A]

[0012] This is a schematic diagram illustrating an exemplary encoding process of a hybrid video coding system consistent with embodiments of the present disclosure. [Figure 2B]

[0013] This schematic diagram illustrates another exemplary encoding process of a hybrid video coding system consistent with embodiments of the present disclosure. [Figure 3A]

[0014] This is a schematic diagram illustrating an exemplary decoding process of a hybrid video coding system consistent with embodiments of the present disclosure. [Figure 3B]

[0015] It is a schematic diagram illustrating another exemplary decoding process of a hybrid video coding system that conforms to an embodiment of the present disclosure. [Figure 4]

[0016] It is a block diagram of an exemplary apparatus for encoding or decoding video according to some embodiments of the present disclosure. [Figure 5]

[0017] It illustrates a block encoded in palette mode according to some embodiments of the present disclosure. [Figure 6]

[0018] An example of a palette predictor update process is shown. [Figure 7]

[0019] An example of a palette coding syntax is shown. [Figure 8]

[0020] It shows a flowchart of a palette predictor update process according to some embodiments of the present disclosure. [Figure 9]

[0021] An example of a palette predictor update process according to some embodiments of the present disclosure is shown. [Figure 10]

[0022] An example of a decoding process in palette mode is shown. [Figure 11]

[0023] An example of a decoding process in palette mode according to some embodiments of the present disclosure is shown. [Figure 12]

[0024] It shows a flowchart of another palette predictor update process according to some embodiments of the present disclosure. [Figure 13]

[0025] An example of another palette predictor update process according to some embodiments of the present disclosure is shown. [Figure 14]

[0026] An example of a palette coding syntax according to some embodiments of the present disclosure is shown. [Figure 15]

[0027] An example of a decoding process in palette mode according to some embodiments of the present disclosure is shown. [Figure 16]

[0028] This diagram illustrates hardware designs for a decoder for palette mode that implement a portion of the palette predictor update process according to some embodiments of the present disclosure. [Figure 17]

[0029] The following are examples of a palette mode decoding process according to some embodiments of the present disclosure. [Figure 18]

[0030] This shows an example of the initialization process for palette mode. [Figure 19]

[0031] The following are examples of the initialization process for palette mode according to some embodiments of the present disclosure. [Figure 20]

[0032] This shows an example of run-length coding for updating the palette predictor and the corresponding reuse flag. [Figure 21]

[0033] Examples of run-length coding for palette predictor updates and corresponding reuse flags according to several embodiments of this disclosure are shown. [Figure 22]

[0034] Examples of run-length coding for palette predictor updates and corresponding reuse flags according to several embodiments of this disclosure are shown. [Figure 23]

[0035] Examples of palette coding syntax according to some embodiments of this disclosure are shown below. [Figure 24]

[0036] An example of palette-coded semantics is shown. [Figure 25]

[0037] Examples of palette-coded semantics according to some embodiments of this disclosure are shown. [Modes for carrying out the invention]

[0012] Detailed explanation

[0038] Herein, exemplary embodiments are described in detail, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, where, unless otherwise noted, the same number in different drawings represents the same or similar elements. The implementations described below in the description of exemplary embodiments do not represent all implementations consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with the embodiments relating to the present invention enumerated in the accompanying claims. Specific embodiments of this disclosure are described in more detail below. In the event of any conflict between terms and definitions used herein and terms and / or definitions incorporated by reference, the description herein shall prevail.

[0013]

[0039] The Joint Video Experts Team (JVET) of the ITU-T Video Coding Expert Group (ITU-T VCEG) and the ISO / IEC Moving Picture Expert Group (ISO / IEC MPEG) is currently developing the Versatile Video Coding (VVC / H.266) standard. The VVC standard aims to double the compression efficiency of its predecessor, the High Efficiency Video Coding (HEVC / H.265) standard. In other words, the goal of VVC is to achieve the same subjective quality as HEVC / H.265 using half the bandwidth.

[0014]

[0040] To achieve the same subjective quality as HEVC / H.265 using half the bandwidth, JVET has developed a technology that surpasses HEVC using the Joint Exploration Model (JEM) reference software. Because coding technology is integrated into JEM, JEM achieves significantly improved coding performance compared to HEVC.

[0015]

[0041] The VVC standard is a recent development and continues to incorporate more coding technologies to provide better compression performance. VVC is based on the same hybrid video coding system used in modern video compression standards such as HEVC, H.264 / AVC, MPEG2, H.263, and so on.

[0016]

[0042] An image is a set of still pictures (or "frames") arranged in chronological order to store visual information. An image capture device (e.g., a camera) can be used to capture and store these pictures in chronological order, and an image playback device (e.g., a television, computer, smartphone, tablet computer, video player, or any end-user terminal with display capabilities) can be used to display such pictures in chronological order. Furthermore, in some applications, such as surveillance, conferencing, or live broadcasting, the image capture device can transmit the captured image in real time to an image playback device (e.g., a computer with a monitor).

[0017]

[0043] To reduce the memory space and transmission bandwidth required for such applications, video can be compressed before storage and transmission, and decompressed before display. This compression and decompression can be implemented by software executed by a processor (e.g., a general-purpose computer processor) or dedicated hardware. The module for compression is generally called an "encoder," and the module for decompression is generally called a "decoder." Encoders and decoders can be collectively called a "codec." Encoders and decoders can be implemented as various appropriate hardware, software, or a combination thereof. For example, hardware implementations of encoders and decoders include circuits such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, or any combination thereof. Software implementations of encoders and decoders include program code, computer executable instructions, firmware, or any appropriate computer implementation algorithm or process fixed in a computer-readable medium. Video compression and decompression can be implemented using various algorithms or standards such as MPEG-1, MPEG-2, MPEG-4, and the H.26x series. In some applications, a codec can decompress video using a first encoding standard and then recompress the decompressed video using a second encoding standard; in this case, the codec can be called a "transcoder."

[0018]

[0044] A video encoding process can identify and retain useful information that can be used to reconstruct a picture, while ignoring information that is not important for reconstruction. If the ignored, non-essential information cannot be fully reconstructed, such an encoding process can be called "reversible." Otherwise, the encoding process can be called "reversible." Most encoding processes are irreversible, which is a trade-off to reduce the required memory space and transmission bandwidth.

[0019]

[0045] Useful information in an encoded picture (referred to as the "current picture") includes changes relative to a reference picture (e.g., a previously encoded and reconstructed picture). Such changes can include changes in pixel position, brightness, or color, of which position changes are the most important. Changes in the position of a group of pixels representing an object can reflect the movement of the object between the reference picture and the current picture.

[0020]

[0046] A picture that is coded without referencing another picture (i.e., a picture is its own reference picture) is called an "I-picture". A picture is called a "P-picture" if some or all of the blocks within it (for example, a block that generally refers to a portion of a video picture) are predicted using intra-prediction or inter-prediction with one reference picture (e.g., one-way prediction). A picture is called a "B-picture" if at least one block within it is predicted using two reference pictures (e.g., two-way prediction).

[0021]

[0047] Figure 1 shows the structure of an example of a video sequence 100 according to some embodiments of the present disclosure. The video sequence 100 can be live video or captured and archived video. The video 100 can be real video, computer-generated video (e.g., computer game video), or a combination thereof (e.g., real video with augmented reality effects). The video sequence 100 can be input from a video capture device (e.g., a camera), a video archive containing previously captured video (e.g., video files stored in a storage device), or a video supply interface for receiving video from a video content provider (e.g., a video broadcast transceiver).

[0022]

[0048] As shown in Figure 1, the video sequence 100 may include a series of pictures arranged in time along a timeline, including pictures 102, 104, 106, and 108. Pictures 102-106 are consecutive, with more pictures between picture 106 and picture 108. In Figure 1, picture 102 is an I picture, and its reference picture is picture 102 itself. Picture 104 is a P picture, and as indicated by the arrow, its reference picture is picture 102. Picture 106 is a B picture, and as indicated by the arrow, its reference pictures are pictures 104 and 108. In some embodiments, the reference picture of a picture (e.g., picture 104) may not be immediately before or after that picture. For example, the reference picture of picture 104 may be a picture preceding picture 102. Please note that the reference pictures 102-106 are merely examples, and this disclosure is not limited to the examples shown in Figure 1 of the reference pictures.

[0023]

[0049] Typically, a video codec does not encode or decode an entire picture at once because the computation of such a task is complex. More precisely, a video codec can divide a picture into basic segments and encode or decode the picture segment by segment. Such basic segments are referred to in this disclosure as Basic Processing Units ("BPUs"). For example, structure 110 in Figure 1 shows an example of the structure of a picture (e.g., any of pictures 102-108) in video sequence 100. In structure 110, the picture is divided into 4x4 Basic Processing Units, the boundaries of which are indicated by dashed lines. In some embodiments, Basic Processing Units may be called "macroblocks" in some video encoding standards (e.g., the MPEG family, H.261, H.263, or H.264 / AVC) or "Coding Tree Units" ("CTUs") in some other video encoding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing unit can have pixels of variable size, such as 128x128, 64x64, 32x32, 16x16, 4x8, 16x32, or any arbitrary shape and size within a picture. The size and shape of the basic processing unit can be selected for each picture based on the balance between coding efficiency and level of detail maintained within the basic processing unit.

[0024]

[0050] A basic processing unit can be a logical unit that may contain various types of video data stored in computer memory (e.g., in a video frame buffer). For example, a basic processing unit of a color picture may include a luminance component (Y) representing achromatic luminance information, one or more chroma components (e.g., Cb and Cr) representing color information, and related syntax elements in which the lumina and chroma components may have basic processing units of the same size. In some video coding standards (e.g., H.265 / HEVC or H.266 / VVC), the lumina and chroma components may be called a "coding tree block" (CTB). Any operation performed on a basic processing unit can be repeated on its lumina and chroma components, respectively.

[0025]

[0051] Video encoding involves multiple operational stages, examples of which are shown in Figures 2A-2B and 3A-3B. At each stage, the size of the basic processing unit may still be too large to process; therefore, it can be further divided into segments referred to in this disclosure as “basic processing subunits.” In some embodiments, a basic processing subunit may be referred to as a “block” in some video encoding standards (e.g., the MPEG family, H.261, H.263, or H.264 / AVC) or as a “coding unit” ("CU") in some other video encoding standards (e.g., H.265 / HEVC or H.266 / VVC). A basic processing subunit may be the same size as or smaller than a basic processing unit. Similar to a basic processing unit, a basic processing subunit is also a logical unit and may contain a collection of various types of video data (e.g., Y, Cb, Cr, and associated syntax elements) stored in computer memory (e.g., in a video frame buffer). Any operation performed on a basic processing subunit can be repeated on its luma and chroma components, respectively. Note that such divisions can be performed at further levels as processing needs require. Also note that various stages can divide the basic processing unit using various methods.

[0026]

[0052] For example, in the mode determination stage (an example of which is shown in Figure 2B), the encoder can determine which prediction mode (e.g., intra-picture prediction or inter-picture prediction) to use for a basic processing unit, but the basic processing unit may be too large to make such a determination. The encoder can divide the basic processing unit into multiple basic processing subunits (e.g., CUs as in H.265 / HEVC or H.266 / VVC) and determine the type of prediction for each individual basic processing subunit.

[0027]

[0053] In another example, during the prediction phase (an example of which is shown in Figures 2A and 2B), the encoder can perform prediction operations at the level of the basic processing subunit (e.g., CU). However, in some cases, the basic processing subunit may still be too large to process. The encoder can further divide the basic processing subunit into smaller segments (e.g., called "prediction blocks" or "PBs" in H.265 / HEVC or H.266 / VVC) and perform prediction operations at those levels.

[0028]

[0054] In another example, during the transformation phase (an example of which is shown in Figures 2A and 2B), the encoder can perform transformation operations on residual subunits (e.g., CUs). However, in some cases, the subunit may still be too large to process. The encoder can further divide the subunit into smaller segments (e.g., called "transformation blocks" or "TBs" in H.265 / HEVC or H.266 / VVC) and perform transformation operations at that level. Note that the division method for the same subunit may differ between the prediction phase and the transformation phase. For example, in H.265 / HEVC or H.266 / VVC, the prediction blocks and transformation blocks of the same CU may have different sizes and numbers.

[0029]

[0055] In the structure 110 of Figure 1, the basic processing unit 112 is further divided into 3x3 basic processing subunits, with the boundaries indicated by dotted lines. Different basic processing units of the same picture can be divided into basic processing subunits in different ways.

[0030]

[0056] In some implementations, a picture can be divided into processing regions to provide parallel processing and error tolerance capabilities for video encoding and decoding. This allows the encoding or decoding process to be independent of the information in any other region of the picture. In other words, each region of the picture can be processed independently. This allows the codec to process different regions of the picture in parallel, increasing encoding efficiency. Furthermore, if the data in a region is corrupted during processing or lost during network transmission, the codec can correctly encode or decode other regions of the same picture without relying on the corrupted or lost data, thus providing error tolerance. Some video encoding standards allow a picture to be divided into various types of regions. For example, H.265 / HEVC and H.266 / VVC offer two types of regions: "slices" and "tiles." It should also be noted that the various pictures in video sequence 100 may have various partitioning schemes for dividing the picture into regions.

[0031]

[0057] For example, in Figure 1, structure 110 is divided into three regions 114, 116, and 118, with their boundaries indicated by solid lines within structure 110. Region 114 contains four basic processing units. Regions 116 and 118 each contain six basic processing units. Note that the basic processing units, basic sub-units, and regions of structure 110 in Figure 1 are merely examples, and this disclosure does not limit its embodiments.

[0032]

[0058] Figure 2A shows a schematic diagram of an example of an encoding process 200A consistent with embodiments of the present disclosure. For example, the encoding process 200A can be performed by an encoder. As shown in Figure 2A, the encoder can encode a video sequence 202 into a video bitstream 228 according to process 200A. Similar to the video sequence 100 in Figure 1, the video sequence 202 may contain a set of pictures arranged in chronological order (referred to as “original pictures”). Similar to the structure 110 in Figure 1, each original picture in the video sequence 202 can be divided by the encoder into a basic processing unit, a basic processing subunit, or a region for processing. In some embodiments, the encoder can perform process 200A at the level of a basic processing unit for each original picture in the video sequence 202. For example, the encoder can perform process 200A iteratively, in which case the encoder can encode a basic processing unit in a single iteration of process 200A. In some embodiments, the encoder can execute process 200A in parallel for each region of the original picture in the video sequence 202 (e.g., regions 114-118).

[0033]

[0059] In Figure 2A, the encoder can supply the basic processing unit of the original picture of the video sequence 202 (called the "original BPU") to the prediction stage 204 to generate the predicted data 206 and the predicted BPU 208. The encoder can subtract the predicted BPU 208 from the original BPU to generate the residual BPU 210. The encoder can supply the residual BPU 210 to the conversion stage 212 and the quantization stage 214 to generate the quantized conversion coefficients 216. The encoder can supply the predicted data 206 and the quantized conversion coefficients 216 to the binary encoding stage 226 to generate the video bitstream 228. Components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 can be called the "forward path". During process 200A, after the quantization stage 214, the encoder can supply the quantized conversion coefficients 216 to the inverse quantization stage 218 and the inverse conversion stage 220 to generate a reconstructed residual BPU 222. The encoder can add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction criterion 224, which is used in the prediction stage 204 for the next iteration of process 200A. Components 218, 220, 222, and 224 of process 200A can be called a “reconstruction path”. The reconstruction path can be used to ensure that both the encoder and the decoder use the same reference data for prediction.

[0034]

[0060] The encoder can iterate through process 200A to encode each original BPU of the original picture (in the forward path) and generate a prediction criterion 224 for encoding the next original BPU of the original picture (in the reconstruction path). After encoding all of the original BPUs of the original picture, the encoder can proceed to encode the next picture in the video sequence 202.

[0035]

[0061] Referring to process 200A, the encoder may receive a video sequence 202 generated by a video acquisition device (e.g., a camera). As used herein, the term “receive” can mean any act of receiving, inputting, acquiring, retrieving, obtaining, reading, accessing, or inputting data.

[0036]

[0062] In prediction stage 204, in the current iteration, the encoder receives the original BPU and prediction criterion 224 and can perform a prediction operation to generate prediction data 206 and the predicted BPU 208. The prediction criterion 224 can be generated from the reconstruction path of the previous iteration of process 200A. The purpose of prediction stage 204 is to reduce information redundancy by extracting prediction data 206, which can then be used to reconstruct the original BPU by extracting it as the predicted BPU 208 from the prediction data 206 and prediction criterion 224.

[0037]

[0063] Ideally, the predicted BPU208 should be identical to the original BPU. However, due to imperfections in the prediction and reconstruction operations, the predicted BPU208 is generally slightly different from the original BPU. To record such differences, after generating the predicted BPU208, the encoder can subtract it from the original BPU to generate the residual BPU210. For example, the encoder can subtract the pixel values ​​(e.g., grayscale or RGB values) of the predicted BPU208 from the corresponding pixel values ​​of the original BPU. Each pixel of the residual BPU210 may have a residual value resulting from such a subtraction between the corresponding pixel of the original BPU and the predicted BPU208. Compared to the original BPU, the predicted data 206 and residual BPU210 may have fewer bits, but they can be used to reconstruct the original BPU without significant quality degradation. In this way, the original BPU is compressed.

[0038]

[0064] To further compress the residual BPU 210, in the transformation step 212, the encoder can reduce the spatial redundancy of the residual BPU 210 by decomposing it into a set of two-dimensional "basis patterns," each basis pattern being associated with "transformation coefficients." The basis patterns can have the same size (e.g., the size of the residual BPU 210). Each basis pattern can represent a variation frequency component (e.g., luminance variation frequency) of the residual BPU 210. No basis pattern can be reproduced from any combination of any other basis patterns (e.g., a linear combination). In other words, the decomposition can decompose the variation of the residual BPU 210 into the frequency domain. Such a decomposition is analogous to the discrete Fourier transform of a function, where the basis patterns are analogous to the basis functions of the discrete Fourier transform (e.g., trigonometric functions), and the transformation coefficients are analogous to the coefficients associated with the basis functions.

[0039]

[0065] Different transformation algorithms may use different basis patterns. Various transformation algorithms can be used in transformation stage 212, such as discrete cosine transform and discrete sine transform. The transformation in transformation stage 212 can be reversed; that is, the encoder can reconstruct the residual BPU 210 by the reverse operation of the transformation (called the "inverse transform"). For example, to reconstruct the pixels of the residual BPU 210, the inverse transform could involve multiplying the values ​​of the corresponding pixels in the basis pattern by the respective coefficients and adding the products to produce a weighted sum. In video coding standards, both the encoder and decoder can use the same transformation algorithm (and therefore the same basis pattern). Therefore, the encoder can record only the transformation coefficients, and the decoder can reconstruct the residual BPU 210 from the transformation coefficients without receiving the basis pattern from the encoder. Compared to the residual BPU 210, the transformation coefficients can have fewer bits and can be used to reconstruct the residual BPU 210 without significant quality degradation. In this way, the residual BPU210 is further compressed.

[0040]

[0066] The encoder can further compress the conversion coefficients in the quantization stage 214. In the conversion process, various basis patterns can represent various fluctuation frequencies (e.g., luminance fluctuation frequencies). Since the human eye generally perceives lower frequency fluctuations better, the encoder can ignore information from higher frequency fluctuations without causing significant quality degradation during decoding. For example, in the quantization stage 214, the encoder can generate quantized conversion coefficients 216 by dividing each conversion coefficient by an integer value (called the "quantization scale factor") and rounding the quotient to its nearest integer. After such an operation, some conversion coefficients of the high-frequency basis patterns can be converted to zero, and the conversion coefficients of the low-frequency basis patterns can be converted to smaller integers. The encoder can ignore the zero-value quantized conversion coefficients 216, thereby further compressing the conversion coefficients. The quantization process can also be reversed, in which case the quantized conversion coefficients 216 can be reconstructed into conversion coefficients by the inverse operation of quantization (called "inverse quantization").

[0041]

[0067] Because the encoder ignores the remainder of such division in rounding operations, the quantization stage 214 can be irreversible. Typically, the quantization stage 214 can be the source of the greatest information loss in process 200A. The greater the information loss, the fewer bits the quantized conversion coefficients 216 may require. To obtain varying levels of information loss, the encoder can use quantization parameters of varying values ​​or any other parameters of the quantization process.

[0042]

[0068] In the binary encoding stage 226, the encoder may encode the predicted data 206 and the quantized conversion coefficients 216 using a binary encoding technique such as entropy encoding, variable-length encoding, arithmetic encoding, Huffman encoding, context-adaptive binary arithmetic encoding, or any other reversible or irreversible compression algorithm. In some embodiments, in addition to the predicted data 206 and the quantized conversion coefficients 216, the encoder may encode other information in the binary encoding stage 226, such as the prediction mode used in the prediction stage 204, the parameters of the prediction operation, the type of transformation in the transformation stage 212, the parameters of the quantization process (e.g., quantization parameters), the encoder control parameters (e.g., bitrate control parameters), and so on. The encoder may use the output data from the binary encoding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 may be further packetized for network transmission.

[0043]

[0069] Referring to the reconstruction path of process 200A, in the inverse quantization stage 218, the encoder can perform inverse quantization on the quantized transformation coefficients 216 to generate reconstructed transformation coefficients. In the inverse transformation stage 220, the encoder can generate reconstructed residual BPU 222 based on the reconstructed transformation coefficients. The encoder can add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction criterion 224 to be used in the next iteration of process 200A.

[0044]

[0070] It should be noted that the video sequence 202 can be encoded using other variations of process 200A. In some embodiments, the steps of process 200A can be performed in a different order by the encoder. In some embodiments, one or more steps of process 200A can be combined into a single step. In some embodiments, a single step of process 200A can be divided into multiple steps. For example, the conversion step 212 and the quantization step 214 can be combined into a single step. In some embodiments, process 200A can include additional steps. In some embodiments, process 200A can omit one or more steps of Figure 2A.

[0045]

[0071] Figure 2B shows a schematic diagram of another example of encoding process 200B that conforms to embodiments of the present disclosure. Process 200B can be modified from process 200A. For example, process 200B can be used with an encoder conforming to a hybrid video encoding standard (e.g., the H.26x series). Compared to process 200A, the forward path of process 200B additionally includes a mode determination stage 230 and divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044. The reconstruction path of process 200B additionally includes a loop filter stage 232 and a buffer 234.

[0046]

[0072] Generally, prediction techniques can be classified into two types: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-picture prediction or "intra-prediction") can predict the current BPU using pixels from one or more adjacent BPUs that have already been coded within the same picture. That is, the prediction criterion 224 in spatial prediction can include adjacent BPUs. Spatial prediction can reduce the spatial redundancy inherent in a picture. Temporal prediction (e.g., inter-picture prediction or "inter-prediction") can predict the current BPU using regions from one or more already coded pictures. That is, the prediction criterion 224 in temporal prediction can include coded pictures. Temporal prediction can reduce the temporal redundancy inherent in a picture.

[0047]

[0073] Referring to process 200B, in the forward path, the encoder performs prediction operations in spatial prediction stage 2042 and temporal prediction stage 2044. For example, in spatial prediction stage 2042, the encoder may perform intra-prediction. In the original BPU of the picture being encoded, the prediction criterion 224 may include one or more adjacent BPUs within the same picture that are encoded (in the forward path) and reconstructed (in the reconstructed path). The encoder may generate a predicted BPU 208 by extrapolating adjacent BPUs. Extrapolation techniques may include, for example, linear extrapolation or linear interpolation, polynomial extrapolation or polynomial interpolation, etc. In some embodiments, the encoder may perform extrapolation at the pixel level, for example, by extrapolating the values ​​of the corresponding pixels for each pixel of the predicted BPU 208. The adjacent BPUs used for extrapolation may be located relative to the original BPU from various directions, such as vertically (e.g., above the original BPU), horizontally (e.g., to the left of the original BPU), diagonally (e.g., to the lower left, lower right, upper left, or upper right of the original BPU), or in any direction defined by the video encoding standard used. In intra-prediction, the prediction data 206 may include, for example, the location (e.g., coordinates) of the adjacent BPUs used, the size of the adjacent BPUs used, the extrapolation parameters, and the orientation of the adjacent BPUs used relative to the original BPU.

[0048]

[0074] In another example, during the temporal prediction stage 2044, the encoder can perform interpretation. In the original BPU of the current picture, the prediction criterion 224 may include one or more pictures (referred to as "reference pictures") that have been encoded (in the forward path) and reconstructed (in the reconstructed path). In some embodiments, the reference pictures can be encoded and reconstructed for each BPU. For example, the encoder can generate a reconstructed BPU by adding the reconstructed residual BPU 222 to the predicted BPU 208. Once all the reconstructed BPUs for the same picture have been generated, the encoder can generate a reconstructed picture as a reference picture. The encoder can perform a "motion estimation" operation to search for a matching region within a range (referred to as a "search window") in the reference picture. The location of the search window in the reference picture can be determined based on the location of the original BPU in the current picture. For example, the search window can be centered in the reference picture at a location with the same coordinates as the original BPU in the current picture and can be extended over a predetermined distance. When the encoder identifies a region similar to the original BPU within the search window (for example, by using a pixel recursive algorithm, a block matching algorithm, etc.), the encoder can determine such a region as a match region. The match region may have different dimensions from the original BPU (e.g., smaller, equal to, larger, or a different shape). Since the reference picture and the current picture are separated in time on the timeline (for example, as shown in Figure 1), the match region can be considered to "move" to the original BPU's position over time. The encoder can record the direction and distance of such movement as a "motion vector". When multiple reference pictures are used (for example, as picture 106 in Figure 1), the encoder can search for a match region for each reference picture and determine its associated motion vector. In some embodiments, the encoder can assign weights to the pixel values ​​of the match region for each matching reference picture.

[0049]

[0075] Motion estimation can be used to identify various types of motion, such as translation, rotation, and scaling. In interpretation, the prediction data 206 may include, for example, the location of the matched region (e.g., coordinates), the motion vector associated with the matched region, the number of reference pictures, the weights associated with the reference pictures, and so on.

[0050]

[0076] To generate the predicted BPU 208, the encoder can perform a “motion compensation” operation. Using motion compensation, the predicted BPU 208 can be reconstructed based on the prediction data 206 (e.g., motion vectors) and the prediction criterion 224. For example, the encoder can move the matching region of a reference picture according to the motion vector, in which case the encoder can predict the original BPU of the current picture. When multiple reference pictures are used (e.g., picture 106 in Figure 1), the encoder can move the matching region of each reference picture according to its respective motion vector and average the pixel values ​​of the matching region. In some embodiments, if the encoder assigns weights to the pixel values ​​of the matching region of each matching reference picture, the encoder can add up the weighted sum of the pixel values ​​of the moved matching regions.

[0051]

[0077] In some embodiments, interpretation can be unidirectional or bidirectional. Unidirectional interpretation can use one or more reference pictures that are in the same temporal direction relative to the current picture. For example, picture 104 in Figure 1 is a unidirectional interpretation-predicted picture, with a reference picture (e.g., picture 102) preceding picture 104. Bidirectional interpretation can use one or more reference pictures that are in both temporal directions relative to the current picture. For example, picture 106 in Figure 1 is a bidirectional interpretation-predicted picture, with reference pictures (e.g., pictures 104 and 108) in both temporal directions relative to picture 104.

[0052]

[0078] Continuing to refer to the forward path of process 200B, after the spatial prediction 2042 and the temporal prediction stage 2044, in the mode determination stage 230, the encoder may select a prediction mode (e.g., one of intra-prediction or inter-prediction) for the current iteration of process 200B. For example, the encoder may perform a rate distortion optimization technique, in which case the encoder may select a prediction mode to minimize the value of the cost function, depending on the bit rate of the candidate prediction mode and the distortion of the reconstructed reference picture under the candidate prediction mode. Depending on the selected prediction mode, the encoder may generate the corresponding predicted BPU 208 and predicted data 206.

[0053]

[0079] In the reconstruction path of process 200B, if intra-prediction mode is selected in the forward path, after generating the prediction criterion 224 (e.g., the current BPU encoded and reconstructed within the current picture), the encoder can directly feed the prediction criterion 224 to the spatial prediction stage 2042 for later use (e.g., to extrapolate the next BPU of the current picture). The encoder can also feed the prediction criterion 224 to the loop filtering stage 232, where the encoder can apply loop filters to the prediction criterion 224 to reduce or remove distortions (e.g., blocking artifacts) caused during the encoding of the prediction criterion 224. In the loop filtering stage 232, the encoder can apply various loop filtering techniques, such as deblocking, sample-adaptive offset, adaptive loop filtering, etc. The loop-filtered reference picture can be stored in the buffer 234 (or "decoded picture buffer") for later use (e.g., to use as an inter-predictive reference picture for subsequent pictures in the video sequence 202). The encoder can store one or more reference pictures in the buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the encoder can encode the loop filter parameters (e.g., the strength of the loop filter) in the binary encoding stage 226, along with the quantized transformation coefficients 216, the prediction data 206, and other information.

[0054]

[0080] Figure 3A shows a schematic diagram of an example of a decoding process 300A consistent with embodiments of the present disclosure. Process 300A can be a decompression process corresponding to the compression process 200A in Figure 2A. In some embodiments, process 300A can be similar to the reconstruction path of process 200A. The decoder can decode the video bitstream 228 into a video stream 304 according to process 300A. The video stream 304 can be very similar to the video sequence 202. However, due to information loss in the compression and decompression processes (e.g., the quantization stage 214 in Figures 2A-2B), the video stream 304 is generally not identical to the video sequence 202. Similar to processes 200A and 200B in Figures 2A-2B, the decoder can execute process 300A at the level of a basic processing unit (BPU) for each picture encoded in the video bitstream 228. For example, the decoder can iterate through process 300A, in which case the decoder can decode a basic processing unit in a single iteration of process 300A. In some embodiments, the decoder can execute process 300A in parallel for each region of picture (e.g., regions 114-118) to be encoded within the video bitstream 228.

[0055]

[0081] In Figure 3A, the decoder can supply a portion of the video bitstream 228 associated with the basic processing unit of the encoded picture (called the "encoded BPU") to the binary decoding stage 302. In the binary decoding stage 302, the decoder can decode the portion into prediction data 206 and quantized conversion coefficients 216. The decoder can supply the quantized conversion coefficients 216 to the inverse quantization stage 218 and the inverse transformation stage 220 to generate the reconstructed residual BPU 222. The decoder can supply the prediction data 206 to the prediction stage 204 to generate the predicted BPU 208. The decoder can add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction criterion 224. In some embodiments, the prediction criterion 224 can be stored in a buffer (e.g., a decoded picture buffer in computer memory). The decoder can supply the prediction criterion 224 to the prediction stage 204 for performing the prediction operation in the next iteration of process 300A.

[0056]

[0082] The decoder can iteratively perform process 300A to decode each encoded BPU of the encoded picture and generate a predictive criterion 224 for encoding the next encoded BPU of the encoded picture. After decoding all encoded BPUs of the encoded picture, the decoder can output the picture to the video stream 304 for display and proceed to decode the next encoded picture in the video bitstream 228.

[0057]

[0083] In the binary decoding stage 302, the decoder can perform the inverse operation of the binary encoding technique used by the encoder (e.g., entropy encoding, variable-length encoding, arithmetic encoding, Huffman encoding, context-adaptive binary arithmetic encoding, or any other lossless compression algorithm). In some embodiments, in addition to the predicted data 206 and the quantized conversion coefficients 216, the decoder can decode other information in the binary decoding stage 302, such as the prediction mode, parameters of the prediction operation, type of transformation, parameters of the quantization process (e.g., quantization parameters), encoder control parameters (e.g., bitrate control parameters), and so on. In some embodiments, if the video bitstream 228 is transmitted in packets over the network, the decoder can depacketize the video bitstream 228 before feeding it to the binary decoding stage 302.

[0058]

[0084] Figure 3B shows a schematic diagram of another example of a decoding process 300B that conforms to embodiments of the present disclosure. Process 300B can be modified from process 300A. For example, process 300B can be used with a decoder compliant with a hybrid video coding standard (e.g., the H.26x series). Compared to process 300A, process 300B further divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and additionally includes a loop filter stage 232 and a buffer 234.

[0059]

[0085] In process 300B, the encoded base processing unit ("current BPU") of the encoded picture ("current picture") being decoded can contain various types of data depending on which prediction mode was used by the encoder to encode the current BPU. For example, if intra-prediction was used by the encoder to encode the current BPU, the prediction data 206 can contain prediction mode indicators (e.g., flag values) indicating the intra-prediction, parameters of the intra-prediction operation, etc. Parameters of the intra-prediction operation can include, for example, the location (e.g., coordinates) of one or more adjacent BPUs used as a reference, the size of the adjacent BPUs, extrapolation parameters, the orientation of the adjacent BPU relative to the original BPU, etc. In another example, if inter-prediction was used by the encoder to encode the current BPU, the prediction data 206 can contain prediction mode indicators (e.g., flag values) indicating the inter-prediction, parameters of the inter-prediction operation, etc. The parameters for the interpretation operation may include, for example, the number of reference pictures associated with the current BPU, the weights associated with each reference picture, the locations (e.g., coordinates) of one or more matching regions within each reference picture, and one or more motion vectors associated with each matching region.

[0060]

[0086] Based on the prediction mode index, the decoder can determine whether to perform a spatial prediction (e.g., intra-prediction) in the spatial prediction stage 2042 or a temporal prediction (e.g., inter-prediction) in the temporal prediction stage 2044. Details of performing such spatial or temporal predictions are explained in Figure 2B and will not be repeated below. After performing such spatial or temporal predictions, the decoder can generate the predicted BPU 208. As explained in Figure 3A, the decoder can generate the prediction criterion 224 by adding the predicted BPU 208 and the reconstructed residual BPU 222.

[0061]

[0087] In process 300B, the decoder can supply the prediction criterion 224 to the spatial prediction stage 2042 or the temporal prediction stage 2044 for performing the prediction operation in the next iteration of process 300B. For example, if the current BPU is decoded using intra-prediction in the spatial prediction stage 2042, after generating the prediction criterion 224 (e.g., the decoded current BPU), the decoder can supply the prediction criterion 224 directly to the spatial prediction stage 2042 for later use (e.g., to extrapolate the next BPU of the current picture). If the current BPU is decoded using inter-prediction in the temporal prediction stage 2044, after generating the prediction criterion 224 (e.g., the reference picture from which all BPUs have been decoded), the decoder can supply the prediction criterion 224 to the loop filter stage 232 to reduce or remove distortion (e.g., blocking artifacts). The decoder can apply the loop filter to the prediction criterion 224 in the manner described in Figure 2B. The loop-filtered reference picture can be stored in buffer 234 (e.g., a decoded picture buffer in computer memory) for later use (e.g., to be used as an inter-prediction reference picture for subsequent encoded pictures of the video bitstream 228). The decoder can store one or more reference pictures in buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the prediction data may further include loop filter parameters (e.g., loop filter strength). In some embodiments, the prediction data includes loop filter parameters when the prediction mode index of the prediction data 206 indicates that inter-prediction was used to encode the current BPU.

[0062]

[0088] Figure 4 is a block diagram of an example of an apparatus 400 for encoding or decoding video, consistent with embodiments of the present disclosure. As shown in Figure 4, the apparatus 400 may include a processor 402. When the processor 402 executes instructions described herein, the apparatus 400 can become a dedicated device for video encoding or decoding. The processor 402 can be any type of circuitry capable of handling or processing information. For example, the processor 402 may include any number of central processing units (i.e., "CPUs"), graphics processing units (i.e., "GPUs"), neural processing units ("NPUs"), microcontroller units ("MCUs"), optical processors, programmable logic controllers, microcontrollers, microprocessors, digital signal processors, IP (Intellectual Property) cores, programmable logic arrays (PLAs), programmable array logic (PALs), general-purpose array logic (GALs), composite programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), systems on a chip (SoCs), application-specific integrated circuits (ASICs), and any combination thereof. In some embodiments, the processor 402 may also be a set of processors grouped together as a single logical component. For example, as shown in Figure 4, the processor 402 may include a plurality of processors, including processor 402a, processor 402b, and processor 402n.

[0063]

[0089] The device 400 may also include a memory 404 configured to store data (e.g., a set of instructions, computer code, intermediate data, etc.). For example, as shown in Figure 4, the stored data may include program instructions (e.g., program instructions for implementing stages in processes 200A, 200B, 300A, or 300B) and processing data (e.g., video sequence 202, video bitstream 228, or video stream 304). The processor 402 can access the program instructions and processing data (e.g., via the bus 410), execute the program instructions, and perform operations or handling on the processing data. The memory 404 may include a high-speed random-access storage device or a non-volatile storage device. In some embodiments, the memory 404 may include any number of random-access memories (RAM), read-only memories (ROM), optical disks, magnetic disks, hard drives, solid-state drives, flash drives, security digital (SD) cards, memory sticks, CompactFlash® (CF) cards, and any combination thereof. Memory 404 can also be a group of memories (not shown in Figure 4) grouped together as a single logical component.

[0064]

[0090] Bus 410 can be a communication device that transfers data between components within the device 400, such as an internal bus (e.g., a CPU memory bus) or an external bus (e.g., a USB (Universal Serial Bus) port, a PCI (Peripheral Component Interconnect) express port).

[0065]

[0091] To ensure clarity and avoid ambiguity, the processor 402 and other data processing circuits are collectively referred to as “data processing circuits” in this disclosure. The data processing circuits can be implemented entirely in hardware, or as a combination of software, hardware, or firmware. In addition, the data processing circuits can be a single, independent module, or they can be fully or partially integrated into any other component of the device 400.

[0066]

[0092] The device 400 may further include a network interface 406 to provide wired or wireless communication with a network (e.g., the Internet, an intranet, a local area network, a mobile communication network, etc.). In some embodiments, the network interface 406 may include any number of network interface controllers (NICs), radio frequency (RF) modules, transponders, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth® adapters, infrared adapters, near-field communication ("NFC") adapters, cellular network chips, and any combination thereof.

[0067]

[0093] In some embodiments, the apparatus 400 may optionally further include a peripheral device interface 408 to provide connectivity to one or more peripheral devices. As shown in Figure 4, peripheral devices may include, but are not limited to, cursor control devices (e.g., mouse, touchpad, or touchscreen), keyboards, displays (e.g., cathode ray tube displays, liquid crystal displays, or light-emitting diode displays), video input devices (e.g., cameras or input interfaces coupled to video archives), and the like.

[0068]

[0094] It should be noted that the video codec (for example, the codec that runs processes 200A, 200B, 300A, or 300B) can be implemented as any combination of any software or hardware modules within the device 400. For example, some or all stages of processes 200A, 200B, 300A, or 300B can be implemented as one or more software modules of the device 400, such as program instructions that can be loaded into memory 404. In another example, some or all stages of processes 200A, 200B, 300A, or 300B can be implemented as one or more hardware modules of the device 400, such as dedicated data processing circuits (e.g., FPGA, ASIC, NPU, etc.).

[0069]

[0095] Figure 5 illustrates a CU coded in palette mode. In VVC Draft 8, palette mode can be used with monochromatic, 4:2:0, 4:2:2, and 4:4:4 color formats. When palette mode is enabled, a flag is sent at the CU level if the CU size is 64x64 or less and there are more than 16 samples indicating whether palette mode is being used. When palette mode is used to code (current) CU500, the sample value at each position in the CU is represented by a small set of representative color values. This set is called palette 510. For sample positions with values ​​that approximate palette colors 501, 502, and 503, the corresponding palette index is signaled. It is also possible to specify color values ​​outside the palette by signaling escape index 504. Next, for all positions in the CU that use the escape color index, the (quantized) color component values ​​are signaled for each of these positions.

[0070]

[0096] For palette coding, a palette predictor is maintained. Figure 6 shows an exemplary process for updating the palette predictor after encoding each coding unit 600. The predictor is initialized to 0 (i.e., empty) at the beginning of each slice in the case of non-wavefronts, and at the beginning of each CTU row in the case of wavefronts. For each entry in the palette predictor, a reuse flag is signaled to indicate whether it should be included in the current palette of the current CU. The reuse flag is sent using zero run-length coding. After this, the number of new palette entries and the component values ​​of the new palette entries are signaled. After encoding the palette-coded CU, the palette predictor is to be updated with the current palette, and entries from previous palette predictors that are not reused in the current palette are appended to the end of the new palette predictor until the maximum allowed size is reached.

[0071]

[0097] An escape flag is signaled for each CU to indicate whether an escape symbol exists within the current CU. If an escape symbol exists, the palette table is expanded by the last index assigned to become the escape symbol. As shown in the example in Figure 5, the palette indices of the samples within a CU form a palette index map. The index map is coded using horizontal or vertical cross-sectional scans. The scan order is explicitly signaled in the bitstream using palette_transpose_flag. The palette index map is coded using index execution mode or index copy mode.

[0072]

[0098] In VVC Draft 8, the deblocking filter process includes defining block boundaries, deriving boundary filter strengths based on the coding modes of two adjacent blocks along the defined block boundary, deriving the number of samples to be filtered, and applying the deblocking filter to the samples. An edge is defined as a block boundary when it is the boundary of a coding unit, coding subblock unit, or transformation unit. Next, the boundary filter strength is calculated based on the coding modes of two adjacent blocks according to the following six rules: (1) If both coding blocks are coded in BDPCM mode, the boundary filter strength is set to 0. (2) Otherwise, if one of the coding blocks is coded in intra mode, the boundary filter strength is set to 2. (3) Otherwise, if one of the coding blocks is coded in CIIP mode, the boundary filter strength is set to 2. (4) Otherwise, if one of the coding blocks contains one or more non-zero coefficient levels, the boundary filter strength is set to 1. (5) Otherwise, if one of the blocks is coded in IBC mode and the other is coded in inter-mode, the boundary filter strength is set to 1. (6) Otherwise (both of the two blocks are coded in either IBC mode or inter-mode), the boundary filter strength is derived using the reference pictures and motion vectors of the two blocks.

[0073]

[0099] VVC Draft 8 provides a more detailed description of the process for calculating boundary filter strength. Specifically, VVC Draft 8 provides eight sequential, exhaustive scenarios. In Scenario 1, if cIdx is equal to 0 and both samples p0 and q0 are in a coding block where intra_bdpcm_luma_flag is equal to 1, then bS[xDi][yDj] is set to equal to 0. Otherwise, in Scenario 2, if cIdx is greater than 0 and both samples p0 and q0 are in a coding block where intra_bdpcm_chroma_flag is equal to 1, then bS[xDi][yDj] is set to equal to 0. Otherwise, in Scenario 3, if either sample p0 or q0 is in a coding block of a coding unit coded using intra-predictive mode, then bS[xDi][yDj] is set to equal to 2. Otherwise, in Scenario 4, if the block edge is also an edge of a coded block and sample p0 or q0 is in a coded block where ciip_flag is equal to 1, then bS[xDi][yDj] is set to equal to 2. Otherwise, in Scenario 5, if the block edge is also an edge of a transform block and sample p0 or q0 is in a transform block that contains one or more non-zero transform coefficient levels, then bS[xDi][yDj] is set to equal to 1. Otherwise, in Scenario 6, if the prediction mode of the coded subblock containing sample p0 is different from the prediction mode of the coded subblock containing sample q0 (i.e., one of the coded subblocks is coded in IBC prediction mode and the other is coded in inter-prediction mode), then bS[xDi][yDj] is set to equal to 1.

[0074]

[0100] Otherwise, in Scenario 7, if cIdx is equal to 0, edgeFlags[xDi][yDj] is equal to 2, and one or more of the following conditions are true, then bS[xDi][yDj] is set to equal to 1.

[0075]

[0101] Condition (1): Both the coded subblock containing sample p0 and the coded subblock containing sample q0 are coded in IBC prediction mode, and the absolute difference between the horizontal or vertical components of the block vectors used for prediction of the two coded subblocks is 8 or greater in 1 / 16 luma sample units.

[0076]

[0102] Condition (2): A different reference picture or a different number of motion vectors is used for the prediction of the coded subblock containing sample p0 than for the prediction of the coded subblock containing sample q0. Note that for condition (2), the determination of whether the reference pictures used for the two coded subblocks are the same or different is based solely on which picture is referenced, regardless of whether the prediction is formed using an index to reference picture list 0 or reference picture list 1, and regardless of whether the index positions within the reference picture lists are different. Also note that for condition (2), the number of motion vectors used for the prediction of the coded subblock where the top-left sample covers (xSb, ySb) is equal to PredFlagL0[xSb][ySb] + PredFlagL1[xSb][ySb].

[0077]

[0103] Condition (3): One motion vector is used to predict the coded subblock containing sample p0, and another motion vector is used to predict the coded subblock containing sample q0, and the absolute difference between the horizontal or vertical components of the motion vectors used is 8 or greater in 1 / 16 luma sample units.

[0078]

[0104] Condition (4): Two motion vectors and two different reference pictures are used to predict the coded subblock containing sample p0, two motion vectors from the same two reference pictures are used to predict the coded subblock containing sample q0, and the absolute difference between the horizontal or vertical components of the two motion vectors used to predict the coded subblocks from the same reference picture is 8 or greater in 1 / 16 luma sample units.

[0079]

[0105] Condition (5): Two motion vectors from the same reference picture are used to predict the coded subblock containing sample p0, and two motion vectors from the same reference picture are used to predict the coded subblock containing sample q0, and both of the following conditions are true: Condition (5.1): The absolute difference between the horizontal or vertical components of the motion vectors in List 0 used to predict the two coded subblocks is 8 or greater in 1 / 16 luma sample units, or the absolute difference between the horizontal or vertical components of the motion vectors in List 1 used to predict the two coded subblocks is 8 or greater in 1 / 16 luma sample units. Condition (5.2): The absolute difference between the horizontal or vertical components of the motion vector of List 0 used in the prediction of the coded subblock containing sample p0 and the motion vector of List 1 used in the prediction of the coded subblock containing sample q0 is 8 or greater in 1 / 16 luma sample units, or the absolute difference between the horizontal or vertical components of the motion vector of List 1 used in the prediction of the coded subblock containing sample p0 and the motion vector of List 0 used in the prediction of the coded subblock containing sample q0 is 8 or greater in 1 / 16 luma sample units.

[0080]

[0106] Otherwise, that is, if none of the seven scenarios mentioned above are met, in scenario 8, the variable bS[xDi][yDj] is set to equal to 0. After deriving the boundary filter strength, the number of samples to be filtered is derived, and the deblocking filter is applied to the samples. Note that when a block is coded in palette mode, the number of samples is set to equal to 0. This means that the deblocking filter is not applied to blocks coded in palette mode.

[0081]

[0107] As mentioned above, the structure of the current palette of a block consists of two parts. Firstly, entries in the current palette can be predicted from the palette predictor. For each entry in the palette predictor, a reuse flag is signaled to indicate whether or not this entry is included in the current palette. Secondly, the component values ​​of the current palette entries can be directly signaled. After obtaining the current palette, the palette predictor is updated using the current palette. Figure 7 shows a portion of section 7.3.10.6 ("Palette-Coded Syntax") of VVC Draft 8. When parsing the syntax of a palette-coded block, the reuse flag (i.e., "palette_predictor_run" 701 in Figure 7) is decoded first, followed by the component values ​​of new palette entries (i.e., "num_signalled_palette_entries" 702 and "new_palette_entries" 703 in Figure 7). In VVC Draft 8, it is necessary to know the number of entries in the palette predictor (i.e., "PredictorPaletteSize[startComp]" 704 in Figure 7) before parsing the reuse flag. This means that when two adjacent blocks are both coded in palette mode, the syntax of the second block cannot be parsed until the palette predictor update process of the first block is complete.

[0082]

[0108] However, the traditional palette predictor update process requires checking the reuse flag for each entry. In a worse scenario, this could require up to 63 checks. These traditional designs can be hardware-unfriendly for at least two reasons. First, Context-based Adaptive Binary Arithmetic Coding (CABAC) parsing must wait until the palette predictor is fully updated. CABAC parsing is typically the slowest module in hardware, which can reduce the CABAC's processing power. Second, implementing the palette predictor update process during the CABAC parsing stage can be hardware-intensive.

[0083]

[0109] In addition, another challenge with these conventional designs is that boundary filter strength is not defined for palette mode. When one of adjacent blocks is coded in palette mode and the other adjacent block is coded in IBC mode or intermode, boundary filter strength is not defined.

[0084]

[0110] Embodiments of this disclosure provide implementations for addressing one or more of the above-described problems. These embodiments may improve the pallet predictor update process and enhance the efficiency, speed, and resource consumption of the pallet prediction process or a system implementing a similar process.

[0085]

[0111] In some embodiments, the pallet predictor update process is simplified to reduce its complexity, thereby freeing up hardware resources for other purposes. Figure 8 shows a flowchart of the pallet predictor update process 800 according to an embodiment of the present disclosure. Method 800 can be executed by an encoder (e.g., by process 200A in Figure 2A or 200B in Figure 2B), or by a decoder (e.g., by process 300A in Figure 3A or 300B in Figure 3B), or by one or more software or hardware components of an apparatus (e.g., apparatus 400 in Figure 4). For example, a processor (e.g., processor 402 in Figure 4) can execute Method 800. In some embodiments, Method 800 can be implemented by a computer program product embodied in a computer-readable medium, including computer-executable instructions such as program code executed by a computer (e.g., apparatus 400 in Figure 4). Referring to Figure 8, Method 800 may include the following steps 802-804.

[0086]

[0112] In step 802, when the palette predictor is updated, all palette entries from the current palette are added as a first set of entries before the new palette predictor. In step 804, all palette entries from the previous palette predictor are added to the end of the new palette predictor, regardless of whether the entries are reused in the current palette as a second set of entries, with the second set of entries following the first set of entries.

[0087]

[0113] For example, Figure 9 shows a simplified pallet predictor update process that matches the process described in Figure 8. The current pallet is generated by processes 901 and 902. The current pallet includes entries reused from the previous pallet predictor and new pallet entries that have been signaled. In process 903, all pallet entries from the current pallet are added as the first set of entries for the new pallet predictor (corresponding to step 802). In process 904, regardless of whether the entries are reused in the current pallet, pallet entries from the previous pallet predictor are added as the second set of entries for the new pallet predictor (corresponding to step 804).

[0088]

[0114] The benefit is that the size of the new pallet predictor is calculated by adding the size of the current pallet and the size of the previous pallet predictor, without checking the value of the reuse flag. The update process is much simpler than the conventional design in VVC Draft 8 (as shown in Figure 10), which could increase the processing power of CABAC.

[0089]

[0115] For example, Figure 11 shows an example of a palette mode decoding process consistent with an embodiment of the present disclosure. Changes between the conventional design in Figure 10 and the disclosed design in Figure 11, including deleted portions 1101, are highlighted in strikethrough text.

[0090]

[0116] In some embodiments, the pallet predictor update process is simplified, but because it does not check for reuse flags, a pallet predictor may have two or more identical entries (i.e., pallet predictor redundancy). Even though this is an improvement, it may mean that the predictive effectiveness of the pallet predictor may be reduced.

[0091]

[0117] Figure 12 shows a flowchart of another palette predictor update process 1200 according to an embodiment of the present disclosure. Method 1200 can be performed by an encoder (e.g., by process 200A in Figure 2A or 200B in Figure 2B), or by a decoder (e.g., by process 300A in Figure 3A or 300B in Figure 3B), or by one or more software or hardware components of a device (e.g., device 400 in Figure 4). For example, a processor (e.g., processor 402 in Figure 4) can perform Method 1200. In some embodiments, Method 1200 can be implemented by a computer program product embodied in a computer-readable medium, which includes computer-executable instructions such as program code executed by a computer (e.g., device 400 in Figure 4). Referring to Figure 12, Method 1200 may include the following steps 1202 to 1206.

[0092]

[0118] In this exemplary embodiment, to eliminate pallet predictor redundancy and simplify the pallet predictor update process, entries from the previous pallet predictor between the first reuse entry and the last reuse entry are directly discarded. Only entries from the first entry to the first reuse entry, and entries from the last reuse entry to the last entry of the previous pallet predictor are added to the new pallet predictor. In summary, the pallet predictor update process is modified as follows: In step 1202, each entry of the current pallet is added to the new pallet predictor as a first set of entries. In step 1204, each entry from the first entry to the first reuse entry of the previous pallet predictor is added to the new pallet predictor as a second set of entries, after the first set of entries. In step 1206, each entry from the last reuse entry to the last entry of the previous pallet predictor is added to the new pallet predictor as a third set of entries, after the second set of entries.

[0093]

[0119] For example, Figure 13 presents a simplified pallet predictor update process that matches the process described in Figure 12. In process 1301, all pallet entries of the current pallet are added to the pallet predictor as the first set of entries of the new pallet predictor (corresponding to step 1202). In process 1302, each entry from the first entry to the first reuse entry of the previous pallet predictor is added to the new pallet predictor as the second set of entries after the first set of entries (corresponding to step 1204). In process 1303, each entry from the last reuse entry to the last entry of the previous pallet predictor is added to the new pallet predictor as the third set of entries after the second set of entries (corresponding to step 1206). In some embodiments, the third set of entries comes after the first set of entries, and the second set of entries comes after the third set of entries.

[0094]

[0120] Note that the first reuse entry and the last entry can be derived when parsing the reuse flag. There is no need to check the value of the reuse flag. The size of the new palette predictor is calculated by adding the size of the current palette, the size from 0 to the first reuse entry, and the size from the last reuse entry to the last entry. Figure 14 shows an example of palette coding syntax that conforms to embodiments of this disclosure. Changes between the conventional design in Figure 7 and the disclosed design in Figure 14, including the added portion 1401, are enclosed in dashed lines. Figure 15 shows an example of a palette mode decoding process that conforms to embodiments of this disclosure. Changes between the conventional design in Figure 10 and the disclosed design in Figure 15, including the deleted portion 1501, are highlighted in strikethrough text, and the added portion 1502 is enclosed in dashed lines.

[0095]

[0121] In the embodiments disclosed earlier, the palette predictor update process is simplified, but this process needs to be implemented during the CABAC parsing phase of the hardware design. In some embodiments, the number of reuse flags is set to a fixed value. Therefore, CABAC can continue parsing without waiting for the palette predictor update process. Furthermore, the palette predictor update process can be implemented at a different pipeline phase other than the CABAC parsing phase, thereby increasing the flexibility of the hardware design.

[0096]

[0122] Figure 16 illustrates the decoder hardware design in palette mode. Data structures 1601 and 1602 are illustrated exemplarily for CABAC and the decoded palette pixels. The decoder hardware design 1603 includes a predictor update module 1631 and a CABAC parsing module 1632, where the predictor update process and the CABAC parsing process proceed concurrently. Therefore, CABAC can continue parsing without waiting for the palette update process.

[0097]

[0123] To set the number of reuse flags to a fixed value for each coded block, in some embodiments, the size of the palette predictor is initialized to a predefined value at the beginning of each slice in the case of non-wavefronts, and at the beginning of each CTU row in the case of wavefronts. The predefined value is set to the maximum size of the palette predictor. In one example, the number of reuse flags is 31 or 63, depending on the slice type and dual-tree mode setting. When the slice type is an I slice and dual-tree mode is on (referred to as case 1), the number of reuse flags is set to 31. Otherwise (the slice type is a B slice / P slice, or the slice type is an I slice and dual-tree mode is off (referred to as case 2)), the number of reuse flags is set to 63. It is possible to use a number of reuse flags other than these two cases, and it may be beneficial to maintain a relationship where the number of reuse flags in case 1 is twice the number of reuse flags in case 2. In addition, when initializing the palette predictor, the value of each entry and each component are set to 0 or (1 << (sequence bit depth - 1)).

[0098]

[0124] For example, Figure 17 shows an example of a palette mode decoding process consistent with embodiments of the present disclosure. Changes between the conventional design in Figure 10 and the disclosed design in Figure 17, including the deleted portion 1701, are highlighted with strikethrough text, and the added portion 1702 is enclosed in a dashed line. Similarly, Figure 19 shows an example of an initialization process consistent with embodiments of the present disclosure. Changes between the conventional design in Figure 18 and the disclosed design in Figure 19, including the deleted portion 1901, are highlighted with strikethrough text, and the added portion 1902 is enclosed in a dashed line.

[0099]

[0125] In some embodiments, if the reuse flag is set to a fixed value, or if the embodiment of Figure 9 is implemented, some redundancy may occur in the pallet predictor. In some embodiments, this may not be a problem because those redundant entries are not selected for prediction on the encoder side. Examples of pallet predictor updates are shown in Figures 20, 21, and 22. Figure 20 shows an example following the procedure specified in VVC Draft 8, Figure 21 shows an example following one of several implementations of the method proposed in Figure 9 (referred to as the first embodiment), and Figure 22 shows another example following one of several implementations of an embodiment in which the reuse flag is set to a fixed value (referred to as the third embodiment). As a comparison of Figures 20, 21, and 22 shows, assuming there are no new signaled palette entries, the method proposed in the first embodiment (Figure 21) can increase the number of bits used to signal the reuse flag 2101, whereas the method proposed in the third embodiment (Figure 22) retains the same number of bits used to signal the reuse flag 2201 as the method in the VVC draft 8 design (Figure 20) for signaling the reuse flag 2001.

[0100]

[0126] In some embodiments, the palette predictor is initially initialized to a fixed value. This means that the palette predictor may have redundant entries. As mentioned earlier, redundant entries may not always be a problem, but the design of this embodiment does not prevent the encoder from using those redundant entries. If the encoder happens to select one of these redundant entries, the encoding performance of the palette predictor, and therefore the performance of the palette mode, may also be degraded. To prevent this case, in some embodiments, bitstream compatibility is added when signaling the reuse flag. In some embodiments, the bitstream compatibility is such that when signaling the reuse flag, the size value of the palette predictor is equal to the maximum size of the palette predictor. More specifically, a range constraint is added to the binarized value of the reuse flag.

[0101]

[0127] For example, Figure 23 shows an example of palette-coded syntax consistent with embodiments of the present disclosure. Changes between the conventional design in Figure 7 and the disclosed design in Figure 23, including the deleted portion 2301, are highlighted with strikethrough text, and the added portion 2302 is enclosed in a dashed line. Similarly, Figure 25 shows an example of palette-coded semantics consistent with embodiments of the present disclosure. Changes between the conventional design in Figure 24 and the disclosed design in Figure 25, including the deleted portion 2501, are highlighted with strikethrough text, and the added portion 2502 is enclosed in a dashed line.

[0102]

[0128] Furthermore, this disclosure addresses the challenges of palette-mode deblocking filters by providing the following methods.

[0103]

[0129] In some embodiments, the palette mode is treated as a subset of the intra-predictive mode. Therefore, if one of the adjacent blocks is coded in palette mode, the boundary filter strength is set to equal 2. More specifically, the part of the VVC Draft 8 specification detailing the process for calculating the boundary filter strength is changed so that Scenario 3 is described as follows: "If sample p0 or q0 is in a coded block of a coded unit coded using intra-predictive mode or palette mode, then bS[xDi][yDj] is set to equal 2." The added language is underlined.

[0104]

[0130] In some embodiments, palette modes are treated as independent coding modes. When the coding modes of two adjacent blocks are different, the boundary filter strength is set to equal 1. More specifically, the part of the VVC Draft 8 specification detailing the process of calculating the boundary filter strength is changed so that Scenario 6 is described as follows: "If the prediction mode of the coding subblock containing sample p0 is different from the prediction mode of the coding subblock containing sample q0, then bS[xDi][yDj] is set to equal 1." The language to be deleted is struck through.

[0105]

[0131] In some embodiments, palette mode is treated as a separate coding mode. Similar to the BDPCM mode setting, when one of the coded blocks is in palette mode, the boundary filter strength is set to equal to 0. More specifically, the part of the VVC Draft 8 specification detailing the process for calculating the boundary filter strength is modified so that a new scenario is inserted between scenarios 3 and 4, and is described as follows: "Otherwise, if a block edge is also an edge of a coded block, and sample p0 or q0 is in a coded block where pred_mode_plt_flag is equal to 1, then bS[xDi][yDj] is set to equal to 0." Using this modified process, scenarios 4-8 are renumbered to scenarios 5-9, resulting in 9 scenarios. It should be noted that a block coded in palette mode is equivalent to a pred_mode_plt_flag equal to 1.

[0106]

[0132] In some embodiments, non-temporary computer-readable storage media containing instructions are also provided, and the instructions may be executed by devices (such as disclosed encoders and decoders) for performing the methods described above. Common forms of non-temporary media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tapes, or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media having a pattern of holes, RAM, PROMs, and EPROMs, FLASH®-EPROMs, or any other flash memory, NVRAMs, caches, registers, any other memory chips or cartridges, and networked versions thereof. Devices may include one or more processors (CPUs), input / output interfaces, network interfaces, and / or memory.

[0107]

[0133] In this specification, relational terms such as “first” and “second” are used solely to distinguish one entity or action from another, and do not imply or require any actual relationship or order between these entities or actions. Furthermore, “comprising,” “having,” “containing,” and “including,” as well as other similar forms of words, are intended to be equivalent in meaning and are intended to be non-restrictive, not intended to be an exhaustive list of such items, nor to be limited to only the listed items.

[0108]

[0134] As used herein, unless otherwise specified, the term “or” encompasses all possible combinations, except in cases where it is impractical. For example, if we say that a database may contain A or B, then unless otherwise specified or impractical, that database may contain A, B, A and B. As a second example, if we say that a database may contain A, B, or C, then unless otherwise specified or impractical, that database may contain A, B, C, A and B, A and C, B and C, A and B and C.

[0109]

[0135] It will be understood that the embodiments described above can be implemented in hardware, software (program code), or a combination of hardware and software. When implemented in software, the software can be stored in the computer-readable medium described above. When executed by a processor, the software can perform the methods disclosed. The computing units and other functional units described in this disclosure can be implemented in hardware, software, or a combination of hardware and software. Those skilled in the art will also understand that multiple of the above modules / units can be combined into a single module / unit, and that each of the above modules / units can be further divided into multiple submodules / subunits.

[0110]

[0136] The embodiments can be further described using the following clauses. 1. A video data processing method comprising processing one or more coded units using one or more palette predictors, One or more pallet predictors, Add all palette entries from the current palette as the first set of entries in the palette predictor, and To add palette entries from the previous palette predictor as a second set of entries in the palette predictor, regardless of the reuse flag value of the palette entries in the previous palette predictor, and as a second set of entries following the first set of entries. A video data processing method that is updated by [this method]. 2. The method according to Clause 1, wherein each pallet entry in the first set of entries and the second set of entries of the pallet predictor includes a reuse flag. 3. Receiving video frames for processing, To generate one or more coded units of video frames, The method described in Clause 1, further including the method described in Clause 1. 4. Each pallet entry in the pallet predictor has a corresponding reuse flag. The method according to any one of clauses 1 to 3, wherein the number of reuse flags for palette predictors is set to a fixed number for the corresponding coding unit. 5. A video data processing method comprising processing one or more coded units using one or more palette predictors, One or more pallet predictors, Add all palette entries from the current palette as the first set of entries in the palette predictor. Adding one or more pallet entries of a previous pallet predictor to the pallet predictor as a second set of one or more entries of the pallet predictor, within a first range which begins with a first pallet entry of the previous pallet predictor and ends with a first pallet entry of the previous pallet predictor having a reuse flag set, and Adding one or more palette entries of a previous palette predictor to the palette predictor as a third set of one or more entries of the palette predictor, which are in a second range, starting from the last palette entry of the previous palette predictor that has a reuse flag set and ending with the last palette entry of the previous palette predictor. A video data processing method in which the data is updated, and a second set of entries and a third set of entries follow the first set of entries. 6. The method according to Clause 5, wherein each pallet entry in the first set of entries, the second set of entries, and the third set of entries of the pallet predictor includes a reuse flag. 7. Receiving video frames for processing, To generate one or more coded units of video frames, The method described in Clause 5, further including the method described in Clause 5. 8. Each pallet entry of one or more pallet predictors has a corresponding reuse flag. The method according to any one of clauses 5 to 7, wherein the number of reuse flags for each palette predictor is set to a fixed number for the corresponding coding unit. 9. A method for processing video data, To process the video frame, To generate one or more coded units of video frames, Processing one or more coded units using one or more palette predictors having palette entries, Includes, Each pallet entry of one or more pallet predictors has a corresponding reuse flag. A video data processing method in which the number of reuse flags for each palette predictor is set to a fixed number for the corresponding coding unit. 10. The palette predictor, Add all palette entries from the current palette as the first set of entries in the palette predictor, and Adding entries from previous palette predictors that are not reused in the current palette as a second set of entries in the palette predictor, which follows the first set of entries. The method described in Article 9, as updated by... 11. The method according to Clause 9, wherein a fixed number is set based on the slice type and dual tree mode setting. 12. The method according to clause 9, wherein the size of one of the one or more palette predictors is initialized to a predefined value at the beginning of the slice if it is not a wavefront. 13. The method according to Clause 9, wherein the size of one of the one or more palette predictors is initialized to a predefined value at the beginning of the coded unit line in the case of a wavefront. 14. The method of clause 12 or 13, further comprising adding a bitstream compatibility that the size value of the palette predictor is equal to the maximum size of the palette predictor when signaling the reuse flag. 15. The method according to any one of clauses 12 to 14, wherein when initializing one or more palette predictors, the value of each entry and each component are set to 0 or (1 << (sequence bit depth - 1)). 16. The method described in any one of clauses 9 to 15, further comprising imposing a range constraint on the binarized value of the reuse flag. 17. A device for performing video data processing, Memory configured to store instructions, It includes a processor coupled to memory, and the processor executes instructions to the device, Use one or more palette predictors to process one or more coded units. It is configured to make the thing happen, One or more pallet predictors, Add all palette entries from the current palette as the first set of entries in the palette predictor, and To add palette entries from the previous palette predictor as a second set of entries in the palette predictor, regardless of the reuse flag value of the palette entries in the previous palette predictor, and as a second set of entries following the first set of entries. A device that is updated by [this method]. 18. The apparatus described in Clause 17, wherein each pallet entry in the first set of entries and the second set of entries of the pallet predictor includes a reuse flag. 19. The processor executes instructions and sends them to the device. To process the video frame, and, To generate one or more coded units of video frames. The apparatus described in Clause 17, further configured to perform the following actions. 20. Each palette entry in the palette predictor has a corresponding reuse flag. The apparatus described in any one of clauses 17 to 19, wherein the number of reuse flags for pallet predictors is set to a fixed number for the corresponding coding unit. 21. A device for performing video data processing, Memory configured to store instructions, It includes a processor coupled to memory, and the processor executes instructions to the device, Use one or more palette predictors to process one or more coded units. It is configured to make the thing happen, One or more pallet predictors, Add all palette entries from the current palette as the first set of entries in the palette predictor. Adding one or more pallet entries of a previous pallet predictor to the pallet predictor as a second set of one or more entries of the pallet predictor, within a first range which begins with a first pallet entry of the previous pallet predictor and ends with a first pallet entry of the previous pallet predictor having a reuse flag set, and Adding one or more palette entries of a previous palette predictor to the palette predictor as a third set of one or more entries of the palette predictor, which are in a second range, starting from the last palette entry of the previous palette predictor that has a reuse flag set and ending with the last palette entry of the previous palette predictor. The device is updated by a second set of entries and a third set of entries, which are located after the first set of entries. 22. The apparatus as described in Clause 21, wherein each pallet entry in the first set of entries and the second set of entries of the pallet predictor includes a reuse flag. 23. The processor executes instructions and sends them to the device. To process the video frame, and, To generate one or more coded units of video frames. The apparatus described in Clause 21, further configured to perform the following actions. 24. Each palette entry in the palette predictor has a corresponding reuse flag. The apparatus according to any one of clauses 21 to 23, wherein the number of reuse flags for pallet predictors is set to a fixed number for the corresponding coding unit. 25. A device for performing video data processing, Memory configured to store instructions, It includes a processor coupled to memory, and the processor executes instructions to the device, Receiving video frames for processing, To generate one or more coded units of video frames, and, Processing one or more coded units using one or more palette predictors that have palette entries. It is configured to execute, Each pallet entry of one or more pallet predictors has a corresponding reuse flag. A device in which the number of reuse flags for each pallet predictor is set to a fixed number for the corresponding coded unit. 26. The processor executes instructions to the device, Add all palette entries from the current palette as the first set of entries in the palette predictor, and Adding entries from previous palette predictors that are not reused in the current palette as a second set of entries in the palette predictor, which follows the first set of entries. The apparatus described in Clause 25, further configured to cause the pallet predictor to perform an update. 27. The apparatus described in Clause 25, wherein the fixed number is set based on the slice type and dual tree mode setting. 28. The apparatus according to Clause 25, wherein the size of one of the one or more pallet predictors is initialized to a predefined value at the beginning of the slice if it is not a wavefront. 29. The apparatus described in Clause 25, wherein the size of one of the one or more pallet predictors is initialized to a predefined value at the beginning of the coded unit line in the case of a wavefront. 30. The processor executes instructions and sends them to the device. When signaling the reuse flag, add a bitstream compatibility condition where the size of the palette predictor is equal to the maximum size of the palette predictor. The apparatus described in Clause 28 or 29, further configured to perform the following: 31. The apparatus described in any one of clauses 28 to 30, wherein when initializing one or more palette predictors, the value of each entry and each component are set to 0 or (1 << (sequence bit depth - 1)). 32. The processor executes instructions and sends them to the device. Add a range constraint to the binarized value of the reuse flag. The apparatus described in any one of the clauses 25 to 31, further configured to perform the following actions. 33. A non-temporary computer-readable medium for storing a set of instructions, which is executable by one or more processors of a device for causing the device to initiate a method for performing video data processing, and the method is This includes processing one or more coded units using one or more palette predictors, One or more pallet predictors, Add all palette entries from the current palette as the first set of entries in the palette predictor, and Add the palette entries from the previous palette predictor as a second set of entries in the palette predictor, regardless of the reuse flag value of the palette entries in the previous palette predictor, and as a second set of entries following the first set of entries. A non-temporary, computer-readable medium that is updated by [a specific source]. 34. Each palette entry in the first set of entries and the second set of entries of the palette predictor includes a reuse flag in the non-temporary computer-readable medium as described in Clause 33. 35. The method is To process the video frame, To generate one or more coded units of video frames, Non-temporary computer-readable media as described in Clause 33, further including the above. 36. Each pallet entry in the pallet predictor has a corresponding reuse flag. A non-temporary computer-readable medium as described in any one of clauses 33 to 35, wherein the number of reuse flags for the palette predictor is set to a fixed number for the corresponding coding unit. 37. A non-temporary computer-readable medium for storing a set of instructions, which is executable by one or more processors of a device for causing the device to initiate a method for performing video data processing, the method being: This includes processing one or more coded units using one or more palette predictors, One or more pallet predictors, Add all palette entries from the current palette as the first set of entries in the palette predictor. Adding one or more pallet entries of a previous pallet predictor to the pallet predictor as a second set of one or more entries of the pallet predictor, within a first range which begins with a first pallet entry of the previous pallet predictor and ends with a first pallet entry of the previous pallet predictor having a reuse flag set, and Adding one or more palette entries of a previous palette predictor to the palette predictor as a third set of one or more entries of the palette predictor, which are in a second range, starting from the last palette entry of the previous palette predictor that has a reuse flag set and ending with the last palette entry of the previous palette predictor. A non-temporary computer-readable medium, updated by such a system, where the second set of entries and the third set of entries follow the first set of entries. 38. Each palette entry in the first set of entries and the second set of entries of the palette predictor includes a reuse flag in the non-temporary computer-readable medium as described in Clause 37. 39. The method is To process the video frame, To generate one or more coded units of video frames, Non-temporary computer-readable media as described in Clause 37, including, furthermore. 40. Each pallet entry in the pallet predictor has a corresponding reuse flag. A non-temporary computer-readable medium as described in any one of clauses 37 to 39, wherein the number of reuse flags for palette predictors is set to a fixed number for the corresponding coding unit. 41. A non-temporary computer-readable medium for storing a set of instructions, which is executable by one or more processors of a device for causing the device to initiate a method for performing video data processing, the method being: To process the video frame, To generate one or more coded units of video frames, Processing one or more coded units using one or more palette predictors having palette entries, Includes, Each pallet entry of one or more pallet predictors has a corresponding reuse flag. A non-temporary, computer-readable medium in which the number of reuse flags for each palette predictor is set to a fixed number for the corresponding coding unit. 42. The palette predictor, Add all palette entries from the current palette as the first set of entries in the palette predictor, and Adding entries from previous palette predictors that are not reused in the current palette as a second set of entries in the palette predictor, which follows the first set of entries. Non-temporary computer-readable media as defined in Article 41, as updated by [the relevant authority]. 43. Non-temporary computer-readable media as described in Clause 41, where a fixed number is set based on the slice type and dual-tree mode setting. 44. A non-temporary computer-readable medium as described in Clause 41, wherein the size of one of the one or more palette predictors is initialized to a predefined value at the beginning of the slice, in the case of a non-wavefront. 45. A non-temporary computer-readable medium as described in Clause 41, in which the size of one of the one or more palette predictors is initialized to a predefined value at the beginning of the coded unit line in the case of a wavefront. 46. ​​When signaling the reuse flag, add a bitstream compatibility condition that the size of the palette predictor is equal to the maximum size of the palette predictor. Non-temporary computer-readable media as described in Clause 44 or 45, further including the following: 47. A non-transient computer-readable medium as described in any one of clauses 44 to 46, in which, when initializing one or more palette predictors, the value of each entry and each component are set to 0 or (1 << (sequence bit depth - 1)). 48. Add a range constraint to the binarized value of the reuse flag. Non-temporary computer-readable media as described in any one of clauses 41 to 47, further including the above. 49. A method for deblocking a filter in palette mode, To process the video frame, The method involves generating one or more coded units of a video frame, wherein each coded unit of the one or more coded units has one or more coded blocks. In response to at least the first of two adjacent coded blocks being coded in palette mode, the boundary filter strength is set to 2. A method that includes this. 50. A method for deblocking a filter in palette mode, To process the video frame, The method involves generating one or more coded units of a video frame, wherein each coded unit of the one or more coded units has one or more coded blocks. In response to the fact that at least the first of two adjacent coded blocks is coded in palette mode, and the second of the two adjacent coded blocks has a coding mode different from palette mode, the boundary filter strength is set to 1. A method that includes this. 51. A device for performing video data processing, Memory configured to store instructions, It includes a processor coupled to memory, and the processor executes instructions to the device, Receiving video frames for processing, The process involves generating one or more coded units of a video frame, wherein each coded unit of the one or more coded units has one or more coded blocks, and The boundary filter strength is set to 1 in response to the fact that at least the first of two adjacent coded blocks is coded in palette mode, and the second of the two adjacent coded blocks has a coding mode different from palette mode. A device configured to perform a certain action. 52. A non-temporary computer-readable medium for storing a set of instructions, which is executable by one or more processors of a device for causing the device to initiate a method for performing video data processing, the method being: To process the video frame, The method involves generating one or more coded units of a video frame, wherein each coded unit of the one or more coded units has one or more coded blocks. In response to the fact that at least the first of two adjacent coded blocks is coded in palette mode, and the second of the two adjacent coded blocks has a coding mode different from palette mode, the boundary filter strength is set to 1. Non-temporary computer-readable media, including [specific examples of such media].

[0111]

[0137] The above specification has described embodiments with respect to numerous specific details that may vary depending on the implementation. Certain adaptations and modifications can be made to the embodiments described. Other embodiments may become apparent to those skilled in the art by examining this specification and practicing the invention disclosed herein. This specification and examples are to be considered merely illustrative, and the true scope and spirit of the invention are intended to be shown by the following claims. The order of steps shown in the figures is also intended to be for illustrative purposes only and is not intended to limit the sequence of steps to any particular order. Therefore, those skilled in the art will understand that these steps can be performed in different orders while implementing the same method.

[0112]

[0138] The drawings and this specification have disclosed exemplary embodiments. However, many variations and modifications can be made to these embodiments. Accordingly, certain terms have been used, but these terms are used only in a general and descriptive sense and not for limiting purposes.

Claims

1. A method for encoding a video sequence, Receiving a video sequence containing one or more video frames, The aforementioned video sequence, The method for generating two or more coding units of the aforementioned video frame, wherein each of the one or more coding units has one or more coding blocks. To determine whether at least one of the first coding block of two adjacent coding blocks and the second coding block of the two adjacent coding blocks is coded in intra-predictive mode. In response to the fact that the first coding block and the second coding block are not coded in the intra-prediction mode, it is determined whether at least one of the first coding block and the second coding block is coded in CIIP (composite intra- and intra-prediction) mode. In response to the fact that the first coding block and the second coding block are not coded in CIIP mode, it is determined whether at least one of the first coding block and the second coding block is coded in palette mode, and In response to the fact that the first of two adjacent coded blocks is coded in palette mode, and the second of the two adjacent coded blocks has a coding mode different from the palette mode, the boundary filter strength is set to 1. Encoding by and Methods that include...

2. A method for decoding a bitstream, Receiving a bitstream and Decoding the bitstream in order to output the video sequence, Includes, The aforementioned decoding process Decoding two or more coding units from the bitstream, wherein each of the two or more coding units has one or more coding blocks. Determining whether at least one of the first coding block and the second coding block is coded in intra-predictive mode, In response to the fact that the first coding block and the second coding block are not coded in the intra-prediction mode, it is determined whether at least one of the first coding block and the second coding block is coded in CIIP (composite intra- and intra-prediction) mode, In response to the fact that the first coded block and the second coded block are not coded in CIIP mode, it is determined whether at least one of the first coded block and the second coded block is coded in palette mode. In response to the fact that the first of two adjacent coded blocks is coded in palette mode, and the second of the two adjacent coded blocks has a coding mode different from the palette mode, the boundary filter strength is set to 1. Methods that include...

3. A method for storing a bitstream of a video sequence, Receiving a video sequence, Encoding one or more pictures of the aforementioned video sequence, Generating a bitstream, The bitstream is stored in a non-temporary computer-readable storage medium, Includes, The above encoding is The method of generating two or more coded units of the aforementioned video frame, wherein each of the one or more coded units has one or more coded blocks, Determining whether at least one of the first coding block of two adjacent coding blocks and the second coding block of the two adjacent coding blocks is coded in intra-predictive mode, In response to the fact that the first coding block and the second coding block are not coded in the intra-prediction mode, it is determined whether at least one of the first coding block and the second coding block is coded in CIIP (composite intra- and intra-prediction) mode, In response to the fact that the first coded block and the second coded block are not coded in CIIP mode, it is determined whether at least one of the first coded block and the second coded block is coded in palette mode. In response to the fact that the first of two adjacent coded blocks is coded in palette mode, and the second of the two adjacent coded blocks has a coding mode different from the palette mode, the boundary filter strength is set to 1. Methods that include...

Images