Video encoding / decoding method and apparatus using palette mode

Abstract

A system and method for video encoding and decoding using palette mode is provided. A decoding method includes receiving, from a bitstream, a plurality of syntax elements associated with a coding unit, the syntax elements indicating a coding tree type for the coding unit and whether a local binary tree mode is enabled for the coding unit, and, in response to determining that the coding tree type is a single tree and that the local binary tree mode is enabled for the coding unit, disabling palette mode for the coding unit if the coding unit has a size equal to or smaller than a predetermined threshold, where disabling palette mode for the coding unit comprises disabling palette mode for both luma and chroma components of the coding unit or disabling palette mode for only the chroma component of the coding unit.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is filed on March 27, 2020, and is related to a patent application entitled "Palette Mode "Video Encoding and Decoding Method and Apparatus" is a provisional application of U.S. Patent Application No. 63 / 001235. No. 60 / 629,999, filed on Oct. 1, 2003, entitled "Patent Application No. 10 / 1 ... Use. [Technical Field]

[0002] The present invention relates generally to video data encoding, decoding and compression, and more particularly to palette mode The present invention relates to a method and system for video encoding and decoding using a digital video decoder. [Background technology]

[0003] Digital television, laptop or desktop computer, tablet computer Computers, digital cameras, digital recording devices, digital media players, Video game consoles, smartphones, video conferencing equipment, video streaming equipment, etc. Various electronic devices support digital video. Electronic devices include MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding(AVC), High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC) ) standard, it performs the video compression / decompression standards defined in the Video compression is usually spatial (within a frame) ) prediction and / or temporal (inter-frame) prediction to account for the redundancies inherent in the video data In block-based video coding, the video frame is divided into multiple coding tree units (CTUs). Each CTU is partitioned into one or more slices, each containing a video block. contains coding units (CUs) or until the minimum syntax-defined CU size is reached It may be recursively partitioned into smaller CUs. Each CU (also called a leaf CU) consists of one or more transform units (TUs) and one or more Each C U can be coded in either intra, inter, or IBC mode. The video blocks within an intra-coded (I) slice in a video frame are The block is a spatial prediction with respect to reference samples in neighboring blocks in the same video frame. Inter-coded (P or B) sequences in one video frame Video blocks in a frame are compared to reference samples in adjacent blocks in the same video frame. spatial prediction of, or reference to, other previous and / or future reference video frames Use a time prediction on the sample.

[0004] Previously coded reference blocks, e.g., spatial or temporal prediction based on neighboring blocks. is the predicted block of the current video block to be coded. The developing process can be realized by a block matching algorithm. Residual data indicating pixel differences between the current block to be coded and the predicted block is Inter-coded blocks are called difference blocks or prediction errors. The motion vector pointing to the reference block in the reference frame is coded according to the residual block. The process of determining motion vectors is usually called motion estimation. The coded block is coded according to the intra prediction mode and the residual block. For the resulting compression, the residual block is transformed from the pixel domain to a transform domain, e.g., the frequency domain. , resulting in residual transform coefficients that are then quantified. Then, first in a two-dimensional matrix The positioned and quantified transform coefficients are scanned to generate a one-dimensional vector of transform coefficients; It is then entropy coded into a video bitstream to achieve further compression. can be.

[0005] and storing the encoded video bitstream on a computer-readable storage medium. stored on another electronic device with digital video capabilities (e.g., flash memory) or transmitted directly to this electronic device via wire or wirelessly; and The electronic device may, for example, analyze an encoded video bitstream and extract the bitstream data. Gets syntax elements from the bitstream and uses a small number of syntax elements from this bitstream. and extracting digital video data based at least in part on this encoded video stream. Video decompression (the opposite of video compression mentioned above) is achieved by reconstructing the video from its original format. This reconstructed digital video data is then transmitted to the digital Reproduced on the display.

[0006] As digital video quality progresses from high definition to 4Kx2K and 8Kx4K, As a result, the amount of video data to be encoded / decoded increases exponentially. More efficiently encode / decode video data while maintaining the image quality of the video data That is always a challenge. Summary of the Invention

[0007] This application relates to video data encoding and decoding, and in particular to video encoding using palette mode. Implementations for encoding and decoding systems and methods are described.

[0008] According to a first aspect of the present application, the code is extracted from the bitstream in relation to the coding unit. The type of coding tree of the coding unit and the local binary tree mode for the coding unit receiving a plurality of syntax elements indicating whether the syntax elements are valid for the encoding unit; The type of the encoding tree is a single tree and the local binary tree mode is In response to determining that the coding unit is valid, the coding unit is below a predetermined threshold. and disabling palette mode for said coding unit if it has a size of The present invention provides a method for decoding video data, including:

[0009] In one embodiment, disabling palette mode for the coding unit comprises: Disabling the palette mode for both the luma and chroma components of the coding unit This includes making

[0010] In one embodiment, disabling palette mode for the coding unit comprises: and disabling the palette mode for only the chroma component of the coding unit. .

[0011] According to a second aspect of the present application, an electronic device includes one or more processing units, a memory, and and a plurality of programs stored in the memory. When executed by the one or more processing units, the electronic device The method performs a method for decoding the data.

[0012] According to a third aspect of the present application, the non-transitory computer-readable storage medium comprises one or A computer program that stores multiple programs to be executed by an electronic device having multiple processing units. The program, when executed by the one or more processing units, The electronic device is caused to perform the method for decoding video data described above. [Brief explanation of the drawings]

[0013] The present invention will be better understood by those skilled in the art through the following detailed description, which is incorporated herein by reference in its entirety. The accompanying drawings illustrate the implementations described above and together with the description serve to explain the underlying principles. The same reference numerals indicate the same or equivalent parts.

[0014] [Figure 1] FIG. 1 is a block diagram illustrating an example video encoding and decoding system according to an embodiment of this disclosure. [Figure 2] FIG. 2 is a block diagram illustrating an example video encoder according to an embodiment of this disclosure. [Figure 3] FIG. 3 is a block diagram illustrating an example video decoder according to an embodiment of this disclosure. [Figure 4] 4A-4E are block diagrams illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, according to one embodiment of this disclosure. [Figure 5] 5A-5D are block diagrams illustrating examples of using palette mode for encoding and decoding video data according to some embodiments of the present disclosure. [Figure 6] FIG. 6 is a flowchart illustrating an example process for implementing a technique for decoding video data by a video decoder, according to one embodiment of this disclosure. [Figure 7] FIG. 7 is a block diagram illustrating an example context-adaptive binary arithmetic coding (CABAV) engine, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Therefore, numerous non-limiting specific details are set forth in order to facilitate understanding of the subject matter set forth herein. However, the present invention may be modified in various ways without departing from the scope and spirit of the claims. It will be apparent to those skilled in the art that modifications can be made to the embodiments described herein. It is understood that the principles described above can be implemented in many types of electronic devices with digital video capabilities. It is clear to the trader.

[0016] FIG. 1 illustrates a block diagram of a video block encoding and decoding system according to an embodiment of the present disclosure. 1 is a block diagram illustrating an exemplary system 10 for 0 is a source that generates and encodes video data that will be decoded by the target device 14. The source device 12 and the destination device 14 may include a desktop or laptop laptops, tablet computers, smartphones, set-top boxes Digital TVs, cameras, display devices, digital media players, video games may include any of a variety of electronic devices, including computers, video streaming devices, etc. In one embodiment, source device 12 and target device 14 are equipped with wireless communication capabilities. .

[0017] In one embodiment, target device 14 receives the encoded video to be decoded via link 16. Link 16 receives encoded video data from source device 12. The communication medium or device may include any type of communication medium or device that can be transferred from the network to the target device 14. In one example, link 16 may include a link for transmitting video data encoded on source device 12 to a destination device. The encoded data may include a communication medium that allows the data to be transmitted directly to the device 14 in real time. The received video data is modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the target device 1. 4. The communication medium is radio frequency (RF) It includes any wireless or wired communications medium, such as spectrum and one or more physical transmission paths. The communication medium can be a local area network, a wide area network, packet-based networks such as networks or global networks like the Internet It may be configured as part of a network. The communication medium may include routers, switches, base stations, etc. , and any other device that facilitates communication from source device 12 to target device 14.

[0018] In another embodiment, the encoded video data is output from the output interface 22. The encoded video data is then sent to the storage device 32. The data is accessed by the target device 14 via the input interface 28 . The storage device 32 may include a hard drive, Blu-ray disc, DVD, CD-R, etc. OM, flash memory, volatile or non-volatile memory, and encoded video data on a variety of distributed or local storage media, such as other suitable digital storage media for storing In another example, the storage device may include any of the data storage media that is accessed. The storage device 32 may be a file server or a file server that receives the encoded video data generated by the source device 12. The target device 14 may correspond to another intermediate storage device that can hold the data. The video data stored in the storage device 32 is streamed or downloaded. The file server stores the encoded video data. and any type of encoding that can transmit this encoded video data to the target device 14. An exemplary file server may be a web server (e.g., a a network-attached storage (NAS) device, or The target device 14 includes a local disk drive. A wireless channel (e.g., a Wi-Fi connection) suitable for accessing the encoded video data, any wired connection (e.g., DSL, cable modem, etc.), or a combination thereof The encoded video data can be accessed via a standard data connection The transmission of the encoded video data from the storage device 32 is performed as streaming transmission. It may be a download transmission, or a combination thereof.

[0019] As shown in FIG. 1, the source device 12 includes a video source 18, a video encoder 20, and and an output interface 22. The video source 18 includes a video capture device (e.g. video cameras), video archives containing previously captured video, video conferencing A video feed interface for receiving video from content providers, and / or or a computer for generating computer graphics data as source video may include sources such as digital graphics systems, or a combination thereof. As an example, if the video source 18 is a video camera in a security surveillance system, If the source device 12 and the destination device 14 are cameras or video cameras, However, the embodiments described herein are generally applicable to video coding. The present invention is applicable to wireless and / or wired applications.

[0020] The video encoder 20 may be used to convert captured video, pre-captured video, or computer generated video. The video generated by the data can be encoded. The encoded video data can be directly to the destination device 14 via the output interface 22 of the source device 12; Additionally (or alternatively), the encoded video data can then be to be accessed by the target device 14 or other devices for decryption and / or playback. The output interface 22 may be a modem and The device may further include a sensor and / or a transmitter.

[0021] The target device 14 includes an input interface 28, a video decoder 30, and a display device 31. 4. Input interface 28 includes a receiver and / or modem, and includes link 16 received encoded video data via the link 16; or The encoded video data provided to the storage device 32 includes the encoded data encoded by the video encoder 20. Many syntaxes are generated and used by video decoder 30 to decode video data. These encoded video data may be transmitted over a communications medium or Whether stored on a storage medium or a file server, It may contain syntax elements such as:

[0022] In some embodiments, target device 14 may include an integrated display or other device capable of communicating with target device 14. The display 34 may include an external display configured to display: The decoded video data is displayed to the user on a liquid crystal display (LCD). ), plasma display, organic light-emitting diode (OLED) display, or another The display device may include any of a variety of display devices, such as a display device of the type.

[0023] The video encoder 20 and the video decoder 30 are compatible with VVC, HEVC, MPEG-4 ,Part 10,Advanced Video Coding (AVC) Operate in accordance with professional or industry standards, such as the International Business Machines Corporation (IBC) or extensions of such standards. Note that this application is not limited to a particular video encoding / decoding standard, and may be used with other video encoding / decoding It should be understood that the present invention is applicable to any video encoding standard. Reader 20 encodes video data according to any of these current or future standards. Similarly, the video decoder 30 of the target device 14 is configured to receive these current and is configured to decode video data according to any of the future standards.

[0024] The video encoder 20 and the video decoder 30 each include one or more microphones. processors, digital signal processors (DSPs), application specific integrated circuits (ASICs) , Field Programmable Gate Array (FPGA), Discrete Logic, Software software, hardware, firmware, or any combination thereof. This can be implemented by any suitable encoder circuit. When implemented in part by software, the electronic device executes the software instructions in an appropriate non-transitory computer. Stored on a computer-readable medium and executed in hardware by one or more processors The video encoding / decoding operations described in this disclosure may be performed by executing the instructions in The video encoder 20 and the video decoder 30 are coupled in their respective devices. One or more encoders integrated as part of an encoder / decoder (CODEC) The signal processing unit 100 may be included in the data processor 102 or the decoder 103.

[0025] FIG. 2 is a block diagram illustrating a video encoder 20 according to one embodiment described herein. 1 is a block diagram. Video encoder 20 performs image coding on a video block within a video frame. Both intra-predictive and inter-predictive coding can be performed. is dependent on spatial prediction and accounts for spatial redundancies in the video data within a given video frame or image. Inter-predictive coding relies on temporal prediction to reduce or eliminate the Reducing or removing temporal redundancy in video data within adjacent video frames or images do.

[0026] As shown in FIG. 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 4 1, a decoded picture buffer (DPB) 64, an adder 50, a conversion processing unit 52, a quantification unit 54, The prediction processing unit 41 includes a motion estimation unit 42, a motion compensation unit 56, and an entropy coding unit 56. unit 44, partition unit 45, intra prediction processing unit 46, intra block copy (BC) unit 48 In an embodiment, video encoder 20 also includes a video block The system further includes an inverse quantification unit 58 for reconstruction, an inverse transformation processing unit 60, and an adder 62. Between the adder 62 and the DPB 64, a block boundary is detected from the reconstructed video. Deblocking filter to remove blocky artifacts (Figure It is also possible to place a filter (not shown) to filter the output of the adder 62. To achieve this, an inner ring filter (not shown) may be used in addition to the deblocking filter. The video encoder 20 may be in the form of a fixed or programmable hardware unit. may be formed of fixed or programmable hardware units as shown The cells may be partitioned within one or more of the slots.

[0027] Video data memory 40 stores the video data to be encoded by components in video encoder 20. The video data in the video data memory 40 is stored in the The DPB64 is obtained from a video source 18. used when encoding (e.g., in intra- or inter-predictive coding mode) The video data memory 40 and the DPB 6 are buffers for storing reference video data. 4 can be formed of any of a variety of memory devices. Video data memory 40 is on-chip with other components in video encoder 20. or may be off-chip to those components.

[0028] As shown in FIG. 2, after receiving the video data, the partition unit 45 in the prediction processing unit 41 The video data is partitioned into video blocks, each of which contains a specific video block associated with the video data. The video frame is divided into sub-frames according to a predetermined partition structure, such as a quad-tree structure. This involves partitioning the image into slices, tiles, or other larger coding units (CUs). A video frame may be divided into multiple video blocks (or video tiles). The prediction processor 41 can divide the current video block into blocks (block sets). For a given block, multiple inputs are selected based on the error results (e.g., coding rate and distortion level). one of the tra-predictive coding modes or one of the inter-predictive coding modes One of multiple possible predictive coding modes is selected so that the predictive coding mode is selected. The prediction processor 41 provides the obtained intra- or inter-prediction coded block to the adder 50. The residual block is then used as part of the subsequent reference frame. The prediction processing unit 41 further reconstructs the motion vector, the intra-mode image, and the Syntax elements such as indicators, partition information and other syntax information are sent to the entropy coder 56. provide.

[0029] The intra prediction unit 46 in the prediction unit 41 generates an intra prediction block suitable for the current video block. To select the intra prediction coding mode, the same frame as the current block to be coded is selected. Intra prediction of the current video block relative to one or more neighboring blocks in the video block. By performing motion estimation coding, spatial prediction can be performed. The motion compensation unit 42 and the motion compensation unit 44 determine one or more predictions in one or more reference frames. By performing inter-predictive coding of the current video block relative to the block, The video encoder 20 performs a multi-pass encoding process to, for example, An appropriate coding mode may be selected for each block of data.

[0030] In one embodiment, the motion estimator 42 estimates a predetermined time period for a sequence of video frames. Prediction in the reference video frame for the current video frame according to the pattern A prediction unit (P The inter prediction mode is determined by generating a motion vector indicating the displacement of the motion vector U. The motion estimation performed by estimator 42 involves generating motion vectors that estimate the motion of video blocks. A motion vector is a vector that represents the current video frame or image. A reference frame (or other coding unit) for the current video block being coded in For a predicted block in the current video frame (or other coding unit), The displacement of the PU of the video block within the sequence can be indicated by a predetermined The pattern determines whether the video frames in this sequence are P-frames or B-frames. The intra BC unit 48 can specify the motion for inter prediction by the motion estimation unit 42. In the same way as the vector determination, vectors for intra BC coding, e.g., block Alternatively, a motion estimator 42 may be used to determine the block vector. The rule may be determined.

[0031] The predicted block is calculated using the sum of absolute differences (SAD), sum of squared differences (SSD) or other difference methods. The pixel difference between the PU of the video block to be coded and the The block in the reference frame to be matched. The coder 20 calculates the values ​​of the sub-integer pixel positions of the reference frame stored in the DPB64. For example, video encoder 20 may use a quarter-pixel position of the reference frame. The values ​​of the pixel positions, 1 / 8 pixel positions, or other fractional pixel positions may be interpolated. The motion estimation unit 42 performs a motion search process for all pixel and fractional pixel positions. It is possible to output motion vectors with fractional pixel accuracy.

[0032] The motion estimation unit 42 determines the position of the PU of the video block in the inter-prediction coded frame, A first reference frame identifying one or more reference frames each stored in the DPB64. Select from the first reference frame list (List0) or the second reference frame list (List1). The motion vector for this PU is calculated by comparing it with the position of the predicted block in the selected reference frame. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44. and then transmits it to the entropy coding unit 56.

[0033] The motion compensation performed by the motion compensation unit 44 involves the use of the motion vectors determined by the motion estimation unit 42. The motion compensation unit may include obtaining or generating a prediction block based on the motion vector. 44 receives the motion vector for the PU of the current video block, Locate the predicted block to which this motion vector points in one of the lists, and 64 and transfers this predicted block to adder 50. Adder 50 calculates the sum of the pixel values ​​of the current video block being coded by motion compensation unit 44. A residual video block of pixel difference values ​​is obtained by subtracting the pixel values ​​of the prediction block provided by The pixel difference values ​​forming the residual video block may be luma or chroma difference components, Alternatively, the motion compensation unit 44 may include a motion compensation unit for a video block of a video frame. It is possible to generate further syntax elements for the video data. Used by coder 30 in decoding video blocks of a video frame. The element may include, for example, a syntax element that defines a motion vector for identifying this prediction block. , any flags indicating prediction modes, or any other syntax information described herein. It should be noted that the motion estimation unit 42 and the motion compensation unit 44 may be referred to separately for conceptual purposes. Although shown, it may be highly integrated.

[0034] In one embodiment, the intra BC unit 48 is related to the motion estimation unit 42 and the motion compensation unit 44. Then, a vector can be generated in the same way as described above to obtain the predicted block. where the predicted block is in the same frame as the current block being coded and The vectors are called block vectors, not motion vectors. 8, determining the intra prediction mode used to encode the current block; In one example, the intra BC unit 48 may perform various The current block is coded using the intra prediction mode of Next, the intra BC unit 48 performs various tests. From the tested intra prediction modes, one suitable intra prediction is selected and used to For example, the intra BC unit 48 generates an intra mode indicator based on the rate-distortion The analysis was performed to calculate the rate distortion values ​​of the various tested intra prediction modes. The intra prediction mode with the best rate-distortion performance is selected as the appropriate intra prediction mode. In rate-distortion analysis, the block being coded and this code are usually The distortion between the coded block and the original uncoded block that was generated by coding. (or error) amount used to generate this coded block. The intra BC unit 48 determines the bit rate (i.e., the number of bits). The ratio is calculated from the distortion and rate of the block to be encoded, and the intra prediction mode is determined. It may be determined whether σ represents the best rate-distortion value for this block.

[0035] In another example, the intra BC unit 48 may be the entire or a part of the motion estimation unit 42 and the motion compensation unit 44. may be used in part to perform functions related to intra BC prediction according to embodiments described herein. In either case, for intra block copies, the predicted block , determined by sum of absolute differences (SAD), sum of squared differences (SSD) or other difference metrics The pixel difference that can be determined is considered to be an exact match with the block to be coded. , and identifying the predicted block may involve calculating values ​​for sub-integer pixel positions. .

[0036] Video encoder 20 may generate a prediction block from the same frame based on intra prediction. whether they are from the same frame or from different frames based on inter prediction. Subtract the pixel values ​​of the predicted block from the pixel values ​​of the current video block being coded By generating pixel difference values, a residual video block can be generated. The pixel difference values ​​forming the lock may include both luminance and chrominance component differences.

[0037] The intra-prediction processing unit 46 is implemented by the motion estimation unit 42 and the motion compensation unit 44 described above. Inter prediction is performed by the intra prediction unit 48, or intra block prediction is performed by the intra BC unit 49. Instead of cross-copy prediction, intra prediction can be performed for the current video block. In particular, the intra prediction processing unit 46 determines one intra prediction mode and applies it to the current block. To achieve this, the intra prediction processing unit 46 performs, for example, For example, in the encoding process of each pass, various intra prediction modes are used to predict the current block. The intra prediction processor 46 (or in some examples, the mode selector) encodes the block, and the intra prediction processor 46 (or in some examples, the mode selector) Alternatively, an appropriate intra prediction mode may be selected from the selected intra prediction modes and used. The intra-prediction processor 46 indicates the intra-prediction mode selected for this block. The information may be provided to the entropy coding unit 56. The entropy coding unit 56 may Information indicating the selected intra-prediction mode can be coded into the bitstream.

[0038] The prediction processor 41 predicts the current video block by inter-prediction or intra-prediction. After determining the block, adder 50 subtracts this predicted block from the current video block. The residual video data in the residual block is The data is provided to the conversion processing unit 52 in the form of a plurality of conversion units (TUs). The unit 52 performs residual binarization, such as by a discrete cosine transform (DCT) or a conceptually similar transform. The video data is transformed into residual transform coefficients.

[0039] The conversion processing unit 52 transmits the obtained conversion coefficients to the quantification unit 54. These transform coefficients are then quantified to further reduce the bit rate. The bit depth associated with some or all of these coefficients can be reduced. or can be changed by adjusting the quantification parameters. In an example, the quantification unit 54 performs a scan on a matrix containing the quantified transform coefficients. This scanning may be performed by the entropy coding unit 56.

[0040] Following quantification, the entropy coding unit 56 may use, for example, a context-adaptive variable length code (CVLC). Context-Adaptive Binary Arithmetic Coding and Decoding (CAVLC), Context-Adaptive Binary Arithmetic Coding and Decoding (CABAC), Syntax-based context-adaptive binary arithmetic coding and decoding (SBAC), probability interval partitioning Entropy (PIPE) encoding / decoding or another entropy coding method or technique, entropy coding the quantified transform coefficients into a video bitstream; and , the encoded bitstream may be transmitted to video decoder 30, or It is then sent to the video decoder 30 or retrieved by the video decoder 30. The entropy coding unit 56 may also be used to archive the image data in the storage device 32. specifies the motion vectors and other syntax elements for the current video frame being coded. It may be entropy coded.

[0041] The inverse quantification unit 58 and the inverse transformation processing unit 60 respectively perform inverse quantification and inverse transformation to obtain: Residuals in the pixel domain for generating reference blocks used to predict other video blocks As described above, the motion compensation unit 44 reconstructs the video blocks in the DPB64 format. generating a motion compensated predicted block from one or more reference blocks of the stored frame; The motion compensation unit 44 may also apply one or more interpolation filters to this prediction block. A filter may be applied to calculate sub-integer pixel values ​​used for motion estimation.

[0042] Adder 62 adds the reconstructed residual block to the motion compensation block generated by motion compensation unit 44. This is added to the compensated predicted block to generate a reference block to be stored in the DPB64. The reference block is a predicted block, and is processed by the intra BC unit 48, the motion estimation unit 42, and the motion estimation unit 43. is used by compensation unit 44 to interpolate another video block in a subsequent video frame. -It is possible to predict.

[0043] FIG. 3 is a block diagram illustrating an exemplary video decoder 30 according to an embodiment of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction The processing unit 81, the inverse quantification unit 86, the inverse transformation processing unit 88, the adder 90, and the DPB 92 are included. The prediction processing unit 81 includes a motion compensation unit 82, an intra prediction processing unit 84, and an intra BC unit 85. The video decoder 30 further includes the following components in relation to the video encoder 20 with reference to FIG. A decoding process can be performed that is roughly the reverse of the encoding process described above. For example, The motion compensation unit 82 predicts the motion vectors based on the motion vectors received from the entropy decoding unit 80. The intra prediction unit 84 generates the measured data from the intra prediction unit 80. Prediction data may be generated based on the tra prediction mode indicator.

[0044] In some examples, one component in video decoder 30 is responsible for performing the implementation of the present invention. Also, in some examples, implementations of the present disclosure may involve one Alternatively, it may be divided into a plurality of components. For example, the intra BC unit 85 may be Alternatively, the motion compensation unit 82, the intra-prediction processing unit 84 and the encoder 86 may be implemented as a single unit. tropy decoding unit 80. In some examples, video decoder 30 may not include intra BC section 85, but may instead The function of the BC unit 85 is the same as that of other components in the prediction processing unit 81, such as the motion compensation unit 82. It may be realized by:

[0045] Video data memory 79 is used for decoding by other components in video decoder 30. It can store video data such as encoded video bitstreams. The video data stored in the video data memory 79 may be, for example, a wired or wireless Wired network communications and physical data storage media (e.g., flash drives and hard disks) access to storage devices 32 and local video sources such as cameras The video data memory 79 stores the data from the encoded video bitstream. It may also include a coded picture buffer (CPB) for storing coded video data. The decoded picture buffer (DPB) 92 in the video decoder 30 video data (e.g., in intra-prediction or inter-prediction coding / decoding modes) The video data memory 79 stores reference video data used for decoding the B92 supports synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), and resistive RAM ( Dynamic Random Access Memory (DRAM), including RRAM, or other types The memory device may be formed by any of a variety of memory devices, such as For convenience of explanation, the video data memory 79 and the DPB 92 are shown as video data 30 as two separate components. However, those skilled in the art will appreciate that The video data memory 79 and the DPB 92 may be the same memory device or separate memory devices. It is clear that the video data memory 79 is provided by a device. , may be on-chip with other components in video decoder 30, may be off-chip relative to the components.

[0046] During the decoding process, video decoder 30 extracts the video data of the encoded video frames. Receives an encoded video bitstream indicating ob-blocks and associated syntax elements. Video decoder 30 may perform video frame level and / or video block level The entropy decoding unit 80 of the video decoder 30 may receive the syntax elements in the form Entropy decoding of the bitstream to obtain quantified coefficients, motion vectors or images Generates intra-prediction mode indicators and other syntax elements. The decoding unit 80 transfers the motion vectors and other syntax elements to a prediction processing unit 81 .

[0047] Video frames are coded as intra-predictive (I) frames or other types When used for intra-coded prediction blocks in the frame, the prediction processing unit 81 The intra prediction processing unit 84 in the of the current video frame based on reference data from previously decoded blocks of the current video frame. Prediction data for the video block can be generated.

[0048] Video frames are coded into inter-predictive (i.e., B or P) frames. In this case, the motion compensation unit 82 in the prediction processing unit 81 receives the entropy from the entropy decoding unit 80. The video blocks of the current video frame are extracted based on the motion vectors and other syntax elements. It is possible to generate one or more prediction blocks for each block. is generated from a reference frame in one of the reference frame lists. 30 is based on the reference frame stored in DPB92 and is constructed using default construction techniques. It is possible to configure these reference frame lists, List0 and List1.

[0049] In one example, a video block is coded according to the intra BC mode described herein. In this case, the intra BC unit 85 in the prediction processing unit 81 receives the data from the entropy decoding unit 80. for the current video block based on the block vector and other syntax elements received from This predictive block is determined by the video encoder 20. The video block may be in the same reconstruction region of the image as the current video block.

[0050] The motion compensation unit 82 and / or the intra BC unit 85 process the motion vectors and other syntax elements. determining prediction information for a video block of the current video frame by analyzing the matrix; This prediction information is then used to generate a predicted block for the current video block being decoded. For example, the motion compensation unit 82 may use some of the received syntax elements to generate this block. a prediction mode (e.g., intra prediction) for encoding a video block of a video frame; Inter-predicted or inter-predicted), inter-predicted frame type (e.g., B or P), this frame Structural information of one or more reference frame lists for a frame, each instance of this frame the motion vectors of the inter-predictive video blocks, Inter prediction state of the video block and the video block in the current video frame Determine other information to decode the

[0051] Similarly, the intra BC unit 85 uses some of the received syntax elements, such as flags, to The current video block is predicted in intra BC mode, and Structural information regarding whether a video block is in the reconstruction area and should be stored in DPB 92 information, the block vectors of each intra BC predicted video block in this frame, The intra BC prediction state of each intra BC predicted video block in the frame, and the current determining other information for decoding a video block in a video frame; can.

[0052] Motion compensation unit 82 also uses the motion compensation data used by video encoder 20 in encoding the video blocks. The interpolation is performed using the interpolation filter used to obtain the sub-integer pixel interpolated value of the reference block. In this case, the motion compensation unit 82 calculates the video encoding from the received syntax elements. Determine the interpolation filter used by the coder 20 and use this interpolation filter to make a prediction. Blocks may be generated.

[0053] Inverse quantification unit 86 performs this calculation to determine the degree of quantification by video encoder 20. The same quantification parameters calculated for each video block in the video frame of , which are provided in the bitstream and are decoded by the entropy decoder 80. The inverse transform processor 88 inversely quantifies the quantified transform coefficients in the pixel domain. To reconstruct a certain residual block, an inverse transform, e.g., an inverse DCT, an inverse integer transform, or an approximate A similar inverse transform process is then applied to these transform coefficients.

[0054] The motion compensation unit 82 or the intra BC unit 85 performs the motion compensation based on the vectors and other syntax elements. After generating the prediction block for the current video block, adder 90 outputs the prediction block to inverse transform processor 8. 8 and the corresponding residual blocks generated by the motion compensation unit 82 and the intra BC unit 85. The decoded video block is calculated by adding the predicted block to the current video block. Between the adder 90 and the DPB 92, an in-loop filter (see FIG. (not shown) can be placed to further process this decoded video block. These decoded video blocks in a given frame are then used to decode the next video block. The reference frame used for future motion compensation of the block is stored in the DPB 92. In addition, the DPB 92 or a memory device separate from the DPB 92 may include the display device 34 shown in FIG. The decoded video may also be stored for subsequent display on a display device such as is possible.

[0055] In a typical video encoding / decoding process, a video sequence is typically Each frame contains a set of frames or images, each containing SL, SCb and SC SL can contain three sample matrices denoted by r. SL is a two-dimensional row of luminance samples. SCb is a two-dimensional matrix of Cb chroma samples. SCr is a two-dimensional matrix of Cr chroma samples. In another example, the frame may be monochrome, in which case It contains only one 2D matrix of luma samples.

[0056] As shown in FIG. 4A, video encoder 20 (or, more specifically, partition 45) ,The encoding of this frame is performed by first partitioning the frame into a set of coding tree units. A video frame is generated in raster scan order, from left to right and top to bottom. It contains an integer number of consecutively ordered CTUs. Each CTU is a maximum logical coding unit. The width and height of all CTUs in the video sequence are 128x128. , to have the same size being one of 64x64, 32x32 and 16x16 , are notified by the video encoder 20 in the sequence parameter set. As shown in Figure 4B, each CTU is a luminance sample. One coding tree block (CTB) for the chroma samples, and two corresponding coding tree blocks for the chroma samples. and syntax elements used to encode samples of the coding tree block. The syntax elements describe the attributes of different types of units of a coding block of pixels and how This describes how the video sequence is reconstructed in the video decoder 30. For example, inter- or intra-prediction, intra-prediction mode, motion vectors and and other parameters. For monochrome images or images with three separate color planes, A CTU encodes a single coding tree block and a sample of this coding tree block. A coding tree block may contain NxN sample blocks. It is possible that it is a lock.

[0057] To achieve better performance, the video encoder 20 may For a tree block, we can create tree partitions such as binary tree partitions, quad tree partitions, or a combination of both. We can recursively partition this CTU into smaller coding units (CUs). To achieve better performance, the video encoder 20 may For the coding tree block, binary tree partition, ternary tree partition, quad tree partition, or a combination of these Then, we recursively divide this CTU into smaller coding units (CUs) by As shown in FIG. 4C, the 64×64 CTU 400 can be first divided into 32 It is partitioned into four smaller CUs of ×32 block size. These four smaller Of the CUs, CU410 and CU420 each have four 16x16 block sizes. The two CUs 430 and 440 with a 16x16 block size are Each CU is further divided into four CUs of 8x8 block size. 4 shows a quadtree data structure representing the final result of the CTU400 partitioning process. Each leaf node corresponds to one CU of each size, from 32x32 to 8x8. Each CU, like the CTU shown in B, is a single component of the frame's luma samples of the same size. A coding block (CB) and two corresponding coding blocks of chroma samples, and syntax elements used to encode the samples of the coding block. For a 3D image or an image with three separate color planes, one CU is assigned to a single coding block. The coding block may include a sample block and the syntax structure used to code the samples of this coding block. Note that the quadtree partitions shown in Figures 4C and 4D are merely examples, and one CTU may be divided into quadtrees. It can be divided into CUs based on cubic / ternary / binary tree partitions to suit various local characteristics. In the multi-type tree structure, one CTU is divided according to the quadtree structure, and each quadtree link The CU can be further divided according to the binary tree and ternary tree structure. As shown, there are five compartment types: four-way compartment, horizontal dual compartment, vertical dual compartment, and water There are horizontal ternary compartments and vertical ternary compartments.

[0058] In some embodiments, video encoder 20 further divides the coding blocks of the CU by one or more can be partitioned into multiple M × N prediction blocks (PBs). A prediction block is a block of the same prediction A rectangular (square or non-square) area to which prediction (inter or intra) is applied. The prediction unit (PU) of the CU is a prediction block of one luma sample. block, two corresponding prediction blocks of chroma samples, and predicting these prediction blocks It may contain syntax elements used to represent monochrome images or images with three separate color planes. In the image, the PU has a single predicted block and the The video encoder 20 may include a syntax structure for determining the luminance prediction block of each PU of a CU. , predictive luminance blocks for Cb and Cr predicted blocks, predictive C A b-block and a predictive Cr-block can be generated.

[0059] The video encoder 20 performs intra prediction or inter prediction on the PU. Video encoder 20 can generate a prediction block from the intra-prediction. When generating a prediction block for a PU, the decoded samples of the frame associated with this PU are used. Based on the PU, a predictive block for this PU can be generated. When a predictive block of a PU is generated by inter-prediction, the predictive block of the PU is This is based on the decoded samples of one or more frames other than the frame being A predictive block of PUs can be generated.

[0060] The video encoder 20 may generate a predictive luminance block of one or more PUs of a CU. After generating the predictive Cb block and the predictive Cr block, the original luminance coding block of the CU is By subtracting the predictive luminance block of a CU from the block, the luminance residual block of this CU is calculated. where each sample in the luma residual block of this CU is the prediction of this CU. The luminance samples in one of the predictive luminance blocks and the It indicates the difference with the corresponding sample in the original luma-coded block. The decoder 20 generates a Cb residual block and a Cr residual block of the CU, respectively, where , each sample in the Cb residual block of this CU is Cb samples in one predictive Cb block and the original Cb coding block of this CU. Each sample in the Cr residual block of this CU indicates the difference between the corresponding sample in the block. The sample is in one of the predictive Cr blocks of this CU. The difference between the Cr sample in show.

[0061] Furthermore, as shown in FIG. 4C, video encoder 20 uses quadtree partitioning to divide the brightness of a CU. luminance residual block, Cb residual block, and Cr residual block are combined into one or more luminance variables. It can be expanded into a transform block, a Cb transform block, and a Cr transform block. A block is a rectangular (square or non-square) block of samples to which the same transformation is applied. The transform unit (TU) of a CU is a transform block of luma samples, a transform block of chroma samples. Two corresponding transform blocks, and the transform block used to transform the samples Therefore, each TU of a CU may contain a luma transform block, a Cb transform block, and In one example, the TU can be associated with the CR and CR transform blocks. The attached luma transform block may be a sub-block of the luma residual block of the CU. The b transform block may be a sub-block of the Cb residual block of the CU. The block can be a sub-block of the Cr residual block of the CU. For images with separate color planes, a TU consists of a single transform block and a sample of this transform block. and a syntactic structure used to transform the pull.

[0062] Video encoder 20 applies one or more transforms to the luma transform blocks of the TU. , a luma coefficient block for this TU can be generated. The coefficient block is a 2× block of transform coefficients. The transform coefficients may be scalar quantities. applies one or more transforms to the Cb transform block of a TU to obtain the Cb coefficient block of this TU. Video encoder 20 may perform one or more transforms to generate a TURN. can be applied to the Cr transform block of this TU to generate the Cr coefficient block of this TU.

[0063] The video encoder 20 may generate a coefficient block (e.g., a luminance coefficient block, a Cb coefficient block, After generating the coefficient blocks (either Cr or Cr coefficient blocks), the coefficient blocks may be quantified. Generally, it is necessary to quantify the transform coefficients and minimize the amount of data representing these transform coefficients. This means that further compression can be achieved. After quantification, the syntax elements representing the quantified transform coefficients can be entropy coded. For example, video encoder 20 may use Context-adaptive binary arithmetic coding and decoding (CABAC) may be performed. Next, video encoder 20 constructs a representation of the encoded frames and associated data. A bit stream containing the bit sequence is output and stored in the storage device 32. , or transmits it to the target device 14 .

[0064] The video decoder 30 receives the bitstream generated by the video encoder 20. After receiving the bitstream, parse this bitstream to get the syntax elements from the bitstream. Video decoder 30 may select at least one of the syntax elements obtained from the bitstream. Based on the portion, the frames of the video data can be reconstructed. The process of generating the image is generally the same as the encoding process performed by video encoder 20. For example, video decoder 30 may apply the following to coefficient blocks associated with the TUs of the current CU: It is possible to perform an inverse transform on the TUs to reconstruct the residual blocks associated with the TUs of the current CU. Also, video decoder 30 samples the prediction blocks for the PU of the current CU. The current CU is calculated by adding the corresponding sample of the transform block of the TU of the current CU to the corresponding sample of the transform block of the current CU. The coding blocks of the CU are reconstructed. The coding blocks of each CU in the frame are reconstructed. Video decoder 30 can then reconstruct this frame.

[0065] As mentioned above, there are two main modes in video coding and decoding: intra-frame and Prediction (or intra-prediction) and inter-frame prediction (or inter-prediction) are used. Palette-based coding is used in many video coding and decoding standards. Another encoding / decoding scheme that is being adopted is Screen Generated Content Code Palette-based coding and decoding, which is particularly suitable for video coding (e.g., video The encoder 20 or the video decoder 30 determines the color representing the video data for a given block. The palette table is generated by creating a palette table of the most important ( Contains (e.g., frequently used) pixel values ​​that are frequently represented in a given block of video data. Pixel values ​​that are not included are either not included in the palette table or are included as escape colors.

[0066] Each entry in the palette table is the index of the corresponding pixel value in the palette table. It encodes the palette index of the samples in the block to create a palette table. It indicates which entry in the table is used to predict or reconstruct which sample. This palette mode can be used for images, slices, tiles, or other video blocks. From the process of generating the palette predictor for the first block of such a group of As will be described later, the palette predictor for a subsequent video block typically begins with It is generated by updating a previously used palette predictor. For illustration purposes, assume The palette predictor is defined at the image level. In other words, each image has its own unique It is possible to have multiple coding blocks with different palette tables, but the entire image There is one palette predictor for

[0067] The bits required to signal palette entries in the video bitstream To reduce this, the video decoder utilizes a palette predictor to reconstruct the video block. You can then determine a new palette entry in the palette table to create an example. For example, the palette predictor uses palette entries from a previously used palette table. This most recent entry in the palette table can be used to include all entries in the palette table. It may be initialized with a recently used palette table. The child has fewer than all the entries from the Recently Used Palette Table. It may also incorporate some of the entries from other previously used pallet tables, including The palette predictor is the same as the palette table for encoding and decoding different blocks. It may be the same size, or a palette table for encoding and decoding different blocks. In one example, the palette predictor may be 64 pixels in size. The palette entry is implemented as a first-in-first-out (FIFO) table containing:

[0068] The video decoder derives a palette table for the block of video data from the palette predictor. To generate a palette, we extract this palette predictor from the encoded video bitstream. You can receive a 1-bit flag for each entry. indicates that the associated entries of the palette predictor should be included in this palette table. The first value (e.g., binary 1) that corresponds to the first entry in this palette predictor. has a second value (e.g., binary zero) indicating that it is not included in the palette table. The video decoder may determine that the size of the palette predictor is for a block of video data. If the pallet table is larger than the maximum size of the pallet table, it will Reception of the above flags may be stopped.

[0069] In one embodiment, some of the entries in the palette table use palette predictors. signaled directly in the coded video bitstream instead of being determined by the The video decoder may then generate the encoded video stream for such an entry. for the luma and two chroma components associated with this entry from the bit stream. Three separate m-bit values ​​may be received to indicate pixel values, where m is the bit depth of the video data. The number of m bits required for a palette entry is directly signaled. Compared to the value, the palette entries derived from the palette predictor only have a 1-bit flag. Therefore, you can use the palette predictor to select some or all of the palette entries. Signaling the part is required to signal a new pallet table entry. This significantly reduces the number of bits required and improves the overall coding / decoding efficiency of palette mode coding / decoding. can be improved.

[0070] In many cases, the palette predictor for a block is derived from one or more previously encoded However, the palette table for encoding the image block is used. When the first coding tree unit in an image, slice, or tile is coded, The palette table of a previously coded block may not be available. Therefore, we generate the palette predictor using previously used palette table entries. In this case, the previously used pallet table is no longer available. The sequence of palette predictor initialization values, which are values ​​for generating the palette predictor, is Sequence parameter set (SPS) and / or image parameters It may be signaled in the picture parameter set (PPS). is commonly called a coded video sequence (CVS). A syntactic structure of syntax elements that apply to a series of consecutive coded video images, each slice Syntax elements found in the PPS referenced by syntax elements found in the segment header The PPS is generally determined by the content of one or more individual images in the CVS. refers to the syntactic structure of the syntactic elements that apply to the image, and the syntactic elements found within each fragment title Therefore, SPS generally has a higher level of syntax than PPS. The syntax elements in an SPS are generally considered to be part of the syntax elements in a PPS. Compared to ,changes occur less frequently and are applied to a larger portion of the video data. This means that

[0071] 5A-5B are diagrams illustrating a configuration of a video encoding / decoding system according to an embodiment of the present disclosure. FIG. 10 is a block diagram showing an example in which a palette mode is used.

[0072] When signaling palette (PFT) mode, the palette mode is The prediction mode for the coding unit is The values ​​can be MODE_INTRA, MODE_INTER, MODE_IBC, and MODE_PLT. When using the bit mode, pixel values ​​within a CU are represented by a small set of representative colors. This set is called the palette. For pixels with values ​​close to the palette colors, An index is signaled. For pixels with values ​​outside the palette, an escape sequence is The pixel values ​​are expressed as symbols and quantified directly in the signal. describes the syntax and associated semantics of palette modes in the current VVC draft specification, respectively. This shows that...

[0073] The decoder uses the bitstream to decode blocks coded in palette mode. The palette color and index must be decoded from the stream. The palette table encoding syntax (e.g., palette_predi ctor_run, num_signaled_palette_entrients, new_palette_entrients) The escape flag palette_escape_val_present_flag indicates the current C Sent to indicate whether an escape symbol is present in U. If a palette exists, an entry is added to the palette table and the last index is The palette index of all pixels in a CU is assigned to the palette mode. and form a palette index map, and use the palette index map encoding syntax (e.g., nu m_palette_indices_minus1, palette_idx_idc, copy_above_indices_for_final_run_flag , palette_transpose_flag, copy_above_palette_indices_flag, palette_run_prefix, p An example of a CU coded in palette mode is shown in Figure 1. 5A, where the palette size is 4. The first three samples in the CU The CUs are reconstructed using palette entries 2, 0, and 3, respectively. The sample "X" represents the escape symbol. The CU-level flag palette_escape_v al_present_flag indicates whether any escape symbols are present in this CU. If an escape symbol is present, the palette size is increased by one and the last index The escape symbol is used to indicate this escape symbol. Index 4 is assigned to this escape symbol.

[0074] The palette index (e.g., index 4 in Figure 5A) is used as the escape symbol. If so, additional overhead is signaled to indicate the corresponding color for the sample. will be done.

[0075] In one embodiment, the encoder side may select the appropriate palette for use with the CU. For lossy coding, the palette needs to be derived using the improved K-means algorithm. A clustering algorithm is used. The first sample in the block is added to the palette. Then, for each subsequent sample from this block, the corresponding sample and each current The sum of absolute differences (SAD) between the palette colors is calculated. The distortion of each component is smaller than the threshold for the palette entry corresponding to the smallest SAD. If so, add the sample to the cluster that belongs to this palette entry. If a sample is mapped to a cluster, it is added as a new palette entry. When the number of samples exceeds a threshold, the centroid of this cluster is updated. This is a palette entry.

[0076] The next step is to sort the clusters in descending order of usage, and then add Update the corresponding palette entry. In general, the cluster centroids are represented as palette entries. However, the cost of encoding palette entries is low, resulting in a rate-distortion The analysis is performed if any of the entries from the palette predictor are more likely to be updated palette entries than the centroid. This process is carried out to analyze whether the product is suitable for use as a The process continues until all clusters have been processed or the maximum palette size is reached. Finally, if a cluster has only one sample and the corresponding palette entry is If the sample is not in the byte predictor, it is converted to an escape symbol. Palette entries that match are removed and their clusters are merged.

[0077] After the palette is derived, each sample in the block is assigned the closest palette (in the SAD). The sample is assigned an index for the entry. is assigned to the "COPY_ABOVE" mode. "INDEX" or "COPY_ABOVE" modes are possible. For each sample that is possible, determine the operation of each mode. Then, encode this mode. The cost of each mode is calculated, and the mode with the lowest cost is selected.

[0078] To encode the palette table, a palette predictor is maintained. The maximum size of the size and palette predictor is SPS (or PPS, slice The palette predictor may be signaled in the packet (or at other coding levels, such as the header). The palette predictor is initialized at the beginning of each slice, reset to 0. For each entry in the palette, a recursive string indicating whether this entry is part of the current palette or not is returned. The reuse flag is signaled. As shown in FIG. 5B, the reuse flag palette_predictor r_run is sent, and the num_signaled_palette_entrients syntax A number of Golomb codes are used to signal the number of new palette entries. The component values ​​of the new palette entries new_palette_entrients[] are signaled. After encoding a CU, the palette predictor is updated using the current palette. Entries from previous palette predictors that are not reused in the current set are The new palette predictor is added to the end of the palette until the size is reached.

[0079] To encode the palette index mapping, the index is The boxes are encoded by horizontal traverse scanning or vertical traverse scanning. The scanning order is , explicitly signaled in the bitstream with palette_transpose_flag.

[0080] The palette index has two main palette samples: "INDEX" and "COPY_ABOVE". In "INDEX" mode, the palette index is explicitly signaled. In "COPY_ABOVE" mode, the palette input for the samples in the upper row is The index is copied. In both "INDEX" mode and "COPY_ABOVE" mode, the same A run value is signaled indicating the number of pixels to be coded in the mode. For rows other than the first row, if the scan mode is vertical or the previous mode is "COPY_ABOVE", In this case, for columns other than the first column, this mode is signaled by a flag.

[0081] In one embodiment, the coding order of the index map is first, ces_minus1 signals the number of index values ​​of the CU, followed by the syntax palette_idx_id c signals the actual index value of the entire CU. Both the index and the associated value are coded in bypass mode. Groups the path coding bins. Then, use the syntax copy_above_palette_indices_flag, palet The te_run_prefix and palette_run_suffix determine the palette mode (INDEX or COPY_AB OVE) and runs are interleaved. The palette_run_prefix codeword is the text encoding flag (only one bin), and the palette_run_prefix codeword is 3, where the first five bins are context-coded. The tte_run_suffix is ​​encoded as a bypass bin. Finally, the escape suffix for the entire CU is The component escape values ​​corresponding to the pull are grouped and coded in bypass mode. The single syntax element copy_above_indices_for_final_run_flag copies the index values ​​to the signal. This syntax element is used in combination with an index number. This eliminates the need to signal the run value corresponding to the last run in the block. do.

[0082] In VVC (VTM), coding unit partitions for luma and chroma components are used for I slices. This allows for a binary tree that separates the luminance (Y component) and chroma (Cb, Cr components) If the binary tree is disabled, the palette is applied to Y, Applied according to the Cb and Cr components. [Table 1-1] [Table 1-2] [Table 1-3] [Table 1-4] [Table 2-1] [Table 2-2] [Table 2-3] [Table 2-4] [Table 3-1] [Table 3-2]

[0083] As a coefficient group (CG) used for transform coefficient coding, one CU is It is divided into multiple line-based coefficient sets, each consisting of m samples, and for each CG Index run for escape mode, palette index value and quantified color are coded sequentially. As a result, syntax elements such as CG index runs, palette indexes, etc. After analyzing the index value, the escape quantification color, the pixels in the row-based CG are reconstructed. can be constructed, and the syntax elements of the entire CU must be parsed (and stored) before reconstruction. This significantly reduces the buffer requirements in palette mode in VTM 6.0. can.

[0084] In this application, as shown in FIG. 5D, based on the traverse scanning mode, the pallet mode Each CU is divided into multiple segments with m samples each (m=8 in this test).

[0085] The coding order of palette run coding in each segment is as follows: For a context-coded bin, run_copy_flag=0 indicates that the pixel is the same as the previous pixel. mode, i.e., the previous scanned pixel and the current pixel are both run-type CO PY_ABOVE or the previous scanned pixel and the current pixel are both of run type INDE X and is signaled to indicate the same index value. If so, run_copy_flag=1 is signaled.

[0086] If the current pixel and the previous pixel are of different modes, the context-encoded bin copy _above_palette_indices_flag indicates whether the pixel is of run type, i.e., INDEX or COPY_ABO In this case, the sample is signaled to indicate that it is a VE. INDEX mode by default when the row is in the first column (vertical traverse scan) or the first column (vertical traverse scan). is used, so the decoder does not need to parse the run type. If the type is COPY_ABOVE, the decoder also does not need to parse the run type.

[0087] After palette run coding of the pixels in one segment, the index value (INDEX mode) is (for the context coding bin) and quantified escape colors from the encoding / analysis of the context coding bin. By encoding and grouping them as bypass bins, the throughput within each row-based CG is The index value is encoded / parsed after the run encoding, so the encoder The number of index values ​​is num_palette_indices_minus1 and the last run type is copy_above_ind There is no need to signal ices_for_final_run_flag. Table 4 shows the CG palette mode The following syntax elements are shown: [Table 4-1] [Table 4-2] [Table 4-3] [Table 4-4]

[0088] FIG. 6 illustrates a video decoder (e.g., video decoder 30) according to an embodiment of the present disclosure. 6 is a flowchart 600 illustrating an exemplary process for implementing a technique for decoding video data. be.

[0089] In the palette mode of VVC, the palette mode is applied to CUs with 64x64 pixels or less. In some embodiments, the minimum palette mode block size is the minimum palette mode block size. Disables palette mode for coding units with a size equal to or less than the palette mode block size. It is used to reduce complexity, e.g., to reduce the number of samples below a certain threshold. For all blocks of size, e.g. 16 samples, disable palette mode. Different chroma formats (e.g. 4:4:4, 4:2:2, 4:2:0) and and different coding tree types (e.g., SINGLE_TREE, DUAL_TREE_LUMA and DUAL_TREE_CHROM This threshold may vary depending on A). The saturation and chrominance components are similarly partitioned and the same palette table and palette reservation are used in palette mode. Conversely, "DUAL_TREE" means that the luminance and chrominance components of an image are shared. are individually partitioned and have different palette tables and palette predictors in palette mode. For example, in the case of the "DUAL_TREE" type, the saturation component is considered separately. For chroma components of CUs with 16 samples or less, reduce the complexity. You must disable palette mode to ensure that the following syntax is used: show. [Table 5]

[0090] Table 5 shows that pred_mode_plt_flag enables palette mode for a coding unit (e.g. Specifies whether the SubWidthC and SubHeight are enabled (e.g., value 1) or disabled (e.g., value 0). Parameters such as C are related to the chroma format of this coding unit as follows: Supported by: In monochrome sampling, there is only one sample array, nominally thought of as the luminance array. In 4:2:0 sampling, each of the two chroma arrays is half the height and width of the luma array. In 4:2:2 sampling, each of the two chroma arrays has a width equal to the luma array In 4:4:4 sampling, each of the two chroma arrays has the same height and half the width. and has the same height and width as the luminance array.

[0091] In another embodiment, for a single tree, for a CU with small luminance blocks, ,disable palette mode. In one example, for a single tree, luminance blocks of 16 pixels or less Disable palette mode for CUs with palette enabled. Since the conversion is conditional on the luma sample size regardless of the chroma size, 8x4 luma samples are used. Enable palette mode for an 8x4 CU containing two 4x2 chroma samples. Good too.

[0092] In one embodiment, for a single tree, the luma component (e.g., Y) and chroma component (e.g., For example, Cb and Cr) are partitioned in the same way. In the case of a binary tree, the luminance and chrominance components are different. In the case of a local binary tree, the luma and chroma components are represented in different patterns than in the case of a single tree. In the case of a local binary tree, the luminance and chrominance components in a CU are treated separately. Palette encoding and decoding is performed.

[0093] In another embodiment, in the case of a local binary tree, for small size blocks, the palette In one example, for local binary trees, performance is reduced for CUs with 32 pixels or less. Disable red mode.

[0094] In one embodiment, the palette mode is applied except in the case of local binary trees. In VVC, for a single tree, a luminance block larger than 4x4 pixels but smaller than 64x64 pixels is Apply palette mode to CUs with 4x4 pixels or less than 64x64 pixels in the case of binary trees. For larger CUs, the palette mode may be applied to both luma and chroma components. In another embodiment, to reduce complexity, the local binary tree is Disable the mode. Table 6 below shows an example of the syntax in a VVC design. If modeType is equal to MODE_TYPE_INTRA in VVC, this CU is The changes to VVC are as follows: [Table 6]

[0095] In another embodiment, for local binary trees, the palette mode is disabled for the chroma component only. In other words, for local binary trees, the palette mode applies to the luminance CU. However, it cannot be applied to chroma CU. Here is an example of the syntax in a design: In a VVC design, the modeType of the CU is MODE_TYPE_ The fact that the treeType of the CU is equal to INTRA and the treeType of the CU is equal to DUAL_TREE_CHROMA means that this CU is a chroma component. This means that the VVC is in the local binary tree. The changes to the VVC are shown in Table 7. [Table 7]

[0096] In one embodiment, for a local binary tree, the color palette prediction is performed using the luma and chroma components. According to the current VVC standard, for a local binary tree, The palette prediction update process is performed only on the chroma component. More specifically, for each luminance C If U is encoded and decoded using a local binary tree in palette mode, the palette prediction is updated. The palette prediction is performed by using the local binary tree for each palette mode saturation C The last chroma component of U may be updated after it is encoded or decoded.

[0097] As mentioned above, the palette prediction update process defined in VVC is In one embodiment disclosed in the present application, in order to improve the encoding / decoding efficiency, In the case of local binary trees, the palette prediction update process is performed for both luma and chroma CUs. More specifically, when encoding and decoding each luminance CU in the local binary tree, First, we update the palette prediction, and then encode each chroma CU with the same local binary tree. Table 8 below shows an example of the syntax for a VVC design. In the design, the variable cIdx specifies the color / video component of the current CU, where 0 is the luminance, 1 is the Cb component, 2 represents the Cr content. The changes to VVC are as follows: [Table 8]

[0098] In one embodiment, the palette prediction is partially updated in the case of a local binary tree. As mentioned above, in the case of a local binary tree, the update process of the palette prediction is performed for the luma and chroma values. More specifically, palette prediction is first performed for both the local and The local binary tree is updated during encoding / decoding of each luma CU, and then the same local binary tree is used to The CU may be encoded and decoded.

[0099] Suppose all CUs in the local binary tree are small size CUs. The process of updating the palette prediction sequentially requires a large number of computation cycles. According to one embodiment, in order to reduce the complexity, in the case of a local binary tree, ,The shared palette table is applied to some or all CUs without ,table update.

[0100] For example, in the case of local binary trees, the palette prediction update process is disabled in palette mode. Table 9 below shows the V with lines 450 to 456 in Table 8 deleted. An example of the syntax of a VVC design is shown below. In a VVC design, the variable cIdx represents the color component of the current CU. Specify the minutes, 0 represents brightness, 1 represents Cb component, and 2 represents Cr component. The VVC changes are shown below. . [Table 9]

[0101] In one embodiment, for a local binary tree, the luminance block size is 32 pixels or less. Disable the palette prediction update process in palette mode for the CU. ,The palette prediction update process in palette mode requires at least 8x8 luma samples. It may be enabled for 8x8 or larger CUs, including

[0102] In yet another embodiment, for local binary trees, palette prediction is performed only for chroma CUs. Table 10 below shows an example of the syntax for a VVC design. In VC design, the variable cIdx specifies the color component of the current CU, where 0 is the luminance, 1 is the Cb component, and 2 is the The change in VVC is shown below. [Table 10]

[0103] In one embodiment, for a local binary tree, the palette modes for the different video components are Separately update the palette prediction update process in

[0104] As mentioned above, in the case of a local binary tree, the update process for the palette prediction is performed sequentially. This also means that the decoding of chroma CUs in palette mode with a local binary tree is performed using the same This means that the decode cannot start until all the luminance CUs in the local binary tree have been decoded. This can introduce unwanted delays in hardware codec execution.

[0105] To solve the above problem, according to one embodiment, the palette prediction update process comprises: It is performed separately for different video components (e.g., luma and chroma) in the local binary tree. Therefore, palette mode encoding and decoding of the chroma component can be performed in parallel with the luma component. In one example of this embodiment, in the local binary tree, the pallet at the head of the local binary tree is Uses the same starting palette for both the luma and chroma CUs.

[0106] In one embodiment, the process of updating the palette prediction in the case of local binary trees is improved. In the case of local binary trees, the palette prediction update process is performed for the luma and chroma components. More specifically, palette prediction is performed by first estimating each The luminance CU is updated during encoding and decoding, and then each chroma CU is updated in the same local binary tree. As a result, while encoding and decoding the luminance CU in the local binary tree, When updating the palette prediction, saturation information for collocated pixels may not be available. , and vice versa.

[0107] In one embodiment, to improve the encoding / decoding efficiency, a local binary tree is used to encode one video. Palette prediction while encoding and decoding CUs of the color components (e.g., luma and / or chroma) When updating the palette, other video components (e.g., saturation and In one example case of a local binary tree, the luminance component During the palette prediction update process, the saturation component of the first available candidate is added. It can be used as the saturation component of a palette entry. Table 11 below lists the VVC settings. An example of the syntax in the design is shown below. The changes in VVC are shown below. [Table 11]

[0108] In one embodiment, as shown in FIG. 6, video decoder 30 converts the bitstream Associated with a coding unit is the type of coding tree of this coding unit and the local binary tree model. Receives several syntax elements that indicate whether the code is valid for this coding unit (61 0).

[0109] Video decoder 30 determines whether the coding tree type of this coding unit is a single tree and whether the coding tree type is a low-level tree. Upon determining that the binary tree mode is valid for this coding unit (6 20), if the coding unit has a size equal to or smaller than a predetermined threshold, Disable palette mode for the encoding unit (630).

[0110] In one embodiment, palette mode is disabled for this coding unit (630). This means that the palette mode is set for both the luma and chroma components of this coding unit. Including disabling (640).

[0111] In one embodiment, disabling palette mode for the coding unit (630 ) disables palette mode for only the chroma component of this coding unit (6 50).

[0112] In some embodiments, palette mode is disabled for only the chroma component of a coding unit. This means that the video component of this coding unit is determined from the multiple syntax elements as either a chroma component or a brightness component. and determining whether the video component of this coding unit is a chroma component. In response to determining that the chroma component of this coding unit is Including disabling.

[0113] In some embodiments, palette mode is disabled for only the chroma component of a coding unit. This means that the video component of this coding unit is determined from the multiple syntax elements as either a chroma component or a brightness component. and determining whether the video component of this coding unit is a luminance component. In response to determining that the coding unit is a luminance component, Receives the palette mode enable flag associated with this palette mode enable. The coding unit is decoded according to the table flag.

[0114] In one embodiment, video decoder 30 performs palette prediction from the bitstream described above. receive an enable syntax and enable this coding unit according to the palette prediction enable syntax. Update the palette prediction of the luminance component of the input image.

[0115] In some embodiments, video decoder 30 may select a palette of the luminance component of this coding unit. Prevents forecast updates.

[0116] In one embodiment, video decoder 30 may define a minimum palette mode luma block size as The size of the luminance component of the coding unit is determined to be the minimum palette mode luminance block size. In response to determining that the size of the coding unit is equal to or smaller than the predetermined size, a palette prediction of the luminance component of the coding unit is performed. Prevent updates.

[0117] In one embodiment, the minimum palette mode luma block size is 32x32 luma samples. It is.

[0118] In one embodiment, the minimum palette mode luma block size is 8x8 luma samples. be.

[0119] In one embodiment, a local binary tree mode is included in the above syntax elements. The syntax element that indicates whether MODE_TYPE_INTRA is valid for the quantization unit is MODE_TYPE_INTRA.

[0120] In some embodiments, the video components of a coding unit are included in a plurality of syntax elements, and the video components of the coding unit are chroma components. The syntax element that indicates whether the color component is a chrominance component or a luminance component is DUAL_TREE_CHROMA.

[0121] In one embodiment, the predetermined threshold in step 630 is 32x32 samples. It is.

[0122] In one embodiment, the predetermined threshold in step 630 is a 16x16 sample. It is.

[0123] FIG. 7 illustrates an exemplary context-adaptive binary arithmetic code according to an embodiment of the present disclosure. FIG. 1 is a block diagram illustrating an example of a CABAV engine.

[0124] Context-adaptive binary arithmetic coding and decoding (CABAC) is a G-4 AVC, High Efficiency Video Coding (HEV Entropy codes are used in many video coding and decoding standards such as C) and VVC. CABAC is a video coding and decoding format based on arithmetic coding and decoding. There have been some innovations and updates to address the needs of coding standards, e.g., CABAC keeps the complexity low and uses probability models of more frequently used bits in any symbol. Encodes binary symbols to enable decoding. ,is locally well correlated with,,so that the probabilistic model better models the probability, It is adaptively selected based on the local context. Finally, CABAC is a quantified Use multiplication-free range division with the given probability range and probability state.

[0125] CABAC has multiple probability models for different contexts. Convert all non-binary symbols into binary symbols. Then, the coder For each pixel (called a bit), we select which probability model to use, and then we select the neighboring pixels. Use information from the elements to optimize the probability estimates. Finally, apply arithmetic coding and decoding to Compress the data.

[0126] Context modeling provides estimates of the conditional probabilities of the coding symbols. The context model allows us to determine whether there are already coded symbols in the vicinity of the current symbol being coded. By switching between different probability models depending on the decoded symbols, a given Inter-symbol redundancy can be utilized. It includes the following steps:

[0127] Binarization: CABAC uses binary arithmetic coding and decoding, which is a process of dividing binary decisions (1 This means that only non-binary value symbols (0 or 1) are coded. The image data (e.g., transform coefficients or motion vectors) are "binarized" or converted to binary code. This process is similar to the process of converting data symbols into variable length codes. However, the binary code is further coded (by an arithmetic coder) before transmission. Each step is repeated for each bin (or "bit") of the resulting symbol.

[0128] Context Model Selection: A "context model" is one of the binary symbols. is a probability model for multiple bins. This model is based on the most recently coded data symbols. Based on the statistics of the context, a choice can be made from a selection of available models. The model stores the probability that each bin is "1" or "0."

[0129] Arithmetic coding: The arithmetic coder encodes each bin based on a selected probability model. ,For each bin, there are only two subranges (corresponding to '0' and '1').

[0130] Probability Update: Update the selected context model based on the actual encoded values (For example, a bin value of "1" increases the frequency count of "1").

[0131] Each non-binary syntax element value is expanded into a bin sequence, and each bin value in CABAC is Further processing can be performed in either normal or bypass mode. The encoding / decoding mode decision depends on the latter mode, which is assumed to be uniformly distributed. bins, resulting in the entire normal binary arithmetic coding (and decoding) process being It is selected to be purely binary. In normal encoding / decoding mode, normal binary Encode each bin value with an arithmetic coding / decoding engine, where the associated probability model is: The type of syntax element and the bin position or bin index in the binarized representation of the syntax element It is either determined as a fixed selection based on the binIdx or based on the associated side information ( For example, the spatial neighborhood, component, depth or size of a CU / PU / TU, or within a TU The probability model is adaptively selected from two or more models according to the position. As a key design decision, the latter mode is generally the most The other mode applies only to the bins that are of most interest, while the other mode applies only to the bins that are of least interest. The smaller bins are typically processed by a zero-order joint probability model. This allows CABAC to: It allows selective adaptive probability modeling at the sub-symbol level, thus improving the overall model Effective methods for exploiting inter-symbol redundancy, significantly reducing coding or training costs. Note that in both the fixed and adaptive cases, in principle, one probabilistic model The switching from one probability model to another is Generally, the design of the context model in CABAC avoids unnecessary overhead in modeling costs and takes advantage of statistical dependencies This reflects the aim of finding a good trade-off between these two conflicting goals. do.

[0132] The parameters of the probability model in CABAC are adaptive, which is based on the statistical Adaptation of model probabilities to variations is backward-adaptive and This means that the process is performed synchronously in each bin. This process is called probability estimation. Therefore, each probability model in CABAC is a correlated model ranging in the interval [0:01875;0:98125]. One of 126 different states can be selected with a probability value p. The two parameters are used to model the least probable symbol (LPS). 6 bits for each of the 63 probability states representing the LPS and the highest possible symbol ( A 7-bit encoding containing one bit for the value nMPS (the most probable symbol) The context memory stores the data as a seq.

[0133] In one or more examples, the functionality described above may be implemented in hardware, software, firmware, or other similar devices. If implemented in software, Those functions may be implemented as one or more instructions or code on a computer-readable medium. stored in or transmitted through the body and executed by a hardware processing unit. A computer-readable medium is a computer program that corresponds to a tangible medium such as a data storage medium. A computer readable storage medium or a computer readable storage medium, for example, according to a communication protocol includes communication media, including any medium that facilitates the transfer of a computer program from one location to another Thus, computer-readable media generally include: (1) a non-uniform (2) any communication medium, such as a signal or carrier wave, that is temporarily or tangibly computer-readable; A data storage medium may correspond to one or more computer and accessed by one or more processors to implement the implementations described herein. Any user who can find instructions, code, and / or data structures to implement a state The computer program product may be any available medium. It may also include a possible medium.

[0134] The terms used to describe embodiments herein are intended to describe specific embodiments. The following statements are for illustrative purposes only and are not intended to limit the scope of the claims. As used in the description of embodiments and the appended claims, the singular forms "one," "one," "an ... and "this" is intended to include the plural forms unless the context clearly indicates otherwise. As used herein, the term "and / or" refers to one or more It should also be understood to mean and include any and all possible combinations of the listed items. As used herein, the term "comprising" means including the stated features, elements, and / or or component, but does not include one or more other features, elements, components, and / or It should further be understood that the expression "a" does not preclude the existence or addition of such sets. .

[0135] The use of terms such as first and second to describe various elements herein means that these elements It should also be understood that the present invention is not limited by these terms. It is used only to distinguish one element from another, e.g., to depart from the scope of the embodiment. Unless otherwise specified, a first electrode may be referred to as a second electrode, and similarly, a second electrode may be referred to as a first electrode. The first electrode and the second electrode are both electrodes, but they are the same electrode. There is no.

[0136] The present description has been presented for purposes of illustration and description and is not intended to be exhaustive or to be construed as limiting the scope of the invention. The invention is not limited to the above embodiments. Various modifications, variations, and alternative implementations are possible. It will be apparent to one skilled in the art having the benefit of the teachings presented in the description and the associated drawings. The embodiments best explain the principles and practical applications of the present invention, and allow those skilled in the art to easily understand the present invention for various implementations. Understand the principles and various implementations with various modifications to suit specific applications. They have been selected and described to make them best available. The claims are not limited to the specific examples of implementations disclosed, but also encompass modifications and other implementations. , should be understood to fall within the scope of the appended claims.

Claims

1. From the bitstream, a tag of the coding tree of the coding unit is obtained in association with the coding unit. Indicates whether local binary tree mode is enabled for the coding unit. receiving a plurality of syntax elements; The type of the coding tree of the coding unit is a single tree and the local binary tree mode is is valid for the encoding unit, If the coding unit has a size equal to or smaller than a predetermined threshold, the coding unit is Disable palette mode for the 1. A method for decoding video data, comprising:

2. Disabling palette mode for the coding unit includes: Disable the palette mode for both the luma and chroma components of the coding unit. To make effective The method of claim 1 , comprising:

3. Disabling palette mode for the coding unit includes: Disabling the palette mode for only the chroma component of the coding unit. The method of claim 1 , comprising:

4. Disabling the palette mode for only the chroma component of the coding unit may include: The plurality of syntax elements determine whether the video component of the coding unit is a chroma component or a luma component. and further determining whether the In response to determining that the video component of the coding unit is a chroma component, Disabling the palette mode for the chroma component of the coding unit; The method of claim 3, comprising:

5. Disabling the palette mode for only the chroma component of the coding unit may include: The plurality of syntax elements determine whether the video component of the coding unit is a chroma component or a luma component. and further determining whether the In response to determining that the video component of the coding unit is a luminance component, A palette associated with the luminance component of the coding unit from the bitstream is extracted. receive the mode enable flag, decoding the coding unit according to the palette mode enable flag; and The method of claim 3, comprising:

6. receiving a palette prediction enable syntax from the bitstream; A palette prediction of the luminance component of the coding unit is performed according to the palette prediction enable syntax. Update your forecast and The method of claim 5 , comprising:

7. inhibiting updating of palette predictions of the luminance component of the coding unit; The method of claim 5 further comprising:

8. determining a minimum palette mode luminance block size; The size of the luminance component of the coding unit is the minimum palette mode luminance block size In response to a determination that: inhibiting updating of a palette prediction of a luminance component of the coding unit; The method of claim 5 further comprising:

9. the minimum palette mode luma block size is 32x32 luma samples; The method of claim 8.

10. the minimum palette mode luma block size is 8x8 luma samples; The method of claim 8.

11. a local binary tree mode included in the plurality of syntax elements and enabled for the coding unit; The syntax element that indicates whether the The method of claim 1.

12. The plurality of syntax elements include a plurality of syntax elements each including a video component of the coding unit, the video component being a chroma component or a brightness component. The syntax element that indicates whether the component is DUAL_TREE_CHROMA The method of claim 4.

13. The predetermined threshold is 32 x 32 samples. The method of claim 1.

14. the predetermined threshold is 16x16 samples; The method of claim 1.

15. 1. An electronic device comprising: one or more processing units; a memory coupled to the one or more processing units; a plurality of programs stored in the memory; Including, The plurality of programs, when executed by the one or more processing units, causing the electronic device to carry out the method according to any one of claims 1 to 14; electronic equipment.

16. A plurality of programs executed by an electronic device having one or more processing units A non-transitory computer-readable storage medium storing: The plurality of programs, when executed by the one or more processing units, The electronic device is caused to carry out the method according to any one of claims 1 to 14. A non-transitory computer-readable storage medium.

Images