Video encoding method and apparatus using adaptive chroma space conversion

Through adaptive chromaticity space conversion, the chromaticity channel in video encoding is processed, and the best conversion method is selected to improve the encoding efficiency and quality of the chromaticity channel, which solves the problem of failure to fully utilize the chromaticity channel correlation in the prior art. More efficient video encoding and decoding is achieved.

CN118947123APending Publication Date: 2024-11-12HYUNDAI MOTOR CO LTD +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380024788.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-02-17
Filing Date
2023-02-22
Publication Date
2024-11-12

Smart Images

  • Figure CN118947123A_ABST
    Figure CN118947123A_ABST
Patent Text Reader

Abstract

Disclosed are a video encoding method and apparatus using adaptive chroma space conversion, and the present embodiment provides a video encoding method and apparatus in which the chroma channel components are encoded on the basis of correlations between the chroma channel components. An optimal method is selected from various chroma spatial conversion methods for an original signal, a predictor or a residual signal of a current chroma block, and then chroma spatial conversion of Cb and Cr components is performed using the selected method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to video encoding methods and apparatus using adaptive chroma space conversion. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] When encoding video in blocks, the encoder first performs prediction to generate a predictor, and then subtracts the predictor from the original signal to generate a residual signal. The residual signal is then transformed into a frequency domain signal by using a transform technique that concentrates the energy within the block into low-frequency regions, making the converted residual signal easier to encode. The encoder selects and uses an appropriate transform technique for the residual signal (e.g., discrete cosine transform (DCT) or discrete sine transform (DST)) to encode the target block, thereby passing information about the selected transform technique to the decoder.

[0004] According to the high-efficiency video coding (HEVC) encoding technology, the residual signal of the luminance (Y) channel is converted into a frequency signal by a discrete cosine transform II (DCT-II) transform usually applied in the horizontal and vertical directions. When the target is a 4×4 block, it is subjected to a discrete sine transform VII (DST-VII) transform or a transform skip mode, in which no transform is performed on the residual signal. However, the development of image compression technology has led to the development of various methods for generating predictors, which can actually generate residual signals with various characteristics. The current prior art general video coding (VVC) technology incorporates new transforms such as discrete cosine transform VIII (DCT-VIII), which allows more diversified transforms to be applied to the residual signal. In addition, the DST-VII transform and transform skip mode previously applied exclusively to 4×4 blocks are now applied to other block sizes.

[0005] At the same time, the residual signal in the chrominance channel of the video is usually transformed separately for the two components (Cb and Cr components) of the channel. For example, the encoder can apply DCT-II transform to the residual signal of each chrominance component in the horizontal and vertical directions, or can apply the transform skip mode to not perform the transform. The encoder passes information about the method selected for each component to the decoder. In addition to the conventional method described above, the transform can be performed by applying the joint coding (JCCR) technology for chrominance residuals to the chrominance channel. The JCCR technology utilizes the fact that the residual signals of the Cb and Cr components are inversely correlated (sign inversion) with each other. That is, the encoder combines the residual signals of the Cb and Cr components into one and transmits the combined residual signal, and then the decoder reconstructs the residual signals of the Cb and Cr components from the transmitted single residual signal. Regarding the combined single residual signal, the combined information can be embedded in the syntax of the single component for transmission.

[0006] However, techniques such as those described above have the disadvantage of not being able to fully utilize the correlation existing between chrominance channels. For example, although the correlation between the Cb component and the Cr component is not as great as the correlation between the original RGB channels, there is still a significant degree of correlation between the Cb component and the Cr component. Applying a separate transform to each of the Cb component and the Cr component has the inefficiency of not being able to further utilize the significant correlation retained between the Cb component and the Cr component. Although the JCCR technique can reduce this inefficiency to some extent, the JCCR suffers from the technical limitation of significantly simplifying the correlation between the Cb component and the Cr component, and it cannot adapt to all correlations between chrominance channels, such as existing in various forms in real-world images. These shortcomings lead to problems such as reduced decoding efficiency and quality degradation of decoded video. Therefore, it is necessary to consider how to effectively encode / decode chrominance channels to enhance video quality and improve coding efficiency. Summary of the invention

[0007] [Technical issues]

[0008] The present disclosure attempts to provide a video encoding method and apparatus for selecting an optimal method from various chroma space conversion methods for an original signal, a predictor or a residual signal for a current chroma block based on correlation between chroma channel components, so as to enhance video quality and improve video encoding efficiency. The video encoding method and apparatus use the selected method to perform chroma space conversion of Cb and Cr components.

[0009] [Technical solution]

[0010] At least one aspect of the present invention provides a method for inversely converting a current chroma block performed by a video decoding device. The method includes obtaining two converted signals generated by a video encoding device through chroma space conversion. The method also includes obtaining inverse conversion information corresponding to the conversion information for the chroma space conversion. The method also includes generating signals in two chroma channels of the current chroma block from the two converted signals by applying an inverse chroma space conversion based on the inverse conversion information.

[0011] Another aspect of the present invention provides a method for converting a current chroma block performed by a video encoding device. The method includes obtaining signals in two chroma channels of the current chroma block. The method also includes determining conversion information. The method also includes generating two converted signals from the signals in the two chroma channels by applying a chroma space conversion based on the conversion information. The method also includes encoding the conversion information.

[0012] Another aspect of the present invention provides a computer-readable recording medium storing a bitstream generated by a video encoding method. The video encoding method includes: obtaining signals in two chrominance channels of a current chrominance block. The video encoding method also includes determining conversion information. The video encoding method also includes generating two converted signals from the signals in the two chrominance channels by applying a chrominance space conversion based on the conversion information. The video encoding method also includes encoding the conversion information.

[0013] [Beneficial Effects]

[0014] As described above, the present disclosure provides a video encoding method and device for selecting the best method from various chroma space conversion methods for the original signal, predictor or residual signal of the current chroma block based on the correlation between chroma channel components. The video encoding method and device use the selected method to perform chroma space conversion of Cb and Cr components. Thus, the video encoding method and device improve video quality and improve video encoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram of a point cloud and mesh encoding device according to at least one embodiment of the present disclosure.

[0016] Figure 2 is a diagram illustrating a form of a texture video and a reference structure of a reference mesh texture map according to at least one embodiment of the present disclosure.

[0017] FIG. 3 is a diagram illustrating a form of a texture video and a reference structure of an attribute image of a reference point cloud according to at least one embodiment of the present disclosure.

[0018] Figure 4 is a block diagram illustrating a point cloud and mesh decoding device according to at least one embodiment of the present disclosure.

[0019] Figure 5 is a block diagram illustrating a point cloud and mesh encoding device according to another embodiment of the present disclosure.

[0020] Figure 6 is a diagram illustrating the process of encoding chrominance components.

[0021] Figure 7 is a diagram showing the residual signal in the chroma channel after applying JCCR.

[0022] Figure 8 and Fig. 9 is a diagram illustrating a video encoding device that applies chroma space conversion and a video decoding device that applies inverse chroma space conversion according to some embodiments of the present disclosure.

[0023] Fig.10 and Fig.11 2 is a diagram illustrating a video encoding device applying chroma space conversion and a video decoding device applying inverse chroma space conversion according to another embodiment of the present disclosure.

[0024] Fig.12 and Fig.13 is a diagram illustrating a video encoding device applying chroma space conversion and a video decoding device applying inverse chroma space conversion according to still another embodiment of the present disclosure.

[0025] Fig.14a and Fig.14b is a diagram illustrating chroma space conversion (CSC) and inverse chroma space conversion (ICSC) according to some embodiments of the present disclosure.

[0026] Fig.15a and Fig.15b is a diagram illustrating a CSC and an ICSC according to other embodiments of the present disclosure.

[0027] Fig.16 is a flowchart of a method performed by a video encoding device for converting a current chroma block according to at least one embodiment of the present disclosure.

[0028] Fig.17 is a flowchart of a method for inversely converting a current chrominance block performed by a video decoding device according to at least one embodiment of the present invention. DETAILED DESCRIPTION

[0029] Hereinafter, some embodiments of the present disclosure are described in detail with reference to the accompanying drawings. In the following description, the same reference numerals represent the same elements, although the elements are shown in different drawings. In addition, in the following description of some embodiments, for the purpose of clarity and brevity, detailed descriptions of related known components and functions may be omitted when it is considered to obscure the subject matter of the present disclosure.

[0030] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present disclosure. Figure 1 A diagram of a video encoding device and components of the device is described.

[0031] The encoding apparatus may include a picture separator 110 , a predictor 120 , a subtractor 130 , a transformer 140 , a quantizer 145 , a rearrangement unit 150 , an entropy encoder 155 , an inverse quantizer 160 , an inverse transformer 165 , an adder 170 , a loop filter unit 180 , and a memory 190 .

[0032] Each component of the encoding device can be implemented as hardware or software or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0033] A video is composed of one or more sequences including multiple pictures. Each picture is divided into multiple regions, and encoding is performed on each region. For example, a picture is split into one or more tiles or / and slices. Here, one or more tiles can be defined as tile groups. Each tile or / and slice is split into one or more coding tree units (CTUs). In addition, each CTU is split into one or more coding units (CUs) through a tree structure. The information applied to each coding unit (CU) is encoded as the syntax of the CU, and the information commonly applied to the CU contained in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting one or more pictures is encoded as a picture parameter set (PPS) or a picture header. In addition, the information commonly referenced by multiple pictures is encoded into a sequence parameter set (SPS). In addition, the information commonly referenced by one or more SPSs is encoded into a video parameter set (VPS). Further, the information commonly applied to a tile or tile group can also be encoded as the syntax of the tile or tile group header. The syntax included in the SPS, PPS, slice header, tile, or tile group header may be referred to as a high-level syntax.

[0034] The picture partitioner 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or a PPS and delivered to a video decoding apparatus.

[0035] The picture divider 110 divides each picture constituting a video into a plurality of coding tree units (CTUs) having a predetermined size, and then recursively divides the CTUs by using a tree structure. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of encoding.

[0036] The tree structure may be a quadtree (QT), in which a higher node (or parent node) is divided into four lower nodes (or child nodes) of the same size. The tree structure may also be a binary tree (BT), in which a higher node is divided into two lower nodes. The tree structure may also be a ternary tree (TT), in which a higher node is divided into three lower nodes at a ratio of 1:2:1. The tree structure may also be a structure in which two or more of the QT structure, the BT structure, and the TT structure are mixed. For example, a quadtree plus a binary tree (QTBT) structure may be used, or a quadtree plus a binary tree ternary tree (QTBTTT) structure may be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to be referred to as a multi-type tree (MTT).

[0037] Figure 2 is a simplified diagram for describing a method of partitioning a block by using a QTBTTT structure.

[0038] like Figure 2 As shown, the CTU can be first split into a QT structure. The quadtree partitioning can be recursive until the size of the partitioned block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four nodes of the lower layer is encoded by the entropy encoder 155 and transmitted to the video decoding device. When the leaf node of QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node can be further divided into at least one of the BT structure or the TT structure. There may be multiple partitioning directions in the BT structure and / or the TT structure. For example, there may be two directions, namely, the direction in which the blocks of the corresponding node are divided horizontally and the direction in which the blocks of the corresponding node are divided vertically. As Figure 2 As shown, when MTT splitting starts, a second flag (mtt_split_flag) indicating whether a node is split and a flag further indicating the split direction (vertical or horizontal) and / or a flag indicating the split type (binary or ternary) if the node is split are encoded by the entropy encoder 155 and notified to the video decoding device with a signal.

[0039] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU as a basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding device first starts encoding the first flag through the above scheme.

[0040] When QTBT is used as another example of a tree structure, there may be two types, namely, a type in which the block of the corresponding node is horizontally divided into two blocks of the same size (i.e., symmetrical horizontal division) and a type in which the block of the corresponding node is vertically divided into two blocks of the same size (i.e., symmetrical vertical division). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of the lower layer and split type information indicating the split type are encoded by the entropy encoder 155 and delivered to the video decoding device. At the same time, a type in which the block of the corresponding node is split into two blocks that are asymmetric to each other may be presented additionally. The asymmetric form may include a form in which the block of the corresponding node is divided into two rectangular blocks having a size ratio of 1:3, or may also include a form in which the block of the corresponding node is divided in a diagonal direction.

[0041] A CU may have different sizes depending on the QTBT or QTBTTT split from a CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of a QTBTTT) is referred to as a "current block". Due to the QTBTTT split, the shape of the current block may be a rectangular shape in addition to a square shape.

[0042] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0043] In general, each of the current blocks in a picture may be predictively encoded. Typically, the prediction of the current block may be performed using intra-prediction techniques (using data from the picture including the current block) or inter-prediction techniques (using data from a picture coded before the picture including the current block). Inter-prediction includes both unidirectional prediction and bidirectional prediction.

[0044] The intra-frame predictor 122 predicts the pixels in the current block by using the pixels (reference pixels) located on the neighbors of the current block in the current picture including the current block. According to the prediction direction, there are multiple intra-frame prediction modes. For example, as shown in FIG. 3A, multiple intra-frame prediction modes may include 2 non-directional modes including a plane mode and a DC mode, and may include 65 directional modes. Neighboring pixels and the arithmetic equations to be used are defined differently according to each prediction mode.

[0045] In order to perform effective directional prediction on the current block having a rectangular shape, a directional mode (#67 to #80, intra prediction mode #-1 to #-14) as illustrated by the dotted arrows in FIG. 3B may be additionally used. The directional mode may be referred to as a "wide-angle intra prediction mode". In FIG. 3B, the arrow indicates the corresponding reference sample used for prediction and does not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to a specific directional mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes that can be used for the current block may be determined by the ratio of the width and height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height less than the width, a wide-angle intra prediction mode (intra prediction mode #67 to #80) with an angle less than 45 degrees is available. When the current block has a rectangular shape with a width greater than the height, a wide-angle intra prediction mode with an angle greater than -135 degrees may be used.

[0046] The intra-frame predictor 122 may determine an intra-frame prediction to be used for encoding the current block. In some embodiments, the intra-frame predictor 122 may encode the current block by using a plurality of intra-frame prediction modes, and may further select an appropriate intra-frame prediction mode to be used from a test mode. For example, the intra-frame predictor 122 may calculate a rate-distortion value by using a rate-distortion analysis for a plurality of tested intra-frame prediction modes, and may further select an intra-frame prediction mode having the best rate-distortion characteristics in the test mode.

[0047] The intra-frame predictor 122 selects an intra-frame prediction mode from among a plurality of intra-frame prediction modes and predicts the current block by using adjacent pixels (reference pixels) and an arithmetic equation determined according to the selected intra-frame prediction mode. Information about the selected intra-frame prediction mode is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0048] The inter-frame predictor 124 generates a prediction block for the current block by using a motion compensation process. The inter-frame predictor 124 searches for a block most similar to the current block in a reference picture that is encoded and decoded earlier than the current picture, and generates a prediction block for the current block by using the searched block. In addition, a motion vector (MV) is generated, which corresponds to the displacement between the current block in the current picture and the prediction block in the reference picture. Typically, motion estimation is performed on the luminance component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. Motion information including information about the reference picture and information about the motion vector for predicting the current block is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0049] The inter-frame predictor 124 may also perform interpolation on a reference picture or a reference block in order to increase the accuracy of the prediction. In other words, the sub-samples between two consecutive integer samples are interpolated by applying the filter coefficients to a plurality of consecutive integer samples including two integer samples. When the process of searching for the block most similar to the current block is performed for the interpolated reference picture, the decimal unit precision may be used instead of the integer sample unit precision for the motion vector representation. The precision or resolution of the motion vector may be set differently for each target region to be encoded (e.g., units such as slices, tiles, CTUs, CUs, etc.). When this adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution may be information representing the precision of the motion vector difference to be described below.

[0050] Meanwhile, the inter-frame predictor 124 may perform inter-frame prediction by using bidirectional prediction. In the case of bidirectional prediction, two reference pictures and two motion vectors representing the block position most similar to the current block in each reference picture are used. The inter-frame predictor 124 selects a first reference picture and a second reference picture from a reference picture list 0 (RefPicList0) and a reference picture list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for a block most similar to the current block in the corresponding reference picture to generate a first reference block and a second reference block. In addition, a prediction block of the current block is generated by averaging or weighted averaging the first reference block and the second reference block. In addition, motion information including information about two reference pictures for predicting the current block and including information about two motion vectors is delivered to the entropy encoder 155. Here, the reference picture list 0 may be composed of pictures before the current picture in the display order among the pre-reconstructed pictures, and the reference picture list 1 may be composed of pictures after the current picture in the display order among the pre-reconstructed pictures. However, although not particularly limited thereto, a pre-reconstructed picture after the current picture in the display order may be additionally included in the reference picture list 0. Conversely, a pre-reconstructed picture before the current picture may also be additionally included in the reference picture list 1.

[0051] In order to minimize the number of bits consumed for encoding motion information, various methods may be used.

[0052] For example, when the reference picture and motion vector of the current block are the same as those of the adjacent block, information capable of identifying the adjacent block is encoded to deliver the motion information of the current block to the video decoding device. This method is called merge mode.

[0053] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as “merge candidates”) from the neighboring blocks of the current block.

[0054] As a neighboring block for deriving merge candidates, such as Figure 4 As illustrated in FIG, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1, and the upper left block B2 adjacent to the current block in the current picture may be used. Further, blocks other than the current picture where the current block is located, which are located within a reference picture (which may be the same as or different from the reference picture used to predict the current block), may also be used as merge candidates. For example, a block located at the same location as the current block or a block adjacent to the block located at the same location in the reference picture may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a zero vector is added to the merge candidate.

[0055] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by using adjacent blocks. A merge candidate to be used as motion information of the current block is selected from the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0056] Merge skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy coding are close to zero, only the neighbor block selection information is transmitted without the residual signal. By using merge skip mode, relatively high coding efficiency can be achieved for pictures with slight motion, still pictures, screen content pictures, etc.

[0057] Hereinafter, the merge mode and the merge skip mode are collectively referred to as the merge / skip mode.

[0058] Another method for encoding motion information is the Advanced Motion Vector Prediction (AMVP) mode.

[0059] In the AMVP mode, the inter-frame predictor 124 derives a motion vector predictor candidate for a motion vector of a current block by using neighboring blocks of the current block. As neighboring blocks for deriving motion vector predictor candidates, the same as in Figure 4All or some of the left block A0, lower left block A1, upper block B0, upper right block B1 and upper left block B2 adjacent to the current block in the current picture illustrated in the figure. In addition, blocks other than the current picture where the current block is located, which are located in a reference picture (which may be the same as or different from the reference picture used to predict the current block), may also be used as adjacent blocks for deriving motion vector prediction sub-candidates. For example, blocks co-located with the current block or blocks adjacent to co-located blocks within the reference picture may be used. If the number of motion vector candidates selected by the above method is less than the preset number, a zero vector is added to the motion vector candidates.

[0060] The inter predictor 124 derives a motion vector predictor candidate by using the motion vectors of the neighboring blocks, and determines a motion vector predictor for the motion vector of the current block by using the motion vector predictor candidate. In addition, a motion vector difference is calculated by subtracting the motion vector predictor from the motion vector of the current block.

[0061] The motion vector predictor can be obtained by applying a predefined function (e.g., center value and average value calculation, etc.) to the motion vector predictor candidate. In this case, the video decoding device also knows the predefined function. In addition, since the neighboring blocks used to derive the motion vector predictor candidate are blocks for which encoding and decoding have been completed, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode the information for identifying the motion vector predictor candidate. Thus, in this case, information about the motion vector difference and information about the reference picture used to predict the current block are encoded.

[0062] Meanwhile, the motion vector predictor may also be determined by selecting any one of the motion vector predictor candidates. In this case, information for identifying the selected motion vector predictor candidate is additionally encoded jointly with information about the motion vector difference and information about the reference picture used to predict the current block.

[0063] The subtractor 130 generates a residual block by subtracting a prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0064] The transformer 140 converts the residual signal in the residual block having the pixel value in the spatial domain into a transform coefficient in the frequency domain. The transformer 140 may transform the residual signal in the residual block by using the total size of the residual block as a transform unit, or may also divide the residual block into a plurality of sub-blocks, and may perform the transform by using the sub-block as a transform unit. Optionally, the residual block is divided into two sub-blocks, namely a transform region and a non-transform region, so as to transform the residual signal using only the transform region sub-block as a transform unit. Here, the transform region sub-block may be one of two rectangular blocks having a size ratio of 1:1 based on the horizontal axis (or vertical axis). In this case, the flag (cu_sbt_flag) indicates that only the sub-block is transformed, and the directional (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or the position information (cu_sbt_pos_flag) are encoded by the entropy encoder 155 and signaled to the video decoding device. In addition, the size of the transform region sub-block may have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) of dividing the corresponding division is additionally encoded by the entropy encoder 155 and notified to the video decoding device by signal.

[0065] Meanwhile, the transformer 140 may perform transformation on the residual block separately in the horizontal direction and the vertical direction. For the transformation, different types of transformation functions or transformation matrices may be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation may be defined as a multiple transformation set (MTS). The transformer 140 may select a transformation function pair with the highest transformation efficiency in the MTS, and may transform the residual block in each of the horizontal and vertical directions. Information (mts_idx) of the transformation function pair in the MTS is encoded by the entropy encoder 155 and notified to the video decoding device with a signal.

[0066] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using the quantization parameter and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 can also immediately quantize the relevant residual block without the need for transforming any block or frame. The quantizer 145 can also apply different quantization coefficients (scaling values) according to the position of the transform coefficient in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in 2 dimensions can be encoded and notified to the video decoding device with a signal.

[0067] The rearrangement unit 150 may perform rearrangement of coefficient values ​​for the quantized residual value.

[0068] The rearrangement unit 150 may change the 2D coefficient array into a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 may output an ID coefficient sequence by scanning the DC coefficient into a high frequency domain coefficient using a zigzag scan or a diagonal scan. Depending on the size of the transform unit and the intra prediction mode, a vertical scan that scans the 2D coefficient array in the column direction and a horizontal scan that scans the 2D block type coefficients in the row direction may also be used instead of the zigzag scan. In other words, depending on the size of the transform unit and the intra prediction mode, the scanning method to be used may be determined among zigzag scanning, diagonal scanning, vertical scanning, and horizontal scanning.

[0069] The entropy encoder 155 generates a bitstream by encoding the sequence of ID-quantized transform coefficients output from the rearrangement unit 150 using various encoding schemes including context-based adaptive binary arithmetic coding (CABAC), exponential Golomb, and the like.

[0070] In addition, the entropy encoder 155 encodes information related to block partitioning (such as CTU size, CTU split flag, QT split flag, MTT partition type, MTT partition direction, etc.) to allow the video decoding device to equally partition the blocks to the video encoding device. In addition, the entropy encoder 155 encodes information about the prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about intra-frame prediction mode) or inter-frame prediction information (in the case of merge mode, merge index, and in the case of AMVP mode, information about reference picture index and motion vector difference) according to the prediction type. In addition, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

[0071] The inverse quantizer 160 dequantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct a residual block.

[0072] The adder 170 reconstructs the current block by adding the reconstructed residual block and the prediction block generated by the predictor 120. When intra prediction is performed on a next sequential block, pixels in the reconstructed current block may be used as reference pixels.

[0073] The loop filter unit 180 performs filtering on the reconstructed pixels in order to reduce block artifacts, ringing artifacts, blur artifacts, etc. that occur due to block-based prediction and transform / quantization. The loop filter unit 180 as a loop filter may include all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0074] The deblocking filter 182 filters the boundaries between the reconstructed blocks to remove the block effect caused by block unit encoding / decoding, and the SAO filter 184 and the ALF 186 perform additional filtering for the deblocked filtered video. The SAO filter 184 and the ALF 186 are filters for compensating the difference between the reconstructed pixels and the original pixels, which is caused by lossy coding. The SAO filter 184 applies an offset as a CTU unit to enhance subjective picture quality and coding efficiency. On the other hand, the ALF 186 performs block unit filtering and applies different filters to compensate for distortion by dividing the boundaries of the corresponding blocks and the degree of variation. Information about the filter coefficients to be used for the ALF can be encoded and notified to the video decoding device with a signal.

[0075] The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-frame prediction of blocks in a picture to be encoded later.

[0076] Figure 5 is a functional block diagram of a video decoding device that can implement the techniques of the present invention. Figure 5 , describes a video decoding device and components of the device.

[0077] The video decoding apparatus may include an entropy decoder 510 , a reordering unit 515 , an inverse quantizer 520 , an inverse transformer 530 , a predictor 540 , an adder 550 , a loop filter unit 560 , and a memory 570 .

[0078] and Figure 1 Similar to the video encoding device of the present invention, each component of the video decoding device can be implemented as hardware or software or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0079] The entropy decoder 510 extracts information related to block partitioning by decoding a bit stream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to reconstruct the current block and information about a residual signal.

[0080] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and divides the picture into CTUs of a determined size. In addition, the CTU is determined to be the highest level of the tree structure, i.e., the root node, and the partition information of the CTU can be extracted to partition the CTU using the tree structure.

[0081] For example, when the CTU is divided using the QTBTTT structure, the first flag (QT_split_flag) related to the division of the QT is first extracted to divide each node into four nodes of the lower layer. In addition, for the node corresponding to the leaf node of the QT, the second flag (mtt_split_flag) related to the division of the MTT, the division direction (vertical / horizontal) and / or the division type (binary / ternary) are extracted to divide the corresponding leaf node into the MTT structure. As a result, each node below the leaf node of the QT is recursively split into a BT or TT structure.

[0082] As another embodiment, when a CTU is divided by using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is divided is extracted. When the corresponding block is divided, a first flag (QT_split_flag) may also be extracted. During the splitting process, for each node, 0 or more recursive MTT splits may occur after 0 or more recursive QT splits. For example, for a CTU, MTT splitting may occur immediately, or conversely, only multiple QT splits may occur.

[0083] For another example, when the CTU is split using the QTBT structure, the first flag (QT_split_flag) related to the splitting of the QT is extracted to split each node into four nodes of the lower layer. In addition, the split flag (split_flag) indicating whether the node corresponding to the leaf node of the QT is further split into BTs and the split direction information are extracted.

[0084] Meanwhile, when the entropy decoder 510 determines the current block to be decoded by using the partitioning of the tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 510 extracts syntax elements of intra-prediction information (intra-prediction mode) for the current block. When the prediction type information indicates inter-prediction, the entropy decoder 510 extracts information representing syntax elements for inter-prediction information (i.e., motion vectors and reference pictures referenced by motion vectors).

[0085] Furthermore, the entropy decoder 510 extracts quantization-related information and extracts information on a quantized transform coefficient of the current block as information on a residual signal.

[0086] The rearrangement unit 515 may change the sequence of 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (ie, block) again in the reverse order of the coefficient scanning order performed by the video encoding apparatus.

[0087] The inverse quantizer 520 dequantizes the quantized transform coefficients and dequantizes the quantized transform coefficients by using quantization parameters. The inverse quantizer 520 may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 may perform inverse quantization by applying a matrix (scaling value) of quantization coefficients from a video encoding device to a 2D array of quantized transform coefficients.

[0088] The inverse transformer 530 generates a residual block for the current block by reconstructing a residual signal by inversely transforming the dequantized transform coefficients from the frequency domain into the spatial domain.

[0089] In addition, when the inverse transformer 530 inversely transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) that only the sub-block of the transform block is transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms the transform coefficient of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal and fills the region that is not inversely transformed with a value of "0" as the residual signal to generate a final residual block for the current block.

[0090] In addition, when MTS is applied, the inverse transformer 530 determines a transform index or a transform matrix applied in each of the horizontal and vertical directions by using MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions by using the determined transform function.

[0091] The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0092] The intra predictor 542 determines an intra prediction mode of a current block among a plurality of intra prediction modes according to a syntax element of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to the intra prediction mode.

[0093] The inter predictor 544 determines a motion vector of a current block and a reference picture to which the motion vector refers by using a syntax element for the inter prediction mode extracted from the entropy decoder 510 .

[0094] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 and the prediction block output from the inter predictor 544 or the intra predictor 542. When intra prediction is performed on a block to be decoded later, pixels within the reconstructed current block are used as reference pixels.

[0095] The loop filter unit 560 as a loop filter may include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundary between the reconstructed blocks in order to remove the block effect that occurs due to block unit decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking filtering in order to compensate for the difference between the reconstructed pixels and the original pixels that occurs due to lossy encoding. The filter coefficients of the ALF are determined by using information about the filter coefficients decoded from the bitstream.

[0096] The reconstructed blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-frame prediction of blocks in a picture to be encoded later.

[0097] In some embodiments, the present invention relates to encoding and decoding a video image as described above. More specifically, the present disclosure provides a video encoding method and apparatus for selecting the best method from various chroma space conversion methods for an original signal, a predictor, or a residual signal of a current chroma block based on the correlation between chroma channel components. The video encoding method and apparatus use the selected method to perform chroma space conversion of Cb and Cr components.

[0098] The following implementations may be performed by a video encoding device. The following implementations may also be performed by a video decoding device.

[0099] The video encoding device may generate signaling information associated with the present embodiment in terms of optimizing rate distortion when encoding the current block. The video encoding device may use the entropy encoder 155 to encode the signaling information and transmit the encoded signaling information to the video decoding device. The video decoding device may use the entropy decoder 510 to decode the signaling information associated with decoding the current block from the bitstream.

[0100] In the following description, the term "target block" may be used interchangeably with a current block or a coding unit (CU). The term "target block" may refer to some areas of a coding unit.

[0101] In addition, a flag value of true indicates when the flag is set to 1. In addition, a flag value of false indicates when the flag is set to 0.

[0102] I. Encoding of Chroma Components

[0103] Figure 6 is a diagram illustrating the process of encoding chrominance components.

[0104] In the prior art, a video encoding device first generates a predictor (predCb, predCr) of a chrominance component by using various types of intra-frame prediction or inter-frame prediction processing based on information from a previously reconstructed picture (recPic). The video encoding device subtracts the predictor from the original signal (orgCb, orgCr) to generate a residual signal (resCb, resCr), and then converts these residual signals into signals in the frequency domain by using a conversion process. The video encoding device finally synthesizes a bitstream by using a quantization process and an entropy encoding process.

[0105] Chroma channel coding generally uses a method of sequentially applying DCT-II transform to residual signals of Cb and Cr components in horizontal and vertical directions, or a transform skip mode method of skipping transform and directly going to the quantization stage. In the conventional technology, the video encoding device notifies the video decoding device of the method to be applied to each component by using transform_skip_flag[x][y][compID] as shown in Table 1.

[0106] [Table 1]

[0107] [TU level]

[0108]

[0109] Here, x, y are the coordinates of the upper left pixel of the current residual block for each channel. complD (if it is 0) indicates the Y component, complD (if it is 1) indicates the Cb component, and complD (if it is 2) indicates the Cr component. If transform_skip_flag[x][y][compID] is 1, the transform skip mode is applied, and if transform_skip_flag[x][y][compID] is 0, the DCT-II transform is applied to the residual signal in the horizontal and vertical directions. In the following, for ease of description, transform_skip_flag[x][y][1] and transform_skip_flag[x][y][2] applied to the two component signals in the chrominance channel, respectively, are generally referred to as transform_skip_flag. In addition, in order to simplify the representation, what is omitted from transform_skip_flag[x][y][compID] is the x, y representation of the coordinates of the upper left pixel of the current residual block to leave transform_skip_flag[compID].

[0110] II. Joint Coding of Chroma Residual (JCCR) Technique

[0111] The joint coding (JCCR) technology for chroma residual is to solve the problems existing in the existing method of encoding the residual signals of the Cb and Cr components separately to improve the coding efficiency. Based on the fact that the residual signals of Cb and Cr are inversely correlated with each other, the JCCR technology combines the residual signals of the two components into one to generate a new residual signal, and then encodes the new residual signal. The video encoding device can use the tu_joint_cbcr_residual_flag of each transform unit (TU) (the basic unit of residual signal encoding) to signal the video decoding device to indicate whether the JCCR technology is used in the chroma channel. If tu_joint_cbcr_residual_flag is 0, the JCCR technology is not used, and the residual signal and transform_skip_flag are transmitted to the video decoding device for each of the Cb component and the Cr component as described above. This is called the case where the JCCR mode is 0. If tu_joint_cbcr_residual_flag is 1, the JCCR technology is applied. Then, the residual signals of Cb and Cr may be combined into one, and the combined residual signal may be encoded and decoded.

[0112] The JCCR technique has a total of three modes (JCCR modes). Depending on the combination of coded block flags (CBFs) indicating whether any one of the conversion factors or coefficient levels of each component has a non-zero value, the three modes are classified as shown in Table 2. In other words, except when the CBF value of the Cb component is CBF cb (=tu_cb_coded_flag) and CBF value of Cr component cr When (=tu_cr_coded_flag) are all zero, the JCCR mode can be classified according to three combinations of 0 and 1.

[0113] [Table 2]

[0114]

[0115] After selecting the JCCR mode, the combined residual signal is encoded and decoded according to the mode as shown in Table 3.

[0116] [Table 3]

[0117]

[0118] The video encoding device combines the residual signal resCb of the Cb component with the residual signal resCr of the Cr component by using a preset equation (resJointC calculation in Table 3) for each JCCR mode to generate a new residual signal resJointC. Then, the video encoding device encodes the new residual signal resJointC and transmits it to the video decoding device. It should be noted that the equations presented in the resJointC calculation in Table 3 represent three simple cases that may not be sufficient to model various correlations present in the video.

[0119] The video decoding device receives the transmitted tu_cb_coded_flag and tu_cr_coded_flag values ​​and determines the JCCR mode according to Table 2. Then, the video decoding device reconstructs the residual signals of the original Cb and Cr components from the transmitted resJointC according to the preset equations for each mode (reconstruction of Cb and Cr residuals in Table 3). At this time, the video decoding device determines which of the Cb and Cr channels immediately uses the transmitted resJointC as the value of the Cb or Cr channel according to the JCCR mode. In the following text, the channel used immediately is called the coding channel, and in the following text, it is referred to as the "representative channel". According to Table 3, when the JCCR mode is 1 or 2, resJointC is used as the Cb channel (that is, the coded channel becomes the Cb channel), and when the JCCR mode is 3, resJointC is used as the Cr channel (that is, the coded channel becomes the Cr channel). In Table 3, the channel using the received resJointC is marked as the coded channel.

[0120] The limitation of the JCCR technology of the prior art is that the encoded channel is always one of the Cb and Cr channels. As described above, this limitation is due to the fact that the prior art models the correlation by using three simple models described in Table 3, even though there may be various correlations between the Cb and Cr channels. In order to solve the prior art in which the representative channel is limited to the Cb channel or the Cr channel, the present disclosure provides a technology for using more new channels as representative channels.

[0121] Meanwhile, cSign included in the equation to derive resJointC is a sign value calculated as shown in Equation 1. ph_joint_cbcr_sign_flag is transmitted on a per-picture basis, and when the JCCR technique is applied, it distinguishes whether the signs of the residual signals of Cb and Cr are the same or have an inverse relationship. If ph_joint_cbcr_sign_flag is 1, cSign=-1, whereby the signs of the residual signals of Cb and Cr are opposite, and if ph_joint_cbcr_sign_flag is 0, cSign=1, whereby the signs of the residual signals of Cb and Cr are the same.

[0122] [Equation 1]

[0123] cSign=(1–2×ph_joint_cbcr_sign_flag)

[0124] The three JCCR modes are specifically described as follows: The following describes the operation of the JCCR mode based on the cSign value being -1.

[0125] First, when the JCCR mode is 1, the relationship between the residual signals of the Cb component and the Cr component is modeled so that the residual signal resCb of the Cb component is -2 times the residual signal resCr of the Cr component, as shown in Equation 2. The video encoding device generates a new residual signal resJointC according to Equation 3. The video encoding device then indicates resJointC and information on whether the transform skip mode is applied to the relevant residual signal in transform_skip_flag[1] of the Cb component according to Table 3, and transmits the generated transform_skip_flag[1] to the video decoding device.

[0126] [Equation 2]

[0127] resCb=cSign×2×resCr

[0128] [Equation 3]

[0129] resJointC=(4×resCb+cSign×2×resCr) / 5

[0130] The video decoding apparatus reconstructs the residual signals of the Cb and Cr components according to Equation 4 by using the transmitted resJointC. At this time, the relationship of Equation 2 may be derived by setting and organizing resJointC set in Equations 3 and 4 as an equation as shown in Equation 5.

[0131] [Equation 4]

[0132] resCb=resJointC

[0133] resCr=(cSign×resJointC)>>1

[0134] [Equation 5]

[0135] (4×resCb+cSign×2×resCr) / 5=resCb

[0136] →4×resCb+cSign×2×resCr=5×resCb→resCb=cSign×2×resCr

[0137] When the JCCR mode is 1, the residual signals of the Cb and Cr components of the 4×4 block can be expressed as Figure 7 Instances in .

[0138] When the JCCR mode is 2, the relationship between the residual signals of the Cb component and the Cr component is modeled so that the residual signal resCb of the Cb component is -1 times the residual signal resCr of the Cr component, as shown in Equation 6. The video encoding device generates a new residual signal resJointC according to Equation 7. Then, the video encoding device indicates resJointC and information on whether the transform skip mode is applied to the relevant residual signal in transform_skip_flag[1] of the Cb component according to Table 3, and transmits the generated transform_skip_flag[1] to the video decoding device.

[0139] [Equation 6]

[0140] resCb=cSign×resCr

[0141] [Equation 7]

[0142] resJointC=(resCb+cSign resCr) / 2

[0143] The video decoding apparatus reconstructs the residual signals of the Cb and Cr components according to Equation 8 by using the transmitted resJointC. At this time, the relationship of Equation 6 may be derived by setting and organizing resJointC set in Equations 7 and 8 as an equation as shown in Equation 9.

[0144] [Equation 8]

[0145] resCb=resjoitC

[0146] resCr=cSign×resJointC

[0147] [Equation 9]

[0148] (resCb+cSign×resCr) / 2=resCb

[0149] →resCb+cSign×resCr=2×resCb→resCb=cSign×resCr

[0150] When the JCCR mode is 3, the relationship between the residual signal of the Cb component and the residual signal of the Cr component is modeled so that the residual signal resCr of the Cr component is -2 times the residual signal resCb of the Cb component, as shown in Equation 10. The video encoding device generates a new residual signal resJointC according to Equation 11. The video encoding device then indicates resJointC and information on whether the transform skip mode is applied to the relevant residual signal in transform_skip_flag[2] of the Cr component according to Table 3, and transmits the generated transform_skip_flag[2] to the video decoding device.

[0151] [Equation 10]

[0152] resCr=cSign×2×resCb

[0153] [Equation 11]

[0154] resJointC=(4×resCr+cSign×2×resCb) / 5

[0155] The video decoding apparatus reconstructs the residual signals of the Cb and Cr components according to Equation 12 by using the transmitted resJointC. At this time, the relationship of Equation 10 may be derived by setting and organizing resJointC set in Equations 11 and 12 as an equation as shown in Equation 13.

[0156] [Equation 12]

[0157] resCb=(cSign×resJointC)>>1

[0158] resCr=resjoitC

[0159] [Equation 13]

[0160] (4×resCr+cSign×2×resCb) / 5=resCr

[0161] →4×resCr+cSign×2×resCb=5×resCr→resCr=cSign×2×resCb

[0162] The problem with the above-mentioned processing of encoding and decoding chrominance channel residual signals is that the JCCR technique ignores the various relationships between the Cb and Cr components. In other words, only three highly simplified proportional relationships are modeled to combine the residual signals of the two components into one, and then encoded and decoded. However, the correlation between the Cb component and the Cr component is more than the three proportional relationships mentioned above. This phenomenon may be more obvious when the chrominance channel has a higher resolution, such as in 4:2:2 and 4:4:4 color formats. Therefore, the usefulness of the traditional JCCR technique may be reduced, especially when the resolution of the chrominance channel is high. These problems of conventional technology can be solved according to the present disclosure by applying an adaptive chrominance space conversion (CSC) that reflects the different correlations between the two components Cb and Cr in the chrominance channel. The signals in the chrominance channel to which the chrominance space conversion according to the present disclosure can be applied may include residual signals, predictors, original signals, etc.

[0163] III. Adaptive Color Space Conversion

[0164] Figure 8 and Fig. 9 is a diagram illustrating a video encoding device that applies chroma space conversion and a video decoding device that applies inverse chroma space conversion according to some embodiments of the present disclosure.

[0165] exist Figure 8 and Fig. 9 In the example of , chroma space conversion (CSC) 810 and inverse chroma space conversion (ICSC) 910 according to the present disclosure are applied to the residual signal in the chroma channel.

[0166] The video encoding device applies CSC 810 to the two-component residual signals resCb, resCr to generate new converted residual signals resC1, resC2, and applies conversion, quantization and entropy coding thereto for generating a compressed bitstream. For the received compressed bitstream, the video encoding device applies entropy decoding, inverse quantization and inverse transform to reconstruct the converted residual signals resC1', resC2'. The video decoding device applies ICSC 910 to the converted residual signal to reconstruct the residual signals resCb', resCr' of the two channels Cb and Cr. The video decoding device then adds the residual signal to the predictors predCb', predCr' generated by using the applicable prediction method and the previously reconstructed picture recPic to generate the final reconstructed signals recCb, recCr.

[0167] Fig.10 and Fig.11 2 is a diagram illustrating a video encoding device applying chroma space conversion and a video decoding device applying inverse chroma space conversion according to another embodiment of the present disclosure.

[0168] exist Fig.10 and 11 In the example of FIG. 8 , CSC 810 , 1010 , 1110 and ICSC 910 according to the present invention are applied to the predictors in the chroma channels.

[0169] The video encoding apparatus applies CSC 810 to two-component Cb and Cr predictors predCb and predCr predicted by using information about a previously reconstructed picture recPic to generate new converted predictors predC1, predC2. Similarly, CSC 1010 is applied to the original signals orgCb, orgCr to generate new converted original signals orgC1, orgC2. The video encoding apparatus then subtracts the converted predictors from the converted original signals to generate residual signals resC1 and resC2, which are converted, quantized, and entropy encoded to generate compressed bitstreams.

[0170] The video decoding device applies entropy decoding, inverse quantization and inverse transformation to the transmitted compressed bit stream to reconstruct new converted residual signals resC1', resC2'. The video decoding device generates predictors predCb', predCr' according to the applicable prediction method, and applies CSC 1110 to the predictors to generate converted predictors predC1', predC2'. The video decoding device then adds the converted residual signals resC1', resC2' to the converted predictors predC1', predC2' to reconstruct the converted reconstructed signals recC1', recC2'. Finally, the video decoding device applies ICSC 910 to the converted reconstructed signals recC1', recC2' to generate reconstructed signals recCb, recCr.

[0171] Fig.12 and Fig.13 is a diagram illustrating a video encoding device applying chroma space conversion and a video decoding device applying inverse chroma space conversion according to still another embodiment of the present disclosure.

[0172] exist Fig.12 and Fig.13 In the example of FIG. 8 , the CSC 810 and ICSC 910 according to the present disclosure are applied to the original signal in the chrominance channel.

[0173] The video encoding apparatus applies CSC 810 to the original signals orgCb, orgCr, and the previously reconstructed picture recPic to generate new converted original signals orgC1, orgC2, and reconstructed converted pictures recPic', and performs a prediction method such as intra prediction or inter prediction on the converted original signals and the reconstructed converted pictures to generate converted predictors predC1, predC2. The video encoding apparatus then subtracts the predictor from the converted original signal orgC1 or orgC2 to generate converted residual signals resC1 and resC2 that are converted, quantized, and entropy encoded to generate compressed bitstreams.

[0174] The video decoding device applies entropy decoding, inverse quantization and inverse transformation to the transmitted compressed bitstream to reconstruct the converted residual signals resC1', resC2'. The video decoding device generates converted predictors predC1', predC2' according to the applicable prediction method and the recPic' information from applying the CSC 1110 to the previously reconstructed picture recPic. The video decoding device then adds the converted residual signals resC1', resC2' to the converted predictors predC1', predC2' to reconstruct the converted reconstructed signals recC1, recC2. Finally, the video decoding device applies ICSC 910 to the converted reconstructed signals recC1, recC2 to generate reconstructed signals recCb, recCr.

[0175] Fig.14a and Fig.14b is a diagram illustrating chroma space conversion (CSC) and inverse chroma space conversion (ICSC) according to some embodiments of the present disclosure.

[0176] like Fig.14a As shown in the example of , CSC 810 generates two converted signals from two component signals in the chrominance channel. In this case, the chrominance space conversion can be implemented in various ways depending on how the two component signals Cb and Cr are combined to generate the two converted signals. In terms of optimizing coding efficiency, the video encoding device selects one of the various chrominance space conversion methods and signals the two converted signals and syntax information related to the selected method to the video decoding device through the bitstream. The video decoding device decodes the chrominance space conversion method from the bitstream. Further, as Fig.14b As shown in the example of FIG. 1 , ICSC 910 generates original Cb and Cr signals from the reconstructed converted signals by using inverse chroma space conversion, which is the inverse process of the chroma space conversion operation used by the video encoding device. The specific implementation method is described below.

[0177] <Implementation 1> Combining Cb and Cr component signals to generate two conversion signals

[0178] This implementation is a chroma space conversion method that combines the Cb and Cr component signals in the chroma channels according to a specific equation for generating two converted signals.

[0179] The video encoding apparatus combines the Cb, Cr component signals, sigCb and sigCr, according to the matrix expression of Equation 14 or its equivalent Equation 15 to generate and encode converted signals sigC1 and sigC2.

[0180] [Equation 14]

[0181] in

[0182] [Equation 15]

[0183] sigC1=a·sigCb+b·sigCr

[0184] sigC2=c·sigCb+d·sigCr

[0185] Here, the combination matrix A is a 2×2 matrix that may be determined in various forms depending on applications. The video encoding apparatus may combine Cb and Cr component signals based on the combination matrix A to generate two converted signals.

[0186] The video decoding device combines the reconstructed conversion signals sigC1' and sigC2' according to the matrix expression of Equation 16 or its equivalent Equation 17, and generates reconstructed Cb, Cr two-component signals sigCb' and sigCr' and then decodes them.

[0187] [Equation 16]

[0188] in

[0189] [Equation 17]

[0190] sigCb′=g·sigC1′+h·sigC2′

[0191] sigCr′=i·sigC1′+j·sigC2′

[0192] Here, the inverse combination matrix A -1is an inverse matrix of a 2×2 matrix A used in a chromaticity space conversion process performed by a video encoding device. Since various chromaticity space conversion methods may exist depending on the value of the combination matrix A, the video encoding device may signal the selected chromaticity space conversion method to the video decoding device to be applied to the Cb and Cr component signals. According to an embodiment, the video encoding device may use a method of signaling the value of the combination matrix A or a method of signaling the index of the preset combination matrix A. In this case, the video decoding device includes calculating an inverse matrix A corresponding to the signaled combination matrix A. -1 In another embodiment, the video encoding device may use signaling A -1 method to set the value of A, or to signal a preset -1 The video encoding device may signal this information to the video decoding device on a block-by-block basis. Alternatively, instead of transmitting the information block-by-block, the information transmitted once may be shared by multiple blocks.

[0193] In this implementation, the method of signaling the combination matrix A or the inverse combination matrix A is signaled. -1 The method relies on one of the following schemes.

[0194] First, when the index of the preset combination matrix A is signaled, the video encoding device may separate the preset combination matrix A by index by using a new syntax named conversion_mode_index[x][y], as shown in Table 4. Then, the video encoding device may signal the index of the combination matrix A to the video decoding device for each chroma block. The video decoding device may use the transmitted index to obtain the value of the combination matrix A from the preset list as shown in Table 4.

[0195] [Table 4]

[0196]

[0197] Here, x, y are the coordinates of the upper left pixel of the current residual block subjected to the syntax. Hereinafter, conversion_mode_index[x][y] is referred to as conversion_mode_index, and its coordinate value is omitted.

[0198] Secondly, when the value of the combined matrix A is directly signaled, the video encoding device utilizes a new array of the syntax cbcr_conversion_matrix[x][y][numComp]. Here, x, y are the coordinates of the upper left pixel of the current residual block that undergoes the corresponding syntax. numComp indicates the number of components in the combined matrix. In a preferred implementation, numComp may have a value of 4. Further, the video encoding device may assign the values ​​of the four components of the combined matrix A to cbcr_conversion_matrix[x][y][4] in the form of an array (e.g., {a, b, c, d}), and may signal the assigned component values ​​to the video decoding device for each chroma block. In the following, cbcr_conversion_matrix[x][y][numComp] is referred to as cbcr_conversion_matrix[numComp], and its x, y coordinate values ​​are omitted.

[0199] Third, when the preset A is signaled -1 When the index is , the video encoding device uses conversion_mode_index as shown in Table 4 to identify the preset inverse combination matrix A through the index. -1 The video encoding device can then inverse the combined matrix A -1 The index of is signaled to the video decoding device. The video decoding device can use the transmitted index to obtain the inverse combination matrix A from the preset list shown in Table 4 -1 The value of .

[0200] Fourth, when the inverse combination matrix A is directly signaled -1 When the value of , the video encoding device uses cbcr_conversion_matrix[numComp]. Here, numComp may have a value of 4. In addition, the video encoding device may inversely combine the matrix A in an array form such as {g, h, i, j} -1 The values ​​of the four components of cbcr_conversion_matrix[4] are assigned to cbcr_conversion_matrix[4] to signal the assigned component values ​​to the video decoding device.

[0201] <Implementation 2> Combine Cb and Cr component signals and add offset to generate two conversion signals

[0202] This implementation is a color space conversion method that combines component signals of Cb and Cr in a chroma channel according to a specific equation and adds an offset, which is a DC component, to generate two converted signals.

[0203] The video encoding apparatus combines the two component signals sigCb and sigCr of Cb and Cr according to the matrix expression of Equation 18 or its equivalent matrix expression of Equation 19. The video encoding apparatus then adds the offsets to generate converted signals sigC1 and sigC2, which can be used in the next encoding step.

[0204] [Equation 18]

[0205] in

[0206] [Equation 19]

[0207] sigC1=a·sigCb+b·sigCr+e

[0208] sigC2=c·sigCb+d·sigCr+f

[0209] Here, the combination matrix A is any 2×2 matrix, which may be in the form illustrated in Table 5. The offset matrix B may be any 2×1 matrix, and may be in the form illustrated in Table 5. Depending on the embodiment, the matrices A and B may have different forms, including but not limited to Table 5. The video encoding apparatus may generate two converted signals using matrices A and B by combining the Cb component signal with the Cr component signal and adding an offset.

[0210] [Table 5]

[0211]

[0212] The video decoding device may combine the reconstructed conversion signals sigC1' and sigC2' according to the matrix expression of Equation 20 or its equivalent Equation 21 to generate reconstructed Cb, Cr two-component signals sigCb' and sigCr', which may then be used in the next decoding step.

[0213] [Equation 20]

[0214]

[0215] in

[0216] [Equation 21]

[0217] SigCb′=g·sigC1′+h·sigC2′-e·gf·h

[0218] sigCr′=i·sigC1′+j·sigC2′-e·if·j

[0219] Here, the inverse combination matrix A -1is an inverse matrix of the 2×2 combination matrix A used by the video encoding apparatus in the chrominance space conversion process, and the offset matrix B is a matrix used in the chrominance space conversion process. Since various chrominance space conversion methods may exist depending on the values ​​of the combination matrix A and the offset matrix B, the video encoding apparatus may signal the best chrominance space conversion method applied to the signals of the two components Cb and Cr to the video decoding apparatus.

[0220] According to an embodiment, the video encoding apparatus may use a method of signaling the values ​​of the combined matrix A and the offset matrix B, or a method of signaling the index of each preset matrix. In this case, the video decoding apparatus may include calculating an inverse matrix A corresponding to the signaled combined matrix A. -1 As another example, the video encoding device may use a signal to notify the inverse combination matrix A -1 The video encoding device may signal this information to the video decoding device on a block-by-block basis. Alternatively, instead of transmitting block by block, the information transmitted at one time may be shared by multiple blocks.

[0221] In this implementation, the method of signaling the combination matrix A and the offset matrix B or signaling the inverse combination matrix A -1 The method of summing the offset matrix B relies on one of the following schemes.

[0222] First, when the index of the preset combination matrix A and the offset matrix B is notified by signaling, the video encoding device can separate the preset combination matrix A and the offset matrix B by the index by using conversion_mode_index defined in implementation 1, as shown in Table 6. The video encoding device can then notify the video decoding device of the index signal described above for each chroma block. The video decoding device can use the transmitted index to obtain the values ​​of the combination matrix A and the offset matrix B from the preset list as shown in Table 6.

[0223] [Table 6]

[0224]

[0225] Second, when the value of the combination matrix A is directly signaled and the index of the preset offset matrix B is signaled, the video encoding device may use cbcr_conversion_matrix[numComp] defined in implementation 1. For example, the video encoding device may set numComp to 4 and may signal the video decoding device by assigning the values ​​of the four components of the combination matrix A to cbcr_conversion_matrix[4] in the form of an array such as {a, b, c, d}. In addition, the video encoding device may notify the index of the preset offset matrix B to the video decoding device for each chroma block by using conversion_mode_index as shown in Table 6.

[0226] Third, when the index of the preset combination matrix A is signaled and the value of the offset matrix B is directly signaled, the video encoding device may notify the video decoding device of the index of the preset combination matrix A by using conversion_mode_index as illustrated in Table 6. In addition, the video encoding device may signal the video decoding device by setting numComp to 2 and assigning the values ​​of two components of the offset matrix B to cbcr_conversion_matrix[2] in the form of an array (such as {e, f}).

[0227] Fourth, when the values ​​of the combination matrix A and the offset matrix B are directly notified by a signal, the video encoding device may set numComp to 6 and assign the four component values ​​of the combination matrix A and the value of the offset matrix B (i.e., a total of six component values) to cbcr_conversion_matrix[6] in an array (such as {a, b, c, d, e, f}), and signal the assigned component values ​​to the video decoding device.

[0228] Fifth, when the preset inverse combination matrix A is notified by signal -1 When the index of the offset matrix B is obtained, the video encoding device may separate the preset inverse combination matrix A by the index by using the conversion_mode_index defined in implementation mode 1 -1 and offset matrix B, as shown in Table 7. The video encoding device may then notify the video decoding device of the index signal described above for each chroma block. The video decoding device may use the transmitted index to obtain the inverse combination matrix A from the preset list as shown in Table 7 -1 and the values ​​of the offset matrix B.

[0229] [Table 7]

[0230]

[0231]

[0232] Sixth, when the inverse combination matrix A is directly notified using the signal -1 When the value of and the index of the preset offset matrix B is notified by a signal, the video encoding device may use cbcr_conversion_matrix[numComp] defined in implementation mode 1. For example, the video encoding device may set numComp to 4 and may inversely combine the matrix A -1 The values ​​of the four components of are assigned to cbcr_conversion_matrix[4] in the form of an array (such as {g, h, i, j}) to signal the assigned component values ​​to the video decoding device. In addition, the video encoding device can signal the index of the preset offset matrix B to the video decoding device for each chroma block by using conversion_mode_index as shown in Table 6.

[0233] Seventh, when the preset inverse combination matrix A is notified by signal -1 When the value of the offset matrix B is directly signaled, the video encoding device may convert the preset inverse combination matrix A to -1 The index signal of the video decoding device is notified to the video decoding device. In addition, the video encoding device can use a signal to notify the video decoding device by setting numComp to 2 and assigning the values ​​of the two components of the offset matrix B to cbcr_conversion_matrix[2] in the form of an array (such as {e, f}).

[0234] Eighth, when the inverse combination matrix A is directly notified -1 and the value of the offset matrix B, the video encoding device may be configured to inverse the combined matrix A by setting numComp to 6 -1 The four component values ​​of and the value of the offset matrix B (i.e., a total of six component values) are assigned to cbcr_conversion_matrix[6] in an array such as {g, h, i, j, e, f}, signaling the video decoding device.

[0235] <Implementation 3> Combine the Cb and Cr component signals to generate two conversion signals, wherein one conversion signal is set to one of the Cb and Cr signals

[0236] Fig.15a and Fig.15b is a diagram illustrating a CSC and an ICSC according to other embodiments of the present disclosure.

[0237] like Fig.15a and Fig.15bAs shown, this embodiment is a chrominance space conversion method that combines two component signals Cb and Cr in a chrominance channel to generate two conversion signals, one of which is set to one of the Cb and Cr signals. In the following, for ease of description, a case where the Cr signal is used as the conversion signal is described, but the same operation is feasible when the conversion signal becomes Cb.

[0238] The linear relationship between the Cb component and the Cr component based on the Cb component can be expressed as shown in Equation 22.

[0239] [Equation 22]

[0240] sigCb=α·sigCr+β

[0241] Here, α is a coefficient by which the Cr component signal is multiplied, which may be various values ​​such as ±1, ±2, ±1 / 2, etc. β is a constant and may be derived according to Equation 23, which is a variation of Equation 22.

[0242] [Equation 23]

[0243] β=sigCb-α·sigCr

[0244] The video encoding device may immediately set sigCr to a conversion signal sigC2, such as Fig.15a 24, and β derived from equation 23 is set as the remaining conversion signal sigC1 to generate a total of two conversion signals, which can then be used in the next encoding step. In other words, the video encoding apparatus can combine the Cb and Cr component signals to generate two converted signals, one of which is set to one of the Cb and Cr components.

[0245] [Equation 24]

[0246] sigC1=β,sigC2=sigCr

[0247] The video decoding device can combine the reconstructed conversion signals sigC1′ and sigC2′ according to equation 25 to generate reconstructed Cb and Cr two-component signals sigCb′ and sigCr′, as shown in Fig.15b As shown in the example, the reconstructed Cb and Cr component signals are used in the next decoding step.

[0248] [Equation 25]

[0249] sigCr′=sigC2′

[0250] sigCb′=α·sigC2′+sigC1′

[0251] Since there may be various chromaticity space conversion methods according to the value of the coefficient α, the video encoding device may signal the best chromaticity space conversion method applied to the signals of the two components Cb and Cr to the video decoding device. According to an embodiment, the video encoding device may use a method of directly signaling an α value or a method of signaling an index of a preset α value. The video encoding device may signal this information to the video decoding device block by block. Alternatively, instead of transmitting information block by block, the information transmitted once may be shared by multiple blocks.

[0252] In this embodiment, signaling the coefficient α relies on one of the following schemes.

[0253] First, when the index of the value of the preset coefficient α is signaled, the video encoding device separates the value of the preset coefficient α by index by using conversion_mode_index defined in implementation 1, as shown in Table 9. The video encoding device may then signal the index described above to the video decoding device for each chroma block.

[0254] [Table 8]

[0255] conversion_mode_index α 0 1 1 2 2 1 / 2

[0256] Secondly, when the value of the coefficient α is directly notified by signal, the video encoding device utilizes the new syntax cbcr_conversion_coefficient[x][y]. Here, x, y are the upper left pixel coordinates of the current residual block subjected to the new syntax. Hereinafter, cbcr_conversion_coefficient[x][y] is referred to as cbcr_conversion_coefficient, and the coordinate value is omitted. For each chroma block, the video encoding device may assign the value of the coefficient α to cbcr_conversion_coefficient, and may signal the assigned value to the video decoding device.

[0257] <Implementation Method 4> Signal Notification and Syntax Related to Chroma Space Conversion Method

[0258] This implementation is a method of signaling syntax information related to a chroma space conversion method according to any one of Implementation 1 to Implementation 3.

[0259] The video decoding device receives the two transmitted conversion signals (sigC1', sigC2'), uses the type of chromaticity space conversion method applied to the two conversion signals and the accompanying additional syntax to signal, and then reconstructs the original Cb, Cr component signals by performing an inverse chromaticity space conversion method, which is an inverse operation of the chromaticity space conversion. As described above, the Cb, Cr component signals can be residual signals, prediction signals or original signals.

[0260] <Implementation Method 4-1> When the Cb and Cr component signals are residual signals

[0261] When the Cb component signal and the Cr component signal are residual signals, the video encoding device signals the CBF value and the conversion method for each component (i.e., whether to apply the transform skip mode or the DCT-II transform) for the two component signals, wherein the CBF value indicates whether the two component signals have non-zero coefficients when converted to the frequency domain. In addition, when implementations 1 to 3 are applied, the video encoding device may signal syntax information related to the chroma space conversion method of each implementation. The chroma space conversion methods of implementations 1 to 3 may be applied in four different ways, as follows.

[0262] <Implementation 4-1-1> The color space conversion methods of Implementations 1 to 3 are applied unconditionally.

[0263] In this implementation, the video decoding device may unconditionally apply the chroma space conversion method of implementations 1 to 3 instead of the conventional method. Here, the conventional method refers to a method of decoding the residual signals of the Cb component and the Cr component separately, or a method of decoding by applying the JCCR technology. As an example, the syntax of the signal associated with the chroma space conversion method may be organized at the TU level as shown in Table 9. In the following, unless otherwise specified, the syntax is presented at the TU level.

[0264] [Table 9]

[0265] [TU level]

[0266]

[0267]

[0268] Here, chromaAavailable is 1 to indicate that the current residual block is a block available for conversion in the chroma channel, and chromaAavailable is 0 to indicate that other blocks are available for conversion.

[0269] According to Table 9, when the chroma space conversion method of implementations 1 to 3 is applied unconditionally, JCCR is not applied, the JCCR deletes the condition for checking whether the existing JCCR is applied, and therefore, the tu_joint_cbcr_residual_flag that signals whether JCCR is applied is deleted. In addition, conversion_mode_index (an index for distinguishing between different chroma space conversion methods within implementations (1, 2, 3)) is added to the existing flag position. The added syntax may also vary according to the specific operation mode of each implementation. For implementation 1 or 2, the conversion_mode_index in Table 9 can be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, the conversion_mode_index in Table 9 can be replaced by cbcr_conversion_coefficient.

[0270] When the chroma space conversion methods of implementations 1 to 3 are applied unconditionally, the CBF values ​​of the original Cb and Cr residual signals in the prior art and the syntax elements indicating whether the transform skip mode is applied are replaced with the syntax of the two converted signals applying the chroma space conversion.

[0271] As another example, the syntax of a signal associated with a chroma space conversion method may be organized as shown in Table 10.

[0272] [Table 10]

[0273]

[0274] Here, chromaAvailable is equal to 1 (indicating that the current residual block is a block that can be converted in the chroma channel), and chromaAvailable is equal to 0 (indicating other blocks that can be used for conversion).

[0275] According to Table 10, if the CBFs of both the Cb and Cr residual signals are zero, it is assumed that the chrominance space conversion method does not need to be performed and the associated syntax is not signaled. If at least one of the CBFs of the Cb and Cr residual signals is non-zero and the chrominance space conversion methods of implementations 1 to 3 are applied unconditionally, JCCR is not applied, and JCCR deletes the condition for checking whether the existing JCCR is applied and tu_joint_cbcr_residual_flag to signal whether JCCR is applied accordingly. In addition, the CBF values ​​tu_c1_coded_flag[xC][yC] and tu_c2_coded_flag[xC][yC] of the two conversion signals are added to the existing flag positions. In the following, for ease of description, the coordinates xC, yC of the upper left pixel of the current residual block are omitted and represented as tu_c1_coded_flag and tu_c2_coded_flag.

[0276] In addition, conversion_mode_index (an index that distinguishes the different color space conversion methods within each implementation) is added to the existing flag position. The added syntax may also vary depending on the specific operation mode of each implementation. For implementation 1 or 2, the conversion_mode_index in Table 10 can be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, the conversion_mode_index in Table 10 can be replaced by cbcr_conversion_coefficient.

[0277] Compared with Table 9, the syntax may be configured in a new manner that does not follow the syntax composition of the existing VVC technology. In the syntax configuration of VVC, the syntax is signaled by the CBF values ​​of the Cb and Cr components, tu_joint_cbcr_residual_flag of the Cb and Cr components in this sequence, and transform_skip_flag, as shown in Table 11.

[0278] [Table 11]

[0279]

[0280]

[0281] This is because when tu_joint_cbcr_residual_flag is true and the JCCR technique is applied, the transform_skip_flag of the remaining components other than the reference component responsible for sending the transform_skip_flag for the combined residual signal is not signaled. Therefore, tu_joint_cbcr_residual_flag is signaled before transform_skip_flag. However, if the chroma space conversion method of implementations 1 to 3 is applied unconditionally according to the new syntax composition, it is necessary to signal the transform_skip_flag to be applied to the two converted signals. Unlike when the JCCR technique is applied, the syntax for distinguishing the chroma space conversion method does not need to be signaled before transform_skip_flag. Therefore, as shown in Table 12, after the CBF values ​​of the two converted signals are signaled, the transform_skip_flag of each signal is signaled.

[0282] [Table 12]

[0283]

[0284]

[0285] In this case, according to the chroma space conversion, the syntax components that distinguish the CBF values ​​of the original Cb and Cr residual signals and whether the transform skip mode is applied in the prior art are replaced with syntax for the two converted signals. Then, an index conversion_mode_index is added to distinguish between different chroma space conversion methods within each implementation.

[0286] The added syntax may also vary according to the specific operation mode of each implementation. For implementation 1 or 2, the conversion_mode_index in Table 12 may be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, the conversion_mode_index in Table 12 may be replaced by cbcr_conversion_coefficient.

[0287] As another example, the syntax of a signal associated with a color space conversion method may be organized as shown in Table 13.

[0288] [Table 13]

[0289]

[0290]

[0291] As shown in Table 13, when the CBFs of both Cb and Cr residual signals are zero, it is assumed that no chroma space conversion method needs to be performed, and the associated syntax is not signaled. If at least one of the CBFs of the Cb and Cr residual signals is non-zero, the CBF values ​​tu_c1_coded_flag and tu_c2_coded_flag of the two converted signals are signaled, followed by transform_skip_flag for each signal. Then, the index conversion_mode_index is added to distinguish between different chroma space conversion methods within each implementation.

[0292] The added syntax may also vary according to the specific operation mode of each implementation. For implementation 1 or 2, the conversion_mode_index in Table 13 may be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, the conversion_mode_index in Table 13 may be replaced by cbcr_conversion_coefficient.

[0293] <Implementation 4-1-2> In addition to the existing method, implementations 1 to 3 are also applied

[0294] In this implementation, in addition to the existing method, the video decoding device can also apply implementations 1 to 3. In this case, a new index chroma_conversion_signaling_index[x][y] can be used to distinguish how to signal the Cb and Cr components of each residual block. Here, x, y are the coordinates of the upper left pixel of the current residual block. In the following, the coordinates x, y of the upper left pixel of the current residual block are omitted and expressed as chroma_conversion_signaling_index.

[0295] Table 14 shows the residual signal transmission method of Cb and Cr components according to chroma_conversion_signaling_index.

[0296] [Table 14]

[0297] chroma_conversion_siganaling_index Cb, Cr signal transmission method 0 Existing methods 1 Implementation 1 2 Implementation 2 3 Implementation 3

[0298] Here, the transmission methods are collectively referred to as the conventional method and the color space conversion method.

[0299] For a method in which implementations 1 to 3 are applied in addition to the conventional method, the syntax composition may be as shown in Table 15.

[0300] [Table 15]

[0301]

[0302]

[0303] According to Table 15, the newly introduced chroma_conversion_signaling_index is signaled to determine the transmission method of the residual signals of the Cb and Cr components. According to Table 14, if chroma_conversion_signaling_index is 0, a method of decoding the residual signals of the Cb and Cr components or a method of decoding by applying JCCR is used respectively, and the relevant legacy syntax is signaled to the video decoding device. If chroma_conversion_signaling_index is greater than or equal to 1, conversion_mode_index is added, which is an index for distinguishing different chroma space conversion methods within each implementation.

[0304] In this case, when the chroma space conversion method is applied and chroma_conversion_signaling_index is greater than or equal to 1, according to the chroma space conversion, the syntax components distinguishing the CBF values ​​of the original Cb and Cr residual signals in the related art and whether the transform skip mode is applied are replaced with syntax for the two converted signals.

[0305] The additional syntax may also vary depending on the specific mode of operation of each implementation. For implementation 1 or 2, the conversion_mode_index in Table 15 may be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, the conversion_mode_index in Table 15 may be replaced by cbcr_conversion_coefficient. As shown in Table 16, the syntax composition for this may be illustrated. Table 16 only shows the syntax composition after row 7 in Table 15.

[0306] [Table 16]

[0307]

[0308]

[0309] Compared to Table 15, the syntax may be configured in a novel manner that does not follow the syntax composition of conventional VVC techniques. For example, the syntax may be organized as shown in Table 17.

[0310] [Table 17]

[0311]

[0312]

[0313] As shown in Table 17, after the CBF values ​​of the two signals in the chroma channel are signaled, transform_skip_flag of each signal is signaled. Then, the newly introduced chroma_conversion_signaling_index may be signaled as shown in row 11 of Table 17 and below to determine the transmission method of the residual signals of the Cb and Cr components.

[0314] According to Table 14, if chroma_conversion_signaling_index is 0, a method for decoding residual signals of Cb and Cr components or a method for decoding by applying JCCR is used, respectively, and the associated legacy syntax is signaled to the video decoding device. If chroma_conversion_signaling_index is greater than or equal to 1, conversion_mode_index is added, which is an index for distinguishing different chroma space conversion methods within each implementation.

[0315] In this case, when the chroma space conversion method is applied and chroma_conversion_signaling_index is greater than or equal to 1, according to the chroma space conversion, the syntax components that distinguish the CBF values ​​of the original Cb and Cr residual signals in the prior art and whether the transform skip mode is applied are replaced with syntax for the two converted signals.

[0316] The additional syntax may also vary depending on the specific mode of operation of each implementation. For implementation 1 or 2, the conversion_mode_index in Table 13 may be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, the conversion_mode_index in Table 13 may be replaced by cbcr_conversion_coefficient. As shown in Table 16, the syntax composition used for this may be illustrated. Table 16 only shows the syntax composition after row 15 in Table 17.

[0317] <Implementation 4-1-3> Applying Implementations 1 to 3 to the step immediately preceding the existing method

[0318] In this implementation, the video decoding device may unconditionally apply the chroma space conversion methods of implementations 1 to 3 to the step immediately preceding the existing method. The video decoding device parses conversion_mode_index to distinguish different chroma space conversion methods in implementations 1, 2, and 3, as shown in Table 18.

[0319] [Table 18]

[0320]

[0321] On the other hand, the specific operation of each implementation may also specify which syntax is added. For implementation 1 or 2, conversion_mode_index in Table 18 may be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, conversion_mode_index in Table 18 may be replaced by cbcr_conversion_coefficient.

[0322] Then, the syntax is signaled in this order by the CBF values ​​tu_joint_cbcr_residual_flag of the Cb and Cr components and the transform_skip_flag of the Cb and Cr components as shown in Table 11. When chroma space conversion is applied, the syntax elements indicating the CBF values ​​of the original Cb and Cr residual signals in the conventional technology and whether the transform skip mode is applied may be replaced by the syntax of the two converted signals according to the chroma space conversion.

[0323] <Implementation 4-1-4> Selectively applying implementations 1 to 3 to the step immediately preceding the conventional method

[0324] In this implementation, the video decoding device may selectively apply the chroma space conversion methods of implementations 1 to 3 to the steps immediately before the conventional method. The video decoding device may first parse chroma_conversion_signaling_index (an index for classifying a method of signaling Cb and Cr components for each residual block as shown in Table 19) to classify a method of signaling the residual of the Cb and Cr components.

[0325] [Table 19]

[0326]

[0327] Here, the residual signal transmission method of the Cb and Cr components according to the chroma_conversion_signaling_index is shown in the above Table 14.

[0328] Then, when applying chroma space conversion according to the value of chroma_conversion_signaling_index, the video decoding device parses conversion_mode_index to distinguish different chroma space conversion methods within implementations 1, 2, and 3.

[0329] At this time, the added syntax may vary depending on the specific operation mode of each implementation. For implementation 1 or 2, conversion_mode_index in Table 19 may be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, conversion_mode_index in Table 19 may be replaced by cbcr_conversion_coefficient. The syntax composition for this may be represented as shown in Table 20.

[0330] [Table 20]

[0331]

[0332] After Table 19 or Table 20, the syntax is signaled by the Cb and Cr component CBF values ​​tu_joint_cbcr_residual_flag and the transform_skip_flag of the Cb and Cr components in this sequence, as shown in Table 11. When chroma space conversion is applied, the syntax elements indicating the CBF values ​​of the original Cb and Cr residual signals in the prior art and whether the transform skip mode is applied may be replaced by the syntax of the two converted signals according to the chroma space conversion.

[0333] <Implementation 4-2> When the Cb and Cr component signals are not residual signals

[0334] When the Cb, Cr component signals are predictor signals or original signals, the chromaticity space conversion and inverse chromaticity space conversion methods are applied at the end of the generation of each signal. For this purpose, when implementations 1 to 3 are applied, additional syntax is signaled for each implementation. On the video decoding device side, with respect to the predictor, the syntax is applied immediately after the predictor generation is completed so that the inverse chromaticity space conversion method is performed. With respect to the original signal, the syntax is applied immediately after the reconstructed signal generation is completed to perform the inverse chromaticity space conversion method. The chromaticity space conversion methods of implementations 1 to 3 can be applied in two ways.

[0335] In the first method, the video decoding device can unconditionally apply the chrominance space conversion method of implementation modes 1 to 3 to replace the traditional method. Here, the traditional method refers to the residual signal of decoding Cb and Cr components respectively. The syntax of these signals is as follows.

[0336] conversion_mode_index[xC][yC]

[0337] For example, an index conversion_mode_index is added to distinguish between different chroma space conversion methods within each implementation. The added syntax may also vary depending on the specific operation of each implementation. For implementation 1 or 2, conversion_mode_index may be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, conversion_mode_index may be replaced by cbcr_conversion_coefficient.

[0338] In the second method, in addition to the existing method, the video decoding apparatus may also apply implementations 1 to 3. In this case, a transmission method of signals of the Cb component and the Cr component may be distinguished block by block by using the above-mentioned index chroma_conversion_signaling_index.

[0339] The signaling method of Cb and Cr components according to chroma_conversion_signaling_index is shown in Table 14. In addition, for a method of applying implementations 1 to 3 in addition to the existing method, the syntax composition may be as shown in Table 21.

[0340] [Table 21]

[0341] chroma_conversion_signaling_index[xC][yC] if (chroma_conversion_signaling_index) conversion_mode_index[xC][yC]

[0342] According to Table 21, the newly introduced chroma_conversion_signaling_index is signaled to determine the transmission method of the Cb and Cr component signals. According to Table 14, if chroma_conversion_signaling_index is 0, the conventional method is used to signal the Cb and Cr component signals to the next step without using the chroma space conversion method. If chroma_conversion_signaling_index is 1 or more, the index conversion_mode_index is added to distinguish between different chroma space conversion methods within each implementation.

[0343] On the other hand, the added syntax may also vary according to the specific operation mode of each implementation. For implementation 1 or 2, conversion_mode_index in Table 21 may be replaced by cbcr_conversion_matrix[numComp]. For implementation 3, conversion_mode_index in Table 21 may be replaced by cbcr_conversion_coefficient. The syntax composition for this may be illustrated as shown in Table 22.

[0344] [Table 22]

[0345]

[0346] <Implementation 5> Signaling whether the color space conversion method is used at a higher level

[0347] This implementation is a method for allowing signaling of a method to be decided at a higher level whether to apply or not apply: whether to apply the chroma space conversion method of implementations 1 to 3, and whether to apply the syntax signaling method in implementation 4.

[0348] For example, sps_conversion_signaling_enable_flag may be signaled by a higher level video encoding device, such as a sequence parameter set (SPS), to determine whether the methods of implementations 1 to 4 will be used. If sps_conversion_signaling_enable_flag is 0, implementations 1 to 4 are not applied. The absence of this flag is inferred to be 0. If sps_conversion_signaling_enable_flag is 1, implementations 1 to 4 are applied. Without any additional conditions, sps_conversion_signaling_enable_flag may be signaled as follows.

[0349] [SPS level]

[0350] sps_conversion_signaling_enable_flag

[0351] Alternatively, sps_conversion_signaling_enable_flag may be signaled under certain conditions in the video. For example, sps_conversion_signaling_enable_flag may be signaled according to a condition after the condition is set by using chroma_format_idc, which is an index for classifying the color format of the video. The color format may be classified according to chroma_format_idc, as shown in Table 23.

[0352] [Table 23]

[0353] chroma_format_idc Color Format 0 4:0:0 1 4:2:0 2 4:2:2 3 4:4:4

[0354] For example, the decision condition for whether to transmit sps_conversion_signaling_enable_flag according to chroma_format_idc may be set as shown in Equation 26.

[0355] [Equation 26]

[0356] color_format_condition=! chroma_format_idc? 1:0

[0357] color_format_condition=chroma_format_idc>1?1:0

[0358] Here, color_format_condition is a flag indicating whether the aforementioned determination condition is satisfied. According to the first condition, if the color format is not 4:0:0, the flag is transmitted at a higher level; or, according to the second condition, if the color format is 4:2:2 or 4:4:4, the flag is transmitted at a higher level. In addition to the above example conditions, various other conditions may be set, and the resulting syntax composition is shown in Table 24.

[0359] [Table 24]

[0360] [SPS level]

[0361] if (color_format_condition) sps_conversion_signaling_enable_flag

[0362] In the following, reference Fig.16 and Fig.17 Describes a method for transforming a current chroma block and a method for inversely transforming a current chroma block.

[0363] Fig.16is a flowchart of a method for converting a current chrominance block performed by a video encoding device according to at least one embodiment of the present invention.

[0364] The video encoding apparatus obtains signals in two chroma channels of the current chroma block (S1600). Here, the signals in the two chroma channels Cb and Cr may be original signals, predictors, or residual signals of the current chroma block.

[0365] The video encoding apparatus determines conversion information in terms of optimizing encoding efficiency (S1602).

[0366] In the above implementation 1, the conversion information may be the value of the combination matrix. For the above implementation 2, the conversion information may be the value of the combination matrix and the offset matrix. For the above implementation 3, the conversion information may be the coefficient.

[0367] The video encoding apparatus applies chroma space conversion based on the conversion information to generate two conversion signals from signals in two chroma channels (S1604).

[0368] In the above-described implementation 1, the video encoding device generates two converted signals by multiplying signals in two chrominance channels by a combination matrix, as shown in Equation 14.

[0369] In the above-mentioned Implementation 2, the video encoding device multiplies the signals in the two chrominance channels by the combination matrix and adds the offset matrix to generate two converted signals, as shown in Equation 18.

[0370] For the above-mentioned embodiment 3, the video encoding device sets one chrominance signal of the signals of the two chrominance channels as one conversion signal of the two conversion signals. In addition, as shown in equation 24, the video encoding device subtracts one conversion signal of the conversion signal multiplied by the coefficient from the residual chrominance signal to generate a residual conversion signal.

[0371] The video encoding device encodes the conversion information (S1606).

[0372] In the above implementation 1, the coded conversion information may be a value of a combination matrix or an index indicating the combination matrix. Alternatively, the coded conversion information may be a value of an inverse combination matrix or an index indicating the inverse combination matrix.

[0373] If the conversion information is an index indicating a combination matrix, the video encoding apparatus may derive the index from a preset combination matrix list. If the conversion information is an index indicating an inverse combination matrix, the video encoding apparatus may derive the index from a preset list of inverse combination matrices.

[0374] In the above implementation 2, the coded conversion information may be the value of the combination matrix and the offset matrix, the index of the combination matrix and the value of the offset matrix, the value of the combination matrix and the index pointing to the offset matrix, or the index pointing to the combination matrix and the offset matrix. Alternatively, the coded conversion information may be the value of the inverse combination matrix and the offset matrix, the index of the inverse combination matrix and the value of the offset matrix, the value of the inverse combination matrix and the index pointing to the offset matrix, or the index pointing to the inverse combination matrix and the offset matrix.

[0375] If the conversion information is an index indicating an offset matrix, the video encoding apparatus may derive the index from a preset offset matrix list.

[0376] In the above implementation manner 3, the conversion information being encoded may be a coefficient or an index indicating a coefficient. If the conversion information is an index indicating a coefficient, the video encoding device may derive the index from a preset coefficient list.

[0377] Fig.17 is a flowchart of a method for inversely converting a current chrominance block performed by a video decoding device according to at least one embodiment of the present invention.

[0378] The video decoding device obtains two conversion signals (S1700). Here, the two conversion signals are generated by the chrominance space conversion of the video encoding device.

[0379] The video decoding apparatus obtains inverse conversion information (S1702). Here, the inverse conversion information corresponds to conversion information used in chroma space conversion.

[0380] In the above implementation 1, the inverse conversion information may be a value of a combination matrix or an index indicating the combination matrix. Alternatively, the inverse conversion information may be a value of an inverse combination matrix or an index indicating the inverse combination matrix.

[0381] If the inverse conversion information is an index of a reference combination matrix, the video decoding apparatus decodes the index from the bitstream and obtains the value of the combination matrix from a preset list by using the index. Alternatively, if the inverse conversion information is an index of a reference inverse combination matrix, the video decoding apparatus decodes the index from the bitstream and obtains the value of the inverse combination matrix from a preset list by using the index.

[0382] Meanwhile, after obtaining the value of the combination matrix as the inverse conversion information, the video decoding apparatus may generate an inverse combination matrix from the value of the combination matrix.

[0383] In the above implementation 2, the inverse conversion information may be the value of the combination matrix and the offset matrix, the index of the combination matrix and the value of the offset matrix, the value of the combination matrix and the index indicating the offset matrix, or the index indicating the combination matrix and the offset matrix. Alternatively, the inverse conversion information may be the value of the inverse combination matrix and the offset matrix, the index of the inverse combination matrix and the value of the offset matrix, the value of the inverse combination matrix and the index indicating the offset matrix, or the index indicating the inverse combination matrix and the offset matrix.

[0384] If the inverse conversion information is an index indicating an offset matrix, the video decoding apparatus decodes the index from a bitstream and obtains a value of the offset matrix from a preset list by using the index.

[0385] In the above implementation manner 3, the inverse conversion information may be a coefficient or an index indicating the coefficient.

[0386] If the inverse transform information is an index indicating a coefficient, the video decoding apparatus decodes the index from a bitstream and obtains a value of the coefficient from a preset list by using the index.

[0387] The video decoding device applies inverse chroma space conversion based on the inverse conversion information to generate signals in two chroma channels of the current chroma block from the two converted signals (S1704). Here, the signals in the two chroma channels Cb, Cr can be the original signal, predictor, or residual signal of the current chroma block.

[0388] For implementation 1 described above, the video decoding device generates signals in two chrominance channels by multiplying the two converted signals by the inverse combination matrix, as shown in Equation 16.

[0389] For the above-mentioned implementation method 2, the video decoding device generates signals in two chrominance channels by multiplying the two converted signals by the inverse combination matrix and adding the offset matrix, as shown in Equation 20.

[0390] For the above-mentioned implementation 3, the video decoding device sets one of the two conversion signals as one chrominance signal among the signals in the two chrominance channels, as shown in Equation 25. In addition, the video decoding device generates a remaining chrominance signal by multiplying the converted signal by a coefficient and adding the remaining converted signal.

[0391] Although the steps in each flowchart are described as being performed sequentially, these steps are merely illustrative of the technical concepts of some embodiments of the present disclosure. Therefore, a person of ordinary skill in the art to which the present disclosure belongs can perform the steps by changing the order described in each figure or by performing two or more steps in parallel. Therefore, the steps in each flowchart are not limited to the time series order shown.

[0392] It should be understood that the above description presents illustrative embodiments that can be implemented in different other ways. The functions described in some embodiments can be implemented by hardware, software, firmware and / or a combination thereof. It should also be understood that the functional components described in this disclosure are marked with "... units" to strongly emphasize the possibility of their independent implementation.

[0393] At the same time, the various methods or functions described in some embodiments may be implemented as instructions stored in a non-transitory recording medium that can be read and executed by one or more processors. For example, the non-transitory recording medium may include various types of recording devices in which data is stored in a form readable by a computer system. For example, the non-transitory recording medium may include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0394] Although the embodiments of the present disclosure have been described for illustrative purposes, however, those of ordinary skill in the art to which the present disclosure belongs should recognize that various modifications, additions, and substitutions are possible without departing from the concept and scope of the present disclosure. Therefore, for the sake of brevity and clarity, the embodiments of the present disclosure have been described. The scope of the technical concept of the embodiments of the present disclosure is not limited by the diagrams. Therefore, those of ordinary skill in the art to which the present disclosure belongs should understand that the scope of the present disclosure should not be limited by the embodiments explicitly described above, but by the claims and their equivalents.

[0395] (reference number)

[0396] 120: Predictor

[0397] 155: Entropy Encoder

[0398] 510: Entropy Decoder

[0399] 540: Predictor

[0400] 810: CSC

[0401] 910: ICSC

[0402] 1010: CSC

[0403] 1110: CSC

[0404] CROSS-REFERENCE TO RELATED APPLICATIONS

[0405] This application claims priority to and the benefit of Korean Patent Application No. 10-2022-0029104, filed on March 8, 2022, and Korean Patent Application No. 10-2023-0021661, filed on February 17, 2023, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method for inversely converting a current chrominance block performed by a video decoding device, the method include: Obtaining two conversion signals generated by a video encoding device through color space conversion; Obtaining inverse conversion information corresponding to the conversion information used for the color space conversion; as well as Signals in two chroma channels of the current chroma block are generated from the two converted signals by applying an inverse chroma space conversion based on the inverse conversion information.

2. The method according to claim 1, in, The signals in the two chrominance channels include: The original signal of the current chroma block, the predictor of the current chroma block, or the residual signal of the current chroma block.

3. The method according to claim 1, in, Obtaining the inverse conversion information includes: An inverse combination matrix is ​​obtained as the inverse conversion information, wherein the inverse combination matrix is ​​an inverse matrix of the combination matrix as the conversion information.

4. The method according to claim 3, in, Obtaining the inverse conversion information includes: decoding an index indicating the combination matrix from a bitstream; Obtaining the value of the combination matrix from a preset list by using the index; and The inverse combination matrix is ​​calculated based on the values ​​of the combination matrix.

5. The method according to claim 3, in, Obtaining the inverse conversion information includes: Obtaining the value of the combination matrix from the bit stream; and The inverse combination matrix is ​​calculated based on the values ​​of the combination matrix.

6. The method according to claim 3, in, Obtaining the inverse conversion information includes: decoding an index indicating the inverse combining matrix from a bitstream; and The value of the inverse combination matrix is ​​obtained from a preset list by using the index.

7. The method according to claim 3, in, Obtaining the inverse conversion information includes: The value of the inverse combining matrix is ​​obtained from the bit stream.

8. The method according to claim 1, in, Obtaining the inverse conversion information includes: An inverse combination matrix and an offset matrix are obtained as the inverse conversion information, wherein the inverse combination matrix is ​​an inverse matrix of the combination matrix in the conversion information, and the offset matrix is ​​included in the conversion information for the chroma space conversion.

9. The method according to claim 8, in, Obtaining the inverse conversion information includes: decoding an index indicating the offset matrix from a bitstream; and The value of the offset matrix is ​​obtained from a preset list by using the index.

10. The method according to claim 8, in, Obtaining the inverse conversion information includes: The values ​​of the offset matrix are decoded from the bitstream.

11. The method according to claim 8, in, Generating the signals in the two chrominance channels comprises: The two converted signals are multiplied by the inverse combining matrix and added with the offset matrix to generate the signals in the two chrominance channels.

12. The method according to claim 1, in, Obtaining the inverse conversion information includes: A coefficient included in the conversion information used for the color space conversion is obtained as the inverse conversion information.

13. The method according to claim 12, in, Obtaining the inverse conversion information includes: decoding from a bitstream an index indicating the coefficient; The value of the coefficient is obtained by using the index.

14. The method according to claim 12, in, Obtaining the inverse conversion information includes: The values ​​of the coefficients are decoded from the bitstream.

15. The method according to claim 12, in, Generating the signals in the two chrominance channels comprises: One of the two conversion signals is set as one chrominance signal among the signals in the two chrominance channels, the one conversion signal is multiplied by the coefficient, and is added to the remaining conversion signal to generate a remaining chrominance signal.

16. A method for converting a current chrominance block performed by a video encoding device, the method include: Obtain signals in two chroma channels of the current chroma block; Determine conversion information; generating two transformed signals from the signals in the two chrominance channels by applying a chrominance space transformation based on the transformation information; as well as The conversion information is encoded.

17. The method according to claim 16, in, The signals in the two chrominance channels include: The original signal of the current chroma block, the predictor of the current chroma block, or the residual signal of the current chroma block.

18. The method according to claim 16, in, The conversion information includes: A combining matrix, an offset matrix having the combining matrix, or coefficients for applying the color space conversion.

19. A computer-readable recording medium storing a bit stream generated by a video encoding method, the video encoding method include: Obtain signals in two chroma channels of the current chroma block; Determine conversion information; generating two transformed signals from the signals in the two chrominance channels by applying a chrominance space transformation based on the transformation information; as well as The conversion information is encoded.