Chroma block prediction method and device

Abstract

To provide a chroma block prediction method and a device.SOLUTION: A prediction method includes: obtaining chroma values of chroma samples at preset locations from neighboring samples of a chroma block; obtaining, on the basis of neighboring samples of a luma block corresponding to the chroma block, luma values of luma samples corresponding to the chroma samples; classifying the obtained luma values into a first luma set and a second luma set; grouping the chroma values into a first chroma set and a second chroma set; determining a scaling coefficient in a linear model on the basis of the average value of luma values in the first luma set and in the second luma set, the average value of chroma values in the first chroma set and in the second chroma set; determining, on the basis of the scaling coefficient, an offset factor in a linear model corresponding to the chroma block; and determining prediction information of the chroma block on the basis of the scaling coefficient, the offset factor, and luma reconstruction information corresponding to the chroma block.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] This application claims the priority of Chinese Patent Application No. 201910005667.3, titled "Chrominance Block Prediction Method and Apparatus", filed on January 3, 2019, and incorporates it herein by reference in its entirety.

[0002] This application relates to the field of video encoding and decoding technologies, and in particular, to a chrominance block prediction method and apparatus.

Background Art

[0003] With the development of Internet technology, the amount of video application programs has been increasing, and video application programs are imposing an increasing amount of requirements on high-definition video. However, since the data volume of high-definition video is relatively large, when it is necessary to transmit high-definition video within a limited network bandwidth, it is necessary to encode the high-definition video. Generally, the encoding process mainly includes intra prediction, inter prediction, transformation, quantization, entropy encoding, intra-loop filtering, and the like.

[0004] In related technologies, when it is necessary to perform intra prediction, the prediction information of a chroma block can be determined by using the Cross Component Linear Mode (CCLM) (which may also be referred to as the Cross-Component Prediction (CCP) mode, or the Cross-Component Intra Prediction (CCIP) mode, or simply the linear mode (LM)). This is a chroma intra prediction method that uses the correlation between luminance and chrominance. In this method, the prediction information of the current chroma block is derived according to a linear model using the reconstructed luma component, which is the following formula: [Number] can be represented using α and β. Here, α and β are intra prediction model parameters, α is a scaling coefficient, β is an offset factor, and pred C (i,j) is the predictor of the chroma sample at position (i,j), and rec i L (i,j) is the luma reconstruction sample value at position (i,j) after the luma reconstruction block corresponding to the current chroma block is downsampled to the chroma component resolution. The scaling coefficient and the offset factor do not need to be encoded for transmission and are derived by using the edge samples of the adjacent reconstruction blocks of the current chroma block and the luma samples corresponding to those edge samples. One method is: [Number] It can be expressed as follows. Here, N is the number of edge samples of the adjacent reconstruction blocks, L(n) is the n-th luma sample, and C(n) is the n-th chroma sample.

[0005] Thus, for each chroma block where CCLM is used, it is necessary to determine the scaling coefficient and the offset factor. However, the above determination process requires a large number of multiplications and is relatively complex. As a result, the prediction efficiency of chroma blocks is relatively low. SUMMARY OF THE INVENTION

[0006] To solve the problems in the related art, embodiments of this application provide a chroma block prediction method and apparatus. The technical solution is as follows.

[0007] According to a first aspect, a chroma block prediction method is provided. The method includes: Obtain the chroma value of the chroma sample at the preset position from adjacent samples of the chroma block, obtain the luma value of the luma sample corresponding to the chroma sample at the preset position based on adjacent samples of the luma block corresponding to the chroma block, classify the obtained luma values into a first luma set and a second luma set, group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the first luma set into a first chroma set, group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the second luma set into a second chroma set, determine a scaling coefficient in the linear model corresponding to the chroma block based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set, determine an offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, and determine prediction information of the chroma block based on the scaling coefficient, the offset factor, and luma reconstruction information corresponding to the chroma block, including this.

[0008] In the solution shown in this embodiment of this application, when it is necessary to perform intra prediction on the current chroma block (hereinafter sometimes abbreviated as the chroma block), the chroma value of the chroma sample at the preset position can be obtained from the adjacent samples of the chroma block. Then, the luma block corresponding to the chroma block can be determined, and based on the adjacent samples of the luma block, the luma value of the luma sample corresponding to the chroma sample at the preset position can be obtained. The obtained luma values are classified into a first luma set and a second luma set. Based on the luma values, the chroma values are automatically classified into a first chroma set and a second chroma set. Then, based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set, the scaling coefficient in the linear model corresponding to the chroma block is determined. Then, based on the scaling coefficient, the offset factor in the linear model corresponding to the chroma block is determined. Finally, based on the luma reconstruction information, the scaling coefficient, and the offset factor corresponding to the chroma block, the prediction information of the chroma block is obtained, and the luma reconstruction information corresponding to the chroma block includes the downsampling information of the luma reconstruction block corresponding to the chroma block.

[0009] Thus, only a smaller number of multiplications are required, so the computational complexity can be reduced, and furthermore, the chroma block prediction efficiency can be improved.

[0010] In one possible implementation, the method further includes obtaining instruction information, determining the intra prediction mode corresponding to the chroma block based on the instruction information, the intra prediction mode including at least one of a linear mode LM, a linear mode above (LMA), and a linear mode left (LML), and determining the preset position based on the intra prediction mode corresponding to the chroma block.

[0011] In the solution shown in this embodiment of this application, when performing encoding, the encoder side can encode the indication information into a bitstream and then send the bitstream to the decoder side. The decoder side can obtain the indication information from the received bitstream and determine the intra prediction mode corresponding to the chroma block based on the indication information. Then, the decoder side determines the preset position based on the intra prediction mode corresponding to the chroma block. In this way, the decoder side can determine the intra prediction mode used for the chroma block and then determine the preset position based on the intra prediction mode.

[0012] In one possible implementation, obtaining the chroma value of the chroma sample at the preset position from the adjacent samples of the chroma block includes obtaining the chroma value of the chroma sample at the preset position from the adjacent samples of the chroma block based on a predetermined correspondence between the intra prediction mode and the preset position and the intra prediction mode corresponding to the chroma block.

[0013] In the solution shown in this embodiment of this application, when performing decoding, the decoder side can obtain a predetermined correspondence between the intra prediction mode and the position and determine the preset position corresponding to the intra prediction mode of the chroma block from the correspondence. Then, the decoder side obtains the chroma value of the chroma sample at the preset position from the adjacent samples of the chroma block based on the preset position.

[0014] In this way, different preset positions can be selected for different intra prediction modes, so that the chroma block prediction result can be made more accurate.

[0015] In a possible implementation, the position coordinates of the upper adjacent samples of the chroma block are (0, -1), (1, -1),..., and (X - 1, -1), the position coordinates of the left adjacent samples of the chroma block are (-1, 0), (-1, 1),..., and (-1, Y - 1), X represents the width of the chroma block, Y represents the height of the chroma block, and when the intra prediction mode corresponding to the chroma block is the cross-component linear mode LM, the position coordinates of the chroma samples at the preset positions are (0, -1), (X - 1, -1), (-1, -0), and (-1, Y - 1).

[0016] In the solution shown in this embodiment of this application, correspondingly, a rectangular plane coordinate system is established for the chroma block. The first chroma sample at the upper left corner of the chroma block is used as the coordinate origin, the horizontal direction to the right is the positive direction of the x-axis, and the vertical direction downward is the positive direction of the y-axis. Thus, the position coordinates of the upper adjacent samples of the chroma block are (0, -1), (1, -1),..., and (X - 1, -1), the position coordinates of the left adjacent samples of the chroma block are (-1, 0), (-1, 1),..., and (-1, Y - 1), X represents the width of the chroma block (i.e., the number of samples in the width direction of the chroma block), Y represents the height of the chroma block (i.e., the number of samples in the height direction of the chroma block). When the intra prediction mode is LM, the position coordinates of the chroma samples at the preset positions are (0, -1), (X - 1, -1), (-1, -0), and (-1, Y - 1).

[0017] In a possible implementation, the position coordinates of the upper adjacent samples of the chroma block are (0, -1), (1, -1),..., and (X - 1, -1), X represents the width of the chroma block, and when the intra prediction mode corresponding to the chroma block is the cross-component linear mode upper LMA, the position coordinates at the preset positions are (0, -1), (X / 4, -1), (X - 1 - X / 4, -1), and (X - 1, -1), or (0, -1), (X / 4, -1), (3×X / 4, -1), and (X - 1, -1).

[0018] In the solution shown in this embodiment of this application, correspondingly, a rectangular plane coordinate system is established for the chroma block. The first chroma sample at the upper left corner of the chroma block is used as the coordinate origin, the horizontal direction to the right is the positive direction of the x-axis, and the vertical direction downward is the positive direction of the y-axis. When the intra prediction mode is LMA, the position coordinates of the preset positions are (0, -1), (X / 4, -1), (X - 1 - X / 4, -1), and (X - 1, -1), or (0, -1), (X / 4, -1), (3×X / 4, -1), and (X - 1, -1), where X represents the width of the chroma block (i.e., the number of samples in the width direction of the chroma block).

[0019] In one possible implementation, the position coordinates of the left adjacent sample of the chroma block are (-1, 0), (-1, 1), …, and (-1, Y - 1), where Y represents the height of the chroma block. When the intra prediction mode corresponding to the chroma block is cross-component linear mode left LML, the position coordinates of the preset positions are (-1, 0), (-1, Y / 4), (-1, Y - 1 - Y / 4), and (-1, Y - 1), or (-1, 0), (-1, Y / 4), (-1, 3×Y / 4), and (-1, Y - 1).

[0020] In the solution shown in this embodiment of this application, correspondingly, a rectangular plane coordinate system is established for the chroma block. The first chroma sample at the upper left corner of the chroma block is used as the coordinate origin, the horizontal direction to the right is the positive direction of the x-axis, and the vertical direction downward is the positive direction of the y-axis. When the intra prediction mode is LML, the position coordinates of the preset positions are (-1, 0), (-1, Y / 4), (-1, Y - 1 - Y / 4), and (-1, Y - 1), or (-1, 0), (-1, Y / 4), (-1, 3×Y / 4), and (-1, Y - 1), where Y represents the height of the chroma block (i.e., the number of samples in the height direction of the chroma block).

[0021] In one possible implementation, obtaining the luma value of the luma sample corresponding to the chroma sample at the preset position based on the adjacent samples of the luma block corresponding to the chroma block includes determining, from the adjacent samples of the luma block, that the position coordinates of the luma sample corresponding to the chroma sample at the i-th preset position within the preset position are (2×X i , 2×Y i ), where the position coordinates of the chroma sample at the i-th preset position are (X i , Y i ), and obtaining the luma value of the luma sample corresponding to the chroma sample at the preset position from the adjacent samples of the luma block corresponding to the chroma block based on the position coordinates of the luma sample corresponding to the chroma sample at the preset position; or determining, from the adjacent samples of the luma block, the position coordinates of a plurality of samples corresponding to the j-th preset position based on the position coordinates of the chroma sample at the j-th preset position within the preset position, determining the position coordinates of the luma sample corresponding to the chroma sample at the j-th preset position based on the position coordinates of the plurality of samples, and obtaining the luma value of the luma sample corresponding to the chroma sample at the preset position from the adjacent samples of the luma block corresponding to the chroma block based on the position coordinates of the luma sample corresponding to the chroma sample at the preset position.

[0022] In the solution shown in this embodiment of this application, when the chroma block is decoded, it can be determined that the position coordinates of the chroma sample at the i-th preset position within the preset position are (X i , Y i ), and the position coordinates of the luma sample corresponding to the chroma sample at the i-th preset position are (2×X i , 2×Y i ) from the adjacent samples of the luma block. Thus, the position coordinates of the luma sample corresponding to the chroma sample at the preset position can be obtained.

[0023] Alternatively, when the chroma block is decoded, the position coordinates of the chroma sample at the j-th preset position within the preset position are (X j , Y j ). For the chroma sample at the j-th preset position, a plurality of sample positions of the adjacent samples of the luma block corresponding to (X j , Y j ) are determined, and then, based on the position coordinates of the plurality of sample positions, the position coordinates of the luma sample corresponding to the chroma sample at the j-th preset position are obtained. For example, a weighting method may be used. The position coordinates of the plurality of sample positions are (2 × X j , 2 × Y j ), (2 × X j , 2 × Y j + 1), (2 × X j + 1, 2 × Y j ), (2 × X j + 1, 2 × Y j + 1), (2 × X j + 2, 2 × Y j ), and (2 × X j + 2, 2 × Y j + 1), and all these sample positions correspond to weight values of 2 / 8, 1 / 8, 1 / 8, 2 / 8, 1 / 8, and 1 / 8, respectively. After weighting, the position coordinates (2 × X j , 2 × Y j + 0.5) of the luma sample corresponding to the chroma sample at the j-th preset position can be obtained. Also, the weighting method may not be used. This is not limited in this embodiment of this application. Thus, the position coordinates of the luma sample corresponding to the chroma sample at the preset position can be obtained. And based on the position coordinates of the luma sample corresponding to the chroma sample at the preset position, the luma value can be obtained from the corresponding luma sample.

[0024] Thus, the luma value can be determined more quickly.

[0025] In one possible implementation, the indication information is obtained from the received bitstream.

[0026] In the solution shown in this embodiment of this application, when performing encoding, the encoder side encodes instruction information indicating the intra prediction mode into the bitstream and then can send the bitstream to the decoder side. The decoder side can obtain the instruction information from the received bitstream.

[0027] In this way, the decoder side can determine the intra prediction mode used for the chroma block.

[0028] In one possible implementation, classifying the obtained luma values into the first luma set and the second luma set includes determining an average value of the luma values of the luma samples corresponding to the chroma samples at the preset position, grouping luma values that are among the luma values of the luma samples corresponding to the chroma samples at the preset position and are less than or equal to the average value of the luma values into the first luma set, and grouping luma values that are among the luma values of the luma samples corresponding to the chroma samples at the preset position and are greater than the average value of the luma values into the second luma set.

[0029] In the solution shown in this embodiment of this application, luma values that are among all the obtained luma values and are less than or equal to the average value of the luma values are determined, and these luma values are grouped into the first luma set. Further, luma values that are among all the obtained luma values and are greater than the average value of the luma values can be determined, and these luma values are grouped into the second luma set. In this way, the luma values in the first luma set are smaller than the luma values in the second luma set.

[0030] In this way, the prediction speed of the chroma block can be increased without degrading the encoding performance.

[0031] In one possible implementation, classifying the obtained luma values into a first luma set and a second luma set includes sorting the luma values of the luma samples corresponding to the chroma samples at the preset position in ascending order to obtain a first luma value queue, and when the number of luma samples in the first luma value queue is even, grouping the luma values in the first half of the first luma value queue into the first luma set and grouping the luma values in the second half of the luma value queue into the second luma set, or sorting the luma values of the luma samples corresponding to the chroma samples at the preset position in descending order to obtain a second luma value queue, and when the number of luma samples in the second luma value queue is even, grouping the luma values in the second half of the second luma value queue into the first luma set and grouping the luma values in the first half of the luma value queue into the second luma set.

[0032] In the solution shown in this embodiment of this application, the obtained luma values are sorted in ascending order to obtain a first luma value queue. When the number of luma samples in the first luma value queue is even, the luma values in the first half of the first luma value queue can be determined and grouped into the first luma set, and further, the luma values in the second half of the first luma value queue can be determined and grouped into the second luma set. Alternatively, the obtained luma values are sorted in descending order to obtain a second luma value queue. When the number of luma samples in the second luma value queue is even, the luma values in the second half of the second luma value queue can be determined and grouped into the first luma set, and further, the luma values in the first half of the second luma value queue can be determined and grouped into the second luma set.

[0033] In this way, the chroma block prediction speed can be increased without degrading the encoding performance.

[0034] In one possible implementation, determining the scaling coefficient in the linear model corresponding to the chroma block based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set is such that α = (C Lmean - C Rmean ) / (L Lmean - L Rmean ), where α is the scaling coefficient in the linear model corresponding to the chroma block, C Lmean is the average value of the chroma values in the first chroma set, C Rmean is the average value of the chroma values in the second chroma set, L Lmean is the average value of the luma values in the first luma set, and L Rmean is the average value of the luma values in the second luma set.

[0035] In one possible implementation, determining the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient includes determining the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, the average value of the chroma values in the first chroma set, and the average value of the luma values in the first luma set.

[0036] In the solution shown in this embodiment of this application, the average value of the chroma values in the first chroma set can be determined, and the average value of the luma values in the first luma set can be determined. Then, based on the scaling coefficient, the average value of the chroma values in the first chroma set, and the average value of the luma values in the first luma set, the offset factor in the linear model corresponding to the chroma block is determined.

[0037] In one possible implementation, determining the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, the average value of the chroma values in the first chroma set, and the average value of the luma values in the first luma set is β = C Lmean -α * L Lmean where α is the scaling coefficient, β is the offset factor in the linear model corresponding to the chroma block, C Lmean is the average value of the chroma values in the first chroma set, and L Lmean is the average value of the luma values in the first luma set.

[0038] In one possible implementation, determining the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient includes determining the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, the average value of the chroma values of the chroma samples at the preset position, and the average value of the luma values of the luma samples corresponding to the chroma samples.

[0039] In the solution shown in this embodiment of this application, the average value of all the obtained chroma values can be determined, and the average value of all the obtained luma values can be determined. Then, based on the scaling coefficient, the average value of all the chroma values, and the average value of all the luma values, the offset factor in the linear model corresponding to the chroma block is determined.

[0040] In one possible implementation, determining the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, the average value of the chroma values of the chroma samples at the preset position, and the average value of the luma values of the luma samples corresponding to the chroma samples is β = C mean -α * L meanincluding being, where α is the scaling coefficient, β is the offset factor in the linear model corresponding to the chroma block, and C mean is the average value of the chroma values of the chroma samples at the preset position, and L mean is the average value of the luma values of the luma samples corresponding to the chroma samples at the preset position.

[0041] According to a second aspect, a chroma block prediction apparatus is provided. The prediction apparatus includes a processor and a memory, the memory is configured to store processor-executable instructions, and the processor executes the instructions to perform the chroma block prediction method provided in the first aspect.

[0042] According to a third aspect, a chroma block prediction apparatus is provided. The apparatus includes one or more modules configured to implement the chroma block prediction method provided in the first aspect.

[0043] According to a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the computer-readable storage medium is executed on a computing device, the computing device is enabled to perform the chroma block prediction method provided in the first aspect.

[0044] According to a fifth aspect, a computer program product including instructions is provided. When the computer program product is executed on a computing device, the computing device is enabled to perform the chroma block prediction method provided in the first aspect.

[0045] The beneficial effects brought by the technical solution provided in the embodiments of this application include at least the following.

[0046] In an embodiment of this application, during encoding or decoding, the chroma values of the chroma samples at the preset positions can be obtained from the adjacent samples of the chroma block. Next, the luma values of the luma samples corresponding to the chroma samples at the preset positions are obtained from the adjacent samples of the luma block corresponding to the chroma block, and the obtained luma values are classified into a first luma set and a second luma set. Based on the classification of the luma values, the corresponding chroma values are classified into a first chroma set and a second chroma set. Next, based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set, the scaling coefficient in the linear model corresponding to the chroma block can be determined. And after the scaling coefficient is determined, based on the scaling coefficient, the offset factor in the linear model corresponding to the chroma block can be determined. Finally, based on the scaling coefficient, the offset factor, and the luma reconstruction information corresponding to the chroma block, the prediction information of the chroma block is determined. Thus, in encoding or decoding, a preset number of chroma values are selected and then classified into two chroma sets, and a preset number of luma values are selected and then classified into two luma sets. Based on the average value of the luma values in each luma set and the average value of the chroma values in each chroma set, the scaling coefficient in the linear model corresponding to the chroma block is determined, and further, the offset factor is determined. Since only a smaller number of multiplications are required, the encoding time and the decoding time can be shortened.

Brief Description of the Drawings

[0047]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14(a)

Figure 14(b)

Figure 15(a)

Figure 15(b)

Figure 16(a)

Figure 16(b)

Figure 16(c)

Figure 16(d)

Figure 17

Embodiments for Carrying Out the Invention

[0048] To make the purpose, technical solution, and advantages of this application clearer, the implementation of this application will be further described in detail below with reference to the accompanying drawings.

[0049] To facilitate the understanding of this application, first, the system architecture and the concepts of the terms of this application will be described below.

[0050] Video coding usually processes a series of pictures that form a video or a video sequence. In the field of video coding, the terms "picture", "frame", and "image" can be used as synonyms. The video coding used in this application (or this disclosure) refers to video encoding or video decoding. Video encoding is performed on the source side and usually includes processing the original video picture (e.g., through compression) to reduce the amount of data required to represent the video picture for more efficient storage and / or transmission. Video decoding is performed on the destination side and usually includes performing the reverse process on the encoder to reconstruct the video picture. The "encoding" of the video picture in the embodiment should be understood as "encoding" or "decoding" related to the video sequence. The combination of encoding and decoding is also referred to as coding (encoding and decoding).

[0051] Each picture in a video sequence is typically divided into a set of non-overlapping blocks, and coding is usually performed at the block level. Specifically, on the encoder side, the video is typically processed, i.e., encoded, at the block level (which is also referred to as a picture block or a video block). For example, prediction blocks are generated through spatial (intra-picture) prediction and temporal (inter-picture) prediction, and the residual block is obtained by subtracting the prediction block from the current block (the block being processed or to be processed), and the residual block is transformed and quantized in the transform domain, reducing the amount of data to be transmitted (compressed). On the decoder side, the inverse process to that of the encoder is applied to the encoded or compressed block, and the current block is reconstructed for presentation. Further, the encoder replicates the decoder's processing loop so that the encoder and decoder generate the same prediction (e.g., intra prediction and inter prediction) and / or reconstruction for processing, i.e., encoding, subsequent blocks.

[0052] The term "block" can be part of a picture or a frame. The key terms in this application are defined as follows.

[0053] Current block: The current block is the block being processed. For example, in encoding, the current block is the block being encoded, and in decoding, the current block is the block being decoded. If the block being processed is a chroma component block, that block is called the current chroma block. The luma block corresponding to the current chroma block can be called the current luma block.

[0054] Reference block: A reference block is a block that provides a reference signal for the current block. In the search process, multiple reference blocks can be considered to find the optimal reference block.

[0055] Prediction block: A block that provides a prediction for the current block is called a prediction block. For example, after considering a plurality of reference blocks, an optimal reference block is found. The optimal reference block provides a prediction for the current block, and that block is called a prediction block.

[0056] Picture block signal: A picture block signal is a sample value, sampling value, or sampling signal within a picture block.

[0057] Prediction signal: A sample value, sampling value, or sampling signal within a prediction block is called a prediction signal.

[0058] Hereinafter, embodiments of the encoder 20, decoder 30, and encoding system 10 will be described based on FIGS. 1 and 2 to 4.

[0059] FIG. 1 is a conceptual or schematic block diagram showing an example of an encoding system 10, such as a video encoding system 10 that can use the technology of this application (this disclosure). The encoder 20 (e.g., video encoder 20) and decoder 30 (e.g., video decoder 30) within the video encoding system 10 represent example devices that can be configured to perform intra prediction according to various examples described in this application. As shown in FIG. 1, the encoding system 10 includes a source device 12 configured to provide encoded data 13, such as encoded picture 13, to a destination device 14 or the like for decoding the encoded data 13.

[0060] The source device 12 includes an encoder 20 and may additionally or optionally include a picture source 16, a preprocessing unit 18 such as a picture preprocessing unit 18, and a communication interface or communication unit 22.

[0061] The picture source 16 can include, or can be, any type of picture capture device configured to capture real-world pictures and the like, and / or any type of device that generates pictures or comments (for screen content encoding, and some text on the screen can also be considered as the picture or part of the picture to be encoded), such as a computer graphics processing unit configured to generate computer animation pictures, or any type of device configured to acquire and / or provide real-world pictures or computer animation pictures (e.g., screen content or virtual reality (VR) pictures) and / or any combination thereof (e.g., augmented reality (AR) pictures).

[0062] A picture can be regarded as a two-dimensional array or matrix of samples having luminance values. Samples within the array can also be referred to as pixels (short for picture elements) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. For color representation, usually three color components are used, that is, a picture can be represented as or can include three sample arrays. In the RGB format or color space, a picture includes the corresponding red, green, and blue sample arrays. However, in video coding, each sample is usually represented in a luminance / chrominance format or color space. For example, an image in the YCbCr format includes a luminance component (sometimes denoted by L) indicated by Y and two chrominance components indicated by Cb and Cr. The luminance (luma for short) component Y indicates the brightness or gray-level intensity (for example, in a grayscale picture these two are the same), and the two chrominance (chroma for short) components Cb and Cr represent components of chrominance or color information. Thus, a picture in the YCbCr format includes a luminance sample array of luminance sample values (Y) and two chrominance sample arrays of chrominance values (Cb and Cr). A picture in the RGB format may be converted to a picture in the YCbCr format, and vice versa. This process is also called color transformation or color conversion. If a picture is monochrome, the picture may include only a luminance sample array.

[0063] The picture source 16 (e.g., the video source 16) can be, for example, a camera configured to capture pictures, a memory such as a picture memory that includes or stores previously captured or generated pictures, and / or any type of (internal or external) interface for acquiring or receiving pictures. The camera can be, for example, a local camera or an integrated camera integrated into the source device, and the memory can be a local memory or an integrated memory integrated into the source device. The interface can be, for example, an external interface for receiving pictures from an external video source. The external video source can be, for example, an external picture capture device such as a camera, an external memory, or an external picture generation device. The external picture generation device can be, for example, an external computer graphics processing unit, a computer, or a server. The interface can be any type of interface according to any unique or standardized interface protocol, such as a wired or wireless interface or an optical interface. The interface for acquiring the picture data 17 can be the same interface as the communication interface 22 or, alternatively, a part of the communication interface 22.

[0064] Unlike the preprocessing unit 18 and the processing executed by the preprocessing unit 18, the picture 17 or the picture data 17 (e.g., the video data 16) can also be referred to as the original picture 17 or the original picture data 17.

[0065] The preprocessing unit 18 is configured to obtain the preprocessed picture 19 or the preprocessed picture data 19 by receiving the (original) picture data 17 and performing preprocessing on the picture data 17. For example, the preprocessing executed by the preprocessing unit 18 can include trimming, color format conversion (e.g., from RGB to YCbCr), color correction, or noise reduction. It can be understood that the preprocessing unit 18 can be an optional component.

[0066] The encoder 20 (e.g., video encoder 20) is configured to receive the preprocessed picture data 19 and provide encoded picture data 21 (details will be further described later, for example, based on FIG. 3 or FIG. 5). In one example, the encoder 20 may be configured to execute the following Embodiments 1 to 7.

[0067] The communication interface 22 of the source device 12 is configured to receive the encoded picture data 21 and transmit the encoded picture data 21 to another device, such as the destination device 14 or some other device, for storage or direct reconstruction, or may be configured to process the encoded picture data 21 before correspondingly storing the encoded data 13 and / or transmitting the encoded data 13 to another device. The other device is, for example, the destination device 14 or any other device used for decoding or storage.

[0068] The destination device 14 includes a decoder 30 (e.g., video decoder 30) and may additionally or optionally include a communication interface or communication unit 28, a post-processing unit 32, and a display device 34.

[0069] For example, the communication interface 28 of the destination device 14 is configured to directly receive the encoded picture data 21 or the encoded data 13 from the source device 12 or any other source. Any other source is, for example, a storage device, and the storage device is, for example, an encoded picture data storage device.

[0070] The communication interfaces 22 and 28 may be configured to transmit or receive the encoded picture data 21 or the encoded data 13 on a direct communication link between the source device 12 and the destination device 14 or on any type of network. The direct communication link may be, for example, a direct wired or wireless connection, and any type of network may be, for example, a wired or wireless network or any combination thereof, or any type of private network or public network or any combination thereof.

[0071] The communication interface 22 may be configured to encapsulate the encoded picture data 21, for example, into a suitable format such as a packet, for transmission on a communication link or a communication network.

[0072] The communication interface 28 as the corresponding part of the communication interface 22 may be configured to, for example, decompose the encapsulation of the encoded data 13 to obtain the encoded picture data 21.

[0073] Both the communication interface 22 and the communication interface 28 may be configured as unidirectional communication interfaces. For example, the arrow pointing from the source device 12 to the destination device 14 is used for the encoded picture data 13 in FIG. 1, or may be configured as a bidirectional communication interface. For example, it may be configured to send and receive messages to establish a connection and to check and exchange other information related to data transmission such as, for example, encoded picture data transmission on the communication link and / or.

[0074] The decoder 30 is configured to receive the encoded picture data 21 and provide decoded picture data 31 or decoded picture 31 (details will be further described later, for example, based on FIG. 4 or FIG. 6). In one example, the decoder 30 may be configured to execute the following Embodiments 1 to 7.

[0075] The post - processing unit 32 of the destination device 14 is configured to post - process decoded picture data 31 (also referred to as reconstructed picture data), such as decoded picture 131, to obtain post - processed picture data 33, such as post - processed picture 33. The post - processing executed by the post - processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, trimming, resampling, or any other processing for preparing the decoded picture data 31 for display by the display device 34.

[0076] The display device 34 of the destination device 14 is configured to receive the post - processed picture data 33 and display the picture to a user, viewer, or the like. The display device 34 can be any type of display configured to present the reconstructed picture, such as an integrated or external display or monitor, or can include it. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro - LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display.

[0077] FIG. 1 depicts the source device 12 and the destination device 14 as separate devices, but the device embodiments may also include both the source device 12 and the destination device 14, or both the functions of the source device 12 and the functions of the destination device 14, i.e., both the source device 12 or corresponding functions, and the destination device 14 or corresponding functions. In such embodiments, the source device 12 or corresponding functions, and the destination device 14 or corresponding functions may be implemented using the same hardware and / or software, separate hardware and / or software, or some combination thereof.

[0078] Based on these descriptions, those skilled in the art can easily understand that the existence and (exact) division of multiple / one function or different unit functions of the source device 12 and / or destination device 14 shown in FIG. 1 may vary depending on the actual device and application.

[0079] The encoder 20 (e.g., video encoder 20) and the decoder 30 (e.g., video decoder 30) can each be implemented as any one of a variety of suitable circuits, such as, for example, one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology is implemented partially in software, the apparatus can store software instructions in a suitable non-transitory computer-readable storage medium and execute the instructions in hardware using one or more processors to perform the technology in this disclosure. Any of the above (including hardware, software, combinations of hardware and software, and the like) may be regarded as one or more processors. The video encoder 20 and the video decoder 30 can each be included in one or more encoders or decoders, and also, any one of the encoders or decoders may be integrated as part of a combined encoder / decoder (codec) in the corresponding apparatus.

[0080] The source device 12 may be referred to as a video encoding device or video encoding equipment. The destination device 14 may be referred to as a video decoding device or video decoding equipment. The source device 12 and the destination device 14 can each be an example of a video encoding device or a video decoding device.

[0081] The source device 12 and the destination device 14 can each include any one of various devices, such as, for example, a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a video camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming transmission device (such as a content service server or a content distribution server, etc.), a broadcast receiver device, or a broadcast transmitter device, including any type of handheld device or fixed device, and may or may not use some type of operating system.

[0082] In some cases, the source device 12 and the destination device 14 can be provided for wireless communication. Accordingly, the source device 12 and the destination device 14 can be wireless communication devices.

[0083] In some cases, the video encoding system 10 shown in FIG. 1 is merely an example, and the technology in this application may be applied to a video encoding setting (for example, video encoding or video decoding) that does not require any data communication between the encoding device and the decoding device. In other examples, the data can be retrieved from local memory, streamed over a network, etc. The video encoding device can encode the data and store the data in memory, and / or the video decoding device can retrieve the data from memory and decode the data. In some examples, the encoding and decoding are performed by devices that do not communicate with each other, but simply encode the data in memory and / or retrieve the data from memory and decode the data.

[0084] It should be understood that for each of the above examples described with reference to the video encoder 20, the video decoder 30 can be configured to perform the reverse process. For signaling syntax elements, the video decoder 30 can be configured to receive and parse the syntax elements and decode the associated video data accordingly. In some examples, the video encoder 20 can entropy encode the syntax elements into the encoded video bitstream. In such examples, the video decoder 30 can parse the syntax elements and decode the associated video data accordingly.

[0085] FIG. 2 is a diagram showing an example of a video coding system 40 including the encoder 20 of FIG. 3 and / or the decoder 30 of FIG. 4 according to an exemplary embodiment. The system 40 can implement combinations of various techniques of this application. In the illustrated implementation, the video coding system 40 can include an imaging device 41, a video encoder 20, a video decoder 30 (and / or a video decoder implemented by the logic circuit 47 of the processing unit 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0086] As shown in FIG. 2, the imaging device 41, the antenna 42, the processing unit 46, the logic circuit 47, the video encoder 20, the video decoder 30, the processor 43, the memory 44, and / or the display device 45 can communicate with each other. As will be described, the video coding system 40 is shown having both a video encoder 20 and a video decoder 30, but in different examples, the video coding system 40 can include only the video encoder 20 or only the video decoder 30.

[0087] In some examples, as shown in FIG. 2, the video coding system 40 may include an antenna 42. For example, the antenna 42 may be configured to transmit or receive an encoded bitstream of video data. Further, in some examples, the video coding system 40 may include a display device 45. The display device 45 may be configured to present video data. In some examples, as shown in FIG. 2, the logic circuit 47 may be implemented by a processing unit 46. The processing unit 46 may include application-specific integrated circuit (ASIC) logic, a graphics processing unit, a general-purpose processor, or the like. The video coding system 40 may also include an optional processor 43. The optional processor 43 may similarly include application-specific integrated circuit logic, a graphics processing unit, a general-purpose processor, or the like. In some examples, the logic circuit 47 may be implemented by hardware such as, for example, video coding dedicated hardware, and the processor 43 may be implemented by universal software, an operating system, or the like. Further, the memory 44 may be any type of memory, such as, for example, volatile memory (e.g., Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM)) or non-volatile memory (e.g., flash memory). In one non-limiting example, the memory 44 may be implemented by cache memory. In some examples, the logic circuit 47 may access the memory 44 (e.g., to implement a picture buffer). In other examples, the logic circuit 47 and / or the processing unit 46 may include a memory (e.g., a cache) for implementing a picture buffer or the like.

[0088] In some examples, a video encoder 20 implemented by a logic circuit may include a picture buffer (implemented, for example, by processing unit 46 or memory 44) and a graphics processing unit (implemented, for example, by processing unit 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include a video encoder 20 implemented by logic circuit 47 to implement various modules described with reference to FIG. 2 and / or any other encoder system or subsystem described in this specification. The logic circuit may be configured to perform various operations described in this specification.

[0089] Similarly, a video decoder 30 may be implemented by logic circuit 47 to implement various modules described with reference to decoder 30 of FIG. 4 and / or any other decoder system or subsystem described in this specification. In some examples, a video decoder 30 implemented by a logic circuit may include a picture buffer (implemented by processing unit 46 or memory 44) and a graphics processing unit (implemented, for example, by processing unit 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include a video decoder 30 implemented by logic circuit 47 to implement various modules described with reference to FIG. 4 and / or any other decoder system or subsystem described in this specification.

[0090] In some examples, the antenna 42 of the video coding system 40 may be configured to receive an encoded bitstream of video data. As will be described, the encoded bitstream may include, for example, data related to an encoded partition (e.g., a transform coefficient or a quantized transform coefficient, an optional indicator (to be described), and / or data defining the encoded partition), data related to video frame encoding as described in this specification, indicators, index values, mode selection data, or the like. The video coding system 40 may further include a video decoder 30 coupled to the antenna 42 and configured to decode the encoded bitstream. The display device 45 is configured to present video frames.

[0091] Encoder & Encoding Method FIG. 3 is a schematic / conceptual block diagram of an example of a video encoder 20 configured to implement the technology in this application (disclosure). In the example of FIG. 3, the video encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy encoding unit 270. The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown in the figure). The video encoder 20 shown in FIG. 3 may also be referred to as a hybrid video encoder or a video encoder based on a hybrid video codec.

[0092] For example, the residual calculation unit 204, the conversion processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy encoding unit 270 form the forward signal path of the encoder 20, and the inverse quantization unit 210, the inverse conversion processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer 230, the prediction processing unit 260, etc. form the backward signal path of the encoder. The backward signal path of the encoder corresponds to the signal path of the decoder (see the decoder 30 in FIG. 4).

[0093] The encoder 20 receives, for example, a picture 201, or a block 203 of the picture 201, such as a picture within a series of pictures forming a video or a video sequence, by using the input 202 or the like. The picture block 203 may also be referred to as the current picture block or the picture block to be encoded, and the picture 201 may also be referred to as the current picture or the picture to be encoded (especially when the current picture is distinguished from another picture in video coding, for example, other pictures within the same video sequence, including pictures previously encoded and / or decoded within the video sequence of the current picture).

[0094] Partition An embodiment of the encoder 20 may include a partitioning unit (not shown in FIG. 3) configured to partition the picture 201 into a plurality of non-overlapping blocks such as the block 203. The partitioning unit may be configured to use the same block size and the corresponding raster defining the block size for all pictures within the video sequence, or alternatively, may be configured to vary the block size among pictures, subsets, or groups of pictures to partition each picture into corresponding blocks.

[0095] In one example, the prediction processing unit 260 of the video encoder 20 may be configured to perform any combination of the above-described partitioning techniques.

[0096] For example, within picture 201, although the size of block 203 is smaller than that of picture 201, block 203 is also a two-dimensional array or matrix of samples having luminance values (sample values), or can be regarded as such a two-dimensional array or matrix. In other words, block 203 can include, for example, one sample array (e.g., the luminance array in the case of a monochrome picture 201), three sample arrays (e.g., one luminance array and two chrominance arrays in the case of a color picture), or any other number and / or type of array based on the color format used. The number of samples in the horizontal and vertical directions (or axes) of block 203 determines the size of block 203.

[0097] The encoder 20 shown in FIG. 3 is configured to encode picture 201 block by block, and is configured to perform encoding and prediction for each block 203, for example.

[0098] Residual calculation The residual calculation unit 204 is configured to calculate a residual block 205 based on the picture block 203 and the prediction block 265 (more details about the prediction block 265 will be provided later), and is configured to obtain the residual block 205 in the sample domain, for example, by subtracting the sample values of the prediction block 265 from the sample values of the picture block 203 sample by sample.

[0099] Transformation The transformation processing unit 206 is configured to apply a transformation such as a discrete cosine transform (DCT) or a discrete sine transform (DST) to the sample values of the residual block 205 to obtain transformation coefficients 207 in the transform domain. The transformation coefficients 207 may also be referred to as residual transformation coefficients and represent the residual block 205 in the transform domain.

[0100] The conversion processing unit 206 may be configured to apply an integer approximation of DCT / DST, such as the conversion specified by HEVC / H.265 for example. This integer approximation is usually scaled proportionally by a factor compared to the orthogonal DCT transform. To maintain the norm of the residual block obtained through the forward and inverse conversions, an additional scale factor is applied as part of the conversion process. The scale factor is usually selected based on some constraints, such as being a power of 2 for example, the bit depth of the conversion coefficients, or the trade-off between the accuracy used in the shift operation and the implementation cost. For example, by using the inverse conversion processing unit 212, a specific scale factor is specified for the inverse conversion on the decoder 30 side (and correspondingly, for the inverse conversion on the encoder 20 side by using the inverse conversion processing unit 212 etc.), and correspondingly, by using the conversion processing unit 206, a corresponding scale factor may be specified for the forward conversion on the encoder 20 side.

[0101] Quantization The quantization unit 208 is configured to quantize the transform coefficients 207 by applying scalar quantization, vector quantization, or the like, to obtain quantized transform coefficients 209. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth for some or all of the transform coefficients 207. For example, assuming n is greater than m, an n-bit transform coefficient can be truncated to an m-bit transform coefficient during quantization. The degree of quantization can be changed by adjusting a quantization parameter (QP). For example, in scalar quantization, multiple different scales can be applied to achieve finer or coarser quantization. A smaller quantization step corresponds to finer quantization, and a larger quantization step corresponds to coarser quantization. The quantization parameter may be used to indicate an appropriate quantization step. For example, the quantization parameter may be an index of a predetermined set of appropriate quantization steps. For example, a smaller quantization parameter can correspond to finer quantization (a smaller quantization step), and a larger quantization parameter can correspond to coarser quantization (a larger quantization step), and vice versa. Quantization can include division by the quantization step and inverse quantization performed by a corresponding quantization or inverse quantization unit 210, etc., or may include multiplication by the quantization step. For example, in embodiments according to some standards such as HEVC, the quantization step can be determined using the quantization parameter. Generally, the quantization step can be calculated based on the quantization parameter through a fixed-point approximation of an expression including division. A further scale factor may be introduced into quantization and inverse quantization to restore a norm that is that of the residual block and that may be modified for the scale used in the fixed-point approximation of the expression used for the quantization step and the quantization parameter. In one implementation example, the scale of the inverse transform may be combined with the scale of the inverse quantization. Alternatively, a customized quantization table may be used and signaled from the encoder to the decoder, for example, within a bitstream.Quantization is an operation with loss, and a larger quantization step shows a larger loss.

[0102] The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients to obtain the inverse quantized coefficients 211. For example, based on or using the same quantization step as the quantization unit 208, it is configured to apply the inverse quantization scheme of the quantization scheme applied by the quantization unit 208. The inverse quantized coefficients 211 may also be referred to as inverse quantized residual coefficients 211 and may correspond to the transform coefficients 207, but usually the loss caused by quantization is different from the transform coefficients.

[0103] The inverse transform processing unit 212 is configured to apply the inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform or an inverse discrete sine transform, to obtain the inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transform inverse quantization block 213 or an inverse transform residual block 213.

[0104] The reconstruction unit 214 (for example, the adder 214) is configured to add the inverse transform block 213 (that is, the reconstruction residual block 213) to the prediction block 265, for example, by adding the sample values of the reconstruction residual block 213 and the sample values of the prediction block 265, to obtain the reconstruction block 215 in the sample domain.

[0105] Optionally, a buffer unit 216 (or simply "buffer" 216), such as a line buffer 216, is configured to buffer or store the sample values of the reconstruction block 215 and corresponding intra prediction, etc. In other embodiments, the encoder may be configured to use the unfiltered reconstruction block and / or corresponding sample values stored in the buffer unit 216 for any type of estimation and / or prediction, such as intra prediction.

[0106] For example, an embodiment of the encoder 20 is configured such that the buffer unit 216 stores the reconstructed block 215 for intra prediction and also stores the filtered block 221 of the loop filter unit 220 (not shown in FIG. 3), and / or the buffer unit 216 and the decoded picture buffer unit 230 may be configured to form one buffer. Other embodiments may be used to input or use as a basis for intra prediction blocks or samples (not shown in FIG. 3) from the filtered block 221 and / or the decoded picture buffer 230.

[0107] The loop filter unit 220 (or simply "loop filter" 220) is configured to perform filtering on the reconstructed block 215 so as to smoothly perform sample conversion or improve video quality, thereby obtaining the filtered block 221. The loop filter unit 220 is intended to represent one or more loop filters, such as, for example, a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a cooperative filter. The loop filter unit 220 is shown as an in-loop filter in FIG. 3, but the loop filter unit 220 may be implemented as a post-loop filter in other configurations. The filtered block 221 may also be referred to as the filtered reconstructed block 221. After the loop filter unit 220 performs a filtering process on the reconstructed coded block, the decoded picture buffer 230 may store the reconstructed coded block.

[0108] Embodiments of the encoder 20 (correspondingly, the loop filter unit 220) can be used to output loop filter parameters (e.g., sample adaptive offset information), for example, directly output the loop filter parameters, or output the loop filter parameters after the entropy encoding unit 270 or some other entropy encoding unit performs entropy encoding, thereby enabling the decoder 30 to receive the same loop filter parameters and apply them to decoding and the like.

[0109] The decoded picture buffer 230 can be a reference picture memory for storing reference picture data for the video encoder 20 to encode video data. The DPB 230 can be any one of a plurality of memories, such as, for example, a dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), or resistive RAM (RRAM)), or other types of memory. The DPB 230 and the buffer 216 may be provided by the same memory or by separate memories. In one example, the decoded picture buffer (DPB) 230 is configured to store the filtered block 221. The decoded picture buffer 230 may further be configured to store other preceding filtered blocks, such as preceding reconstructed and filtered blocks 221 of the same current picture or different pictures such as, for example, a preceding reconstructed picture, and may also provide preceding complete reconstructions, i.e., decoded pictures (along with corresponding reference blocks and corresponding samples), and / or partially reconstructed current pictures (along with corresponding reference blocks and corresponding samples) for inter prediction, etc. In one example, when the reconstruction block 215 is reconstructed without in-loop filtering, the decoded picture buffer 230 is configured to store the reconstruction block 215.

[0110] The prediction processing unit 260, also referred to as the block prediction processing unit 260, receives or acquires the block 203 (the current block 203 of the current picture 201) and reconstruction picture data such as reference samples from the same (current) picture in the buffer 216 and / or reference picture data 231 from one or more previously decoded pictures in the decoded picture buffer 230, and processes such data for prediction, that is, is configured to provide a prediction block 265 that can be an inter-prediction block 245 or an intra-prediction block 255.

[0111] The mode selection unit 262 may be configured to select a prediction mode (e.g., an intra-prediction mode or an inter-prediction mode) and / or a corresponding prediction block 245 or 255 as the prediction block 265 in order to calculate the residual block 205 and reconstruct the reconstruction block 215.

[0112] Using an embodiment of the mode selection unit 262, a prediction mode may be selected (e.g., from the prediction modes supported by the prediction processing unit 260). The prediction mode provides the best match or the smallest residual (the smallest residual means better compression in transmission or storage), or provides the smallest signaling overhead (the smallest signaling overhead means better compression in transmission or storage), or considers or balances these two. The mode selection unit 262 may be configured to determine the prediction mode based on rate distortion optimization (RDO), that is, select a prediction mode that provides the smallest rate distortion optimization, or select a prediction mode whose associated rate distortion satisfies at least the prediction mode selection criterion.

[0113] The prediction processing (e.g., by using the prediction processing unit 260) and mode selection (e.g., by using the mode selection unit 262) performed by an example of the encoder 20 will be described in detail later.

[0114] As described above, the encoder 20 is configured to determine or select the best or optimal prediction mode from a (predetermined) set of prediction modes. The set of prediction modes may include, for example, an intra prediction mode and / or an inter prediction mode.

[0115] The intra prediction mode set can include 35 different intra prediction modes, or can include 67 different intra prediction modes, or can include the intra prediction modes under development defined in H.266.

[0116] The inter prediction mode set depends on the available reference pictures (e.g., at least a part of the decoded pictures stored in the DBP 230) and other inter prediction parameters. For example, it depends on whether the entire reference picture is used, or only a part of the reference picture is used, such as when the search window area surrounding the area of the current block for the best matching reference block is searched, and / or depends on whether sample interpolation such as 1 / 2 sample and / or 1 / 4 sample interpolation is applied.

[0117] In addition to the above prediction modes, a skip mode and / or a direct mode may also be applied.

[0118] The prediction processing unit 260 may further be configured to divide the block 203 into smaller block partitions or sub-blocks by repeatedly using, for example, a quad-tree (QT) partition, a binary-tree (BT) partition, a triple-tree (TT) partition, or some combination thereof, and perform prediction and the like for each of the block partitions or sub-blocks. The mode selection includes selecting the tree structure of the block 203 to be divided and selecting the prediction mode applied to each of the block partitions or sub-blocks.

[0119] The inter-prediction unit 244 may include a motion estimation (ME) unit (not shown in FIG. 3) and a motion compensation (MC) unit (not shown in FIG. 3). The motion estimation unit is configured to receive or obtain, for performing motion estimation, the picture block 203 (the current picture block 203 of the current picture 201) and the decoded picture 231, or at least one or more previous reconstruction blocks such as, for example, one or more other reconstruction blocks different from the previous decoded picture 231. For example, the video sequence may include the current picture and the previous decoded picture 31. In other words, the current picture and the previous decoded picture 31 may be part of a series of pictures that form the video sequence, i.e., form the picture sequence.

[0120] For example, the encoder 20 may be configured to select a reference block from the reference blocks of the same picture or a plurality of different pictures among a plurality of other pictures, and provide an offset (spatial offset) between the position (X-Y coordinates) of the reference picture and / or the reference block and the position of the current block as an inter-prediction parameter to the motion estimation unit (not shown in FIG. 3). This offset is also referred to as a motion vector (MV).

[0121] The motion compensation unit is configured to obtain an inter prediction parameter, for example, by receiving it, and perform inter prediction based on or by using the inter prediction parameter to obtain an inter prediction block 245. The motion compensation performed by the motion compensation unit (not shown in FIG. 3) may include fetching or generating a prediction block based on the motion / block vector determined through motion estimation (optionally performing interpolation with sub-sample accuracy). In interpolation filtering, additional samples can be generated from known samples, which may increase the number of candidate prediction blocks that can be used to encode a picture block. When receiving the motion vector used for the PU of the current picture block, the motion compensation unit 246 may locate the prediction block pointed to by the motion vector within the reference picture list. The motion compensation unit 246 can further generate syntax elements related to the block and video slice, and may be configured to use these syntax elements when the video decoder 30 decodes the picture block of the video slice.

[0122] The intra prediction unit 254 is configured to obtain, for example, by receiving, the picture block 203 (current picture block) of the same picture and one or more previous reconstructed blocks such as, for example, the reconstructed adjacent blocks, in order to perform intra estimation. For example, the encoder 20 may be configured to select an intra prediction mode from a plurality of intra prediction modes.

[0123] Embodiments of the encoder 20 may be configured to select an intra prediction mode based on an optimization criterion, such as, for example, based on minimum residual (e.g., the intra prediction mode that provides the prediction block 255 most similar to the current picture block 203) or minimum rate distortion.

[0124] The intra prediction unit 254 is further configured to determine an intra prediction block 255 based on the intra prediction parameters of the selected intra prediction mode. In any case, after selecting the intra prediction mode to be used for the block, the intra prediction unit 254 is further configured to provide the intra prediction parameters to the entropy coding unit 270, that is, to provide information indicating the selected intra prediction mode to be used for the block. In one example, the intra prediction unit 254 may be configured to perform any combination of the following intra prediction techniques.

[0125] The entropy coding unit 270 is configured to apply an entropy coding algorithm or scheme (e.g., variable length coding (VLC) scheme, context adaptive VLC (CAVLC) scheme, arithmetic coding scheme, context adaptive binary arithmetic coding (CABAC) scheme, syntax-based context-adaptive binary arithmetic coding (SBAC) scheme, probability interval partitioning entropy (PIPE) coding scheme, or other entropy coding method or technique) to one or more (or none) of the quantized residual coefficients 209, inter prediction parameters, intra prediction parameters, and / or loop filter parameters, so as to obtain encoded picture data 21 output using output 272, for example, in the form of an encoded bitstream. The encoded bitstream may be transmitted to the video decoder 30, or may be archived for later transmission or retrieval by the video decoder 30. The entropy coding unit 270 may further be configured to perform entropy coding on other syntax elements of the current video slice being encoded.

[0126] To encode the video stream, another video encoder 20 that is structurally somewhat different may be configured. For example, the non-conversion-based encoder 20 may directly quantize the residual signal for some blocks or frames without using the conversion processing unit 206. In another implementation, the encoder 20 may have a quantization unit 208 and an inverse quantization unit 210 coupled to one unit.

[0127] FIG. 4 shows an example of a video decoder 30 configured to implement the technology in this application. The video decoder 30 is configured to receive encoded picture data (e.g., an encoded bitstream) 21 encoded by the encoder 20 or the like and obtain a decoded picture 31. In the decoding process, the video decoder 30 receives video data from the video encoder 20, such as an encoded video bitstream indicating picture blocks of an encoded video slice and related syntax elements.

[0128] In the example of FIG. 4, the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse conversion processing unit 312, a reconstruction unit 314 (e.g., an adder 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. The prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. In some examples, the video decoder 30 may perform decoding calculations generally inverse to the encoding calculations described with reference to the video encoder 20 of FIG. 3.

[0129] Entropy decoding unit 304 is configured to perform entropy decoding on the encoded picture data 21 to obtain the quantized coefficients 309, the decoded encoding parameters (not shown in FIG. 4), and / or the like, for example, any one or all of the inter prediction parameters, intra prediction parameters, loop filter parameters, and / or other (decoded) syntax elements. The entropy decoding unit 304 is further configured to transfer the inter prediction parameters, intra prediction parameters, and / or other syntax elements to the prediction processing unit 360. The video decoder 30 may receive syntax elements at the video slice level and / or syntax elements at the video block level.

[0130] The inverse quantization unit 310 can have the same function as the inverse quantization unit 110, the inverse transform processing unit 312 can have the same function as the inverse transform processing unit 212, the reconstruction unit 314 can have the same function as the reconstruction unit 214, the buffer 316 can have the same function as the buffer 216, the loop filter 320 can have the same function as the loop filter 220, and the decoded picture buffer 330 can have the same function as the decoded picture buffer 230.

[0131] The prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354. The inter prediction unit 344 can have a function similar to that of the inter prediction unit 244, and the intra prediction unit 354 can have a function similar to that of the intra prediction unit 254. The prediction processing unit 360 is generally configured to perform block prediction and / or obtain a prediction block 365 from the encoded data 21, and to receive or obtain information about prediction-related parameters and / or selected prediction modes (explicitly or implicitly) from, for example, the entropy decoding unit 304.

[0132] Also, when a video slice is encoded as an intra-coded (I) slice, the intra prediction unit 354 of the prediction processing unit 360 is configured to generate a prediction block 365 to be used for a picture block of the current video slice based on the signaled intra prediction mode and data from a previously decoded block of the current frame or picture. When a video frame is encoded as an inter-coded (i.e., B or P) slice, the inter prediction unit 344 (e.g., motion compensation unit) of the prediction processing unit 360 is configured to generate a prediction block 365 to be used for a video block of the current video slice based on the motion vector and other syntax elements received from the entropy decoding unit 304. In inter prediction, a prediction block may be generated from one of the reference pictures within one reference picture list. The video decoder 30 may construct reference frame lists 0 and 1 by using a default construction technique based on the reference pictures stored in the DPB 330.

[0133] The prediction processing unit 360 is configured to determine prediction information to be used for a video block of the current video slice by analyzing the motion vector and other syntax elements, and to generate a prediction block to be used for the current video block being decoded by using the prediction information. For example, the prediction processing unit 360 determines, by using some of the received syntax elements, a prediction mode (e.g., intra prediction or inter prediction) used to encode a video block of the video slice, an inter prediction slice type (e.g., B slice, P slice, or GPB slice), configuration information of one or more of the pictures within the reference picture list used for the slice, a motion vector of each inter-coded video block used for the slice, an inter prediction state of each inter-coded video block used for the slice, and other information, and decodes the video blocks of the current video slice.

[0134] The inverse quantization unit 310 may be configured to perform inverse quantization (i.e., dequantization) on the quantized transform coefficients provided in the bitstream and decoded by the entropy decoding unit 304. The inverse quantization process may include determining the degree of quantization to be applied and the degree of inverse quantization to be applied, using the quantization parameters calculated by the video encoder 20 for each video block in the video slice.

[0135] The inverse transform processing unit 312 is configured to apply an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to generate a residual block in the sample domain.

[0136] The reconstruction unit 314 (e.g., adder 314) is configured to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365, for example, by adding the sample values of the reconstructed residual block 313 to the sample values of the prediction block 365, to obtain a reconstructed block 315 in the sample domain.

[0137] The loop filter unit 320 (either in or after the encoding loop) is configured to filter the reconstruction block 315 to obtain a filtered block 321 so as to perform sample conversion smoothly or improve video quality. In one example, the loop filter unit 320 may be configured to perform some combination of the following filtering techniques. The loop filter unit 320 is intended to represent one or more loop filters, such as, for example, a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters such as a bilateral filter, an adaptive loop filter (ALF), an edge enhancement or smoothing filter, or a cooperative filter. Although the loop filter unit 320 is shown as an in-loop filter in FIG. 4, the loop filter unit 320 may be implemented as a post-loop filter in other configurations.

[0138] The filtered block 321 within a given frame or picture is then stored in a decoded picture buffer 330 that stores reference pictures used for subsequent motion compensation.

[0139] The decoder 30 is configured to output the decoded picture 31 using the output 332 and the like to present the decoded picture 31 to the user or provide the decoded picture 31 for the user to view.

[0140] Another variation of the video decoder 30 may be configured to decode the compressed bitstream. For example, the decoder 30 may generate an output video stream without using the loop filter unit 320. For example, a non-transform-based decoder 30 may directly dequantize the residual signal for some blocks or frames without using the inverse transform processing unit 312. In another implementation, the video decoder 30 may have an inverse quantization unit 310 and an inverse transform processing unit 312 coupled in one unit.

[0141] FIG. 5 is a schematic configuration diagram of a video coding apparatus 400 (e.g., a video encoding apparatus 400 or a video decoding apparatus 400) according to an embodiment of this application. The video coding apparatus 400 is suitable for implementing an embodiment described in this specification. In one embodiment, the video coding apparatus 400 can be a video decoder (e.g., the video decoder 30 in FIG. 1) or a video encoder (e.g., the video encoder 20 in FIG. 1). In another embodiment, the video coding apparatus 400 may be one or more components within the video decoder 30 in FIG. 1 or the video encoder 20 in FIG. 1.

[0142] The video coding apparatus 400 includes an input port 410 and a receiver unit (Rx) 420 configured to receive data, a processor, logic unit, or central processing unit (CPU) 430 configured to process data, a transmitter unit (Tx) 440 and an output port 450 configured to transmit data, and a memory 460 configured to store data. The video coding apparatus 400 may further include optical - electrical conversion components and electro - optical (EO) conversion components coupled to the input port 410, the receiver unit 420, the transmitter unit 440, and the output port 450 to function as an outlet or inlet for optical or electrical signals.

[0143] Processor 430 is implemented by using hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, or DSPs. Processor 430 communicates with an input port 410, a receiver unit 420, a transmitter unit 440, an output port 450, and a memory 460. Processor 430 includes a coding module 470 (e.g., an encoding module 470 or a decoding module 470). The encoding / decoding module 470 implements the embodiments disclosed above. For example, the encoding / decoding module 470 performs, processes, or provides various coding operations. Accordingly, the encoding / decoding module 470 substantially expands the functions of the video coding device 400 and affects the conversion of the video coding device 400 into a plurality of different states. Alternatively, the encoding / decoding module 470 is implemented by using instructions stored in the memory 460 and executed by the processor 430.

[0144] Memory 460 includes one or more disks, tape drives, and solid state drives and can be used as an overflow data storage device and is configured to store those programs when the programs are selectively executed and to store the instructions and data read during program execution. Memory 460 can be volatile and / or non-volatile and can be a read only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random access memory (SRAM).

[0145] FIG. 6 is a simplified block diagram of an apparatus 500 that can be used as either or both of the source apparatus 12 and the destination apparatus 14 of FIG. 1 according to an exemplary embodiment. The apparatus 500 can implement the technology in this application. The apparatus 500 configured to implement chroma block prediction may use the form of a computing system including a plurality of computing devices, or alternatively, may use the form of a single computing device such as, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, or a desktop computer.

[0146] The processor 502 within the apparatus 500 can be a central processing unit. Alternatively, the processor 502 can be any other type of existing or future one or more devices capable of controlling or processing information. As shown in FIG. 6, although the disclosed implementation can be carried out by using a single processor such as, for example, the processor 502, advantages in speed and efficiency may be achieved by using two or more processors.

[0147] In one implementation, the memory 504 within the apparatus 500 may be a read-only memory device or a random access memory device. Some other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 that are accessed by the processor 502 using the bus 512. The memory 504 may further include an operating system 508 and an application program 510. The application program 510 includes at least one program that enables the processor 502 to execute the methods described in this specification. For example, the application program 510 may include applications 1 through N, and further, applications 1 through N include a video encoding application for executing the methods described in this specification. The apparatus 500 may further include additional memory in the form of secondary memory 514. The secondary memory 514 may be, for example, a memory card used with a mobile computing device. Since a video communication session may include a large amount of information, the information may be stored completely or partially in the secondary memory 514 and loaded into the memory 504 for processing as needed.

[0148] Device 500 may further include one or more output devices, such as, for example, display 518. In one example, display 518 may be a touch-sensitive display that combines a touch-sensing element that can be operated to sense touch input and a display. Display 518 may be coupled to processor 502 by using bus 512. In addition to display 518, another output device may be further provided to enable a user to program device 500 or use device 500 in another way, or another output device may be provided in place of display 518. If the output device is or includes a display, the display can be implemented in various ways, for example, a liquid crystal display, a cathode-ray tube (CRT) display, a plasma display, or a light emitting diode (LED) display such as an organic LED (OLED) display.

[0149] Device 500 can further include a picture sensing device 520 or can be connected to a picture sensing device 520. Picture sensing device 520 is, for example, a camera or any other existing or future picture sensing device 520 that can sense a picture. The picture is, for example, a picture of the user operating device 500. Picture sensing device 520 can be arranged directly facing the user operating device 500. In one example, the position and optical axis of picture sensing device 520 can be set such that the field of view of picture sensing device 520 includes an area adjacent to display 518 and display 518 can be seen from that area.

[0150] Device 500 can further include, or be connectable to, a sound sensing device 522. The sound sensing device 522 can be, for example, a microphone or any other existing or future sound sensing device capable of sensing sound near the device 500. The sound sensing device 522 may be disposed directly facing the user operating the device 500, or may be configured to receive sounds such as, for example, voice or another sound made by the user when the user operates the device 500.

[0151] The processor 502 and the memory 504 of the device 500 are integrated into one unit as shown in FIG. 6, but other configurations may be used. The execution of the processor 502 may be distributed among a plurality of machines (each machine having one or more processors) that may be directly coupled, or may be distributed within a local area or within another network. The memory 504 may be distributed among a plurality of machines such as, for example, network-based memory and memory within a plurality of machines operating the device 500. Although a single bus is depicted here, there may be a plurality of buses 512 in the device 500. Further, the secondary memory 514 may be directly coupled to other components of the device 500 or may be accessed via a network, and may include a single integrated unit such as, for example, a memory card, or a plurality of units such as, for example, a plurality of memory cards. Accordingly, the device 500 may be implemented in a plurality of configurations.

[0152] As described above in this application, a color video further includes chroma components (U, V) in addition to the luma (Y) component. Accordingly, in addition to the luma component, the chroma components also need to be encoded. YUV4:4:4, YUV4:2:2, and YUV4:2:0 generally exist according to different methods for sampling the luma and chroma components of a color video. As shown in FIG. 7, the x marks represent luma component samples and the o marks represent chroma component samples.

[0153] 4:4:4 format: The chroma component is not downsampled.

[0154] 4:2:2 format: For the luminance component, 2:1 horizontal downsampling is performed on the chroma component, and no vertical downsampling is performed. Each row contains four Y samples for every two U or V samples.

[0155] 4:2:0 format: For the luminance component, 2:1 horizontal downsampling is performed on the chroma component, and 2:1 vertical downsampling is performed.

[0156] YUV4:2:0 is the most common format. When the video picture is in the YUV4:2:0 sampling format, if the luminance component of the picture block is a 2M×2N picture block, the chroma component of the picture block is an M×N picture block. Therefore, the chroma component of the picture block is also referred to as a chroma block or a chroma component block in this application. This application is described using YUV4:2:0 as an example, but it is also applicable to other methods for sampling the luminance and chroma components.

[0157] In this application, the samples in the chroma picture are abbreviated as chroma samples or chroma points, and the samples in the luma picture are abbreviated as luma samples or luma points.

[0158] Similar to the luma component, in chroma intra prediction, the boundary samples of the adjacent reconstructed blocks around the current chroma block are also used as the reference samples of the current block, and the reference samples are mapped to the samples in the current chroma block according to a specific prediction mode to serve as predictors for the samples in the current chroma block. Since the texture of the chroma component is usually relatively simple, there is a difference in that the number of intra prediction modes for the chroma component is usually less than the number of intra prediction modes for the luma component.

[0159] The cross-component prediction mode is also referred to as the cross-component intra prediction mode or the CCLM prediction mode. The CCLM prediction mode may sometimes be abbreviated as the linear model mode. The LM mode (abbreviated as the linear model or the linear mode) is a chroma intra prediction method in which the texture correlation between luminance and chrominance is used. In LM, the predictor of the current chroma block is derived according to a linear model using the reconstructed luma component, which is given by the following formula:

Equation

[0160] Here, α and β are linear model coefficients, α is a scaling coefficient, β is an offset factor, and pred C (i,j) is the predictor of the chroma sample at position (i,j), and rec i L(i, j) is the luma reconstruction sample value at position (i, j) after the luma reconstruction block corresponding to the current chroma block (hereinafter abbreviated as the corresponding luma block) is downsampled to the chroma component resolution. In a video in YUV4:2:0 format, the resolution of the luma component is 4 times that of the chroma component (the width and height of the luma component are each 2 times those of the chroma component). To obtain a luma block with the same resolution as the chroma block, it is necessary to downsample the luma component to the chroma resolution according to the same downsampling method as that for the chroma component.

[0161] The linear model coefficients α and β do not need to be encoded for transmission and are derived by using the edge samples of the adjacent reconstruction blocks of the current chroma block and the luma samples corresponding to those edge samples. FIG. 8 shows an embodiment of the cross-component prediction mode. In FIG. 8, recL is the reconstructed luma block (the current chroma block corresponds to the reconstructed block and adjacent reference samples of the luma block), recL' is the downsampled luma block, and recC' is the adjacent reconstruction reference sample of the current chroma block. The size of the current chroma block is W×H. By using the upper adjacent reconstruction sample and the left adjacent reconstruction sample of the current chroma block as reference samples, the size of the corresponding luma block is 2W×2H. By downsampling the luma block and the reference samples of the luma block to the chroma resolution, the sample block shown in FIG. 8(b) is obtained. The adjacent reference samples in FIGS. 8(b) and 8(c) have a one-to-one correspondence.

[0162] For ease of explanation, in this application, the upper adjacent sample and the left adjacent sample used to calculate the linear model coefficients are referred to as adjacent samples, the sample above is the upper adjacent sample, and the sample on the left is the left adjacent sample. Samples adjacent to a chroma block are referred to as adjacent samples (which include the upper adjacent sample and the left adjacent sample), and samples adjacent to a luma block are referred to as adjacent samples (which include the upper adjacent sample and the left adjacent sample). Chroma samples have a one-to-one correspondence with luma samples, and the values of the samples form value pairs.

[0163] In an embodiment of this application, luma samples usually need to be obtained through downsampling (because the resolution of the chroma component is different from that of the luma component), and are denoted as Luma’ samples. Chroma samples are usually the upper adjacent samples of one or two rows of the current chroma block and the left adjacent samples of one or two columns of the current chroma block. FIG. 9 is a schematic diagram of using one row and one column, and FIG. 10 is a schematic diagram of using two rows and two columns.

[0164] In the LM mode, the correlation between the luma component and the chroma component can be effectively utilized. Compared with the directional prediction mode, the LM mode is more flexible, and thus provides a more accurate prediction signal for the chroma component.

[0165] Furthermore, there is also a multiple model linear model (MMLM) mode, and there are multiple α and β. In an example of two linear models, there are two groups of linear model coefficients, α1 and β1, and α2 and β2.

[0166] This application provides a chroma block prediction method. Hereinafter, with reference to a specific implementation, the processing procedure shown in FIG. 11 will be described in detail. This process exists in both the encoding process and the decoding process. In this embodiment of this application, the decoder side is used as an example of the solution explanation, and the content may be as follows.

[0167] Step 1101: Obtain the chroma value of the chroma sample at the preset position from the adjacent samples of the chroma block.

[0168] The chroma block is the chroma block for which intra prediction should be performed currently. The number of chroma samples at the preset position (which may be abbreviated as the preset number) can be preset. For example, the preset number is 2, 4, 6, or 8. The adjacent samples of the chroma block are the reconstructed samples adjacent to the chroma block. Please refer to FIGS. 9 and 10.

[0169] In a specific implementation, when it is necessary to perform intra prediction on the current chroma block (hereinafter, this may be abbreviated as the chroma block), the chroma value of the chroma sample at the preset position is obtained from the adjacent samples of the chroma block and can be represented as set φ, where φ = {C0, C1, …, C M-1}, and M represents the preset number, that is, the number of chroma values to be obtained.

[0170] Optionally, when there is only one linear mode (which may also be called the intra linear prediction mode), in the coordinate system shown in FIG. 12, the preset positions are usually (0, -1), (X - 1, -1), (-1, 0), and (-1, Y - 1), and the chroma values of the chroma samples at the preset positions can be obtained from the adjacent samples of the chroma block.

[0171] Optionally, the decoder side can determine the intra prediction mode corresponding to the chroma block based on the indication information in the bitstream and determine the preset position based on the intra prediction mode. The corresponding processing is Obtaining indication information, determining an intra prediction mode corresponding to a chroma block based on the indication information, and determining a preset position based on the intra prediction mode corresponding to the chroma block may be included.

[0172] The intra prediction mode corresponding to the chroma block may include at least one of LM, LMA, and LML. The indication information is used to indicate the intra prediction mode corresponding to the chroma block.

[0173] In a specific implementation, when performing encoding, the encoder side encodes the intra prediction mode corresponding to the chroma block and transmits the intra prediction mode to the decoder side (this process will be described later).

[0174] The decoder side may obtain the indication information from the received bitstream and determine the intra prediction mode corresponding to the chroma block based on the indication information. Then, the decoder side determines the preset position based on the intra prediction mode corresponding to the chroma block.

[0175] Optionally, the indication information may be set in a syntax table as shown in Table 1.

Table 1

[0176] In Table 1, the syntax element CuPredMode[x0][y0] in the syntax table is used to indicate whether the prediction mode of the current chroma block is the intra prediction mode or the inter prediction mode. For example, if CuPredMode[x0][y0] is MODE_INTRA, it indicates that the intra prediction mode is used for the current chroma block; if CuPredMode[x0][y0] is MODE_INTER, it indicates that the inter prediction mode is used for the current chroma block. Here, x0 and y0 represent the coordinates of the current chroma block in the video picture.

[0177] The syntax element intra_croma_pred_mode[x0][y0] is used to indicate the intra prediction mode information of the current chroma block (i.e., the above-mentioned indication information). For example, when intra_croma_pred_mode[x0][y0]=0, it indicates that the LM0 mode is used for the current block; when intra_croma_pred_mode[x0][y0]=1, it indicates that the LM1 mode is used for the current chroma block; when intra_croma_pred_mode[x0][y0]=2, it indicates that the LM2 mode is used for the current chroma block; and when intra_croma_pred_mode[x0][y0]=N - 1, it indicates that the LM(N - 1) mode is used for the current chroma block. Here, N represents N different LMs that can be selected on the encoder side for the current chroma block, and LM0, LM1, …, and LM(N - 1) represent different intra prediction modes.

[0178] Optionally, when there are multiple linear modes, there are different preset positions for different linear modes, and the corresponding processing in step 1101 is Based on a predetermined correspondence between an intra prediction mode and a preset position, and an intra prediction mode corresponding to a chroma block, it may include obtaining a chroma value of a chroma sample at the preset position from adjacent samples of the chroma block.

[0179] In a specific implementation, when performing decoding, the decoder side determines an intra prediction mode corresponding to a chroma block, then obtains a predetermined correspondence between the intra prediction mode and the preset position, and may determine a preset position corresponding to the intra prediction mode of the chroma block from the correspondence. Then, the decoder side obtains a chroma value of a chroma sample at the preset position from adjacent samples of the chroma block.

[0180] In the above correspondence, the recorded preset position may be the position coordinates of the preset position.

[0181] Optionally, to more fully describe the preset position, as shown in FIG. 12, in this embodiment of this application, a rectangular plane coordinate system is correspondingly established for the chroma block. The first chroma sample at the upper left corner of the chroma block is used as the coordinate origin, the horizontal direction to the right is the positive direction of the x-axis, and the vertical direction downward is the positive direction of the y-axis. Thus, the position coordinates of the upper adjacent samples of the chroma block are (0, -1), (1, -1),..., and (X - 1, -1), and the position coordinates of the left adjacent samples of the chroma block are (-1, 0), (-1, 1),..., and (-1, Y - 1), where X represents the width of the chroma block (i.e., the number of samples in the width direction of the chroma block), and Y represents the height of the chroma block (i.e., the number of samples in the height direction of the chroma block).

[0182] When the intra prediction mode is LM0, in the correspondence, the position coordinates of the preset position may be (0, -1), (X - 1, -1), (-1, 0), and (-1, Y - 1).

[0183] When the intra prediction mode is LM1, in the correspondence relationship, the position coordinates of the preset positions can be (0, -1), (X / 4, -1), (X - 1 - X / 4, -1), and (X - 1, -1).

[0184] When the intra prediction mode is LM2, in the correspondence relationship, the position coordinates of the preset positions can be (-1, 0), (-1, Y / 4), (-1, Y - 1 - Y / 4), and (-1, Y - 1).

[0185] When the intra prediction mode is LM3, in the correspondence relationship, the position coordinates of the preset positions can be (X - 2, -1), (X - 1, -1), (-1, Y - 2), and (-1, Y - 1).

[0186] When the intra prediction mode is LM(N - 1), in the correspondence relationship, the position coordinates of the preset positions can be (0, -1), (1, -1), (X - 2, -1), and (X - 1, -1).

[0187] It can be understood that when there are multiple linear modes, different linear modes can correspond to different preset positions.

[0188] Also, when the linear modes are LM, LMA, and LML, the position coordinates of the preset positions can be listed separately as follows.

[0189] When the intra prediction mode is LM, in the correspondence relationship, the position coordinates of the preset positions can be (0, -1), (X - 1, -1), (-1, 0), and (-1, Y - 1), as shown by the circled marks with shaded diagonal lines in Figure 13.

[0190] When the intra prediction mode is LMA, in the correspondence relationship, the position coordinates of the preset positions can be (0, -1), (X / 4, -1), (X - 1 - X / 4, -1), and (X - 1, -1), as shown by the circled marks filled with diagonal lines in Figure 14(a), or (0, -1), (X / 4, -1), (3×X / 4, -1), and (X - 1, -1), as shown by the circled marks filled with diagonal lines in Figure 14(b).

[0191] When the intra prediction mode is LML, in the correspondence relationship, the position coordinates of the preset positions can be (-1, 0), (-1, Y / 4), (-1, Y - 1 - Y / 4), and (-1, Y - 1) as indicated by the circled marks filled with diagonal lines in FIG. 15(a), or (-1, 0), (-1, Y / 4), (-1, 3×Y / 4), and (-1, Y - 1) as indicated by the circled marks filled with diagonal lines in FIG. 15(b).

[0192] Note that the position coordinates of these preset positions are merely examples. This is not limited to this embodiment of this application.

[0193] Furthermore, when the intra prediction mode is LM, the number of presets may instead be 2, and the position coordinates of the preset positions may be (X - 1, -1) and (-1, Y - 1) as indicated by the circled marks filled with diagonal lines in FIG. 16(a), or (X / 2, -1) and (-1, Y / 2) as indicated by the circled marks filled with diagonal lines in FIG. 16(b), or (X / 2, -1) and (-1, Y - 1) as indicated by the circled marks filled with diagonal lines in FIG. 16(c), or (X - 1, -1) and (-1, Y / 2) as indicated by the circled marks filled with diagonal lines in FIG. 16(d). This is merely an example and is not limited to this embodiment of this application.

[0194] Note that X and Y are usually 4 or more. X and Y in FIGS. 13 and 16(a) to 16(d) are respectively 8 and 8, X in FIG. 14(a) is 8, X in FIG. 14(b) is 16, Y in FIG. 15(a) is 8, and Y in FIG. 15(b) is 16. The above values are merely one possible implementation. This is not limited to this embodiment of this application. Also, in FIGS. 12 to 16(d), the box represents a chroma block.

[0195] Step 1102: Based on the adjacent samples of the luma block corresponding to the chroma block, obtain the luma value of the luma sample corresponding to the chroma sample at the preset position.

[0196] In a specific implementation, after the chroma value is obtained, the luma block corresponding to the chroma block can be determined, and from the adjacent samples of the luma block, the luma value of the luma sample corresponding to the chroma sample at the preset position is obtained and can be represented as set Ψ, where Ψ = {L0, L1, …, L M-1}, and M represents the number of presets, that is, the number of luma values to be obtained.

[0197] Optionally, in Step 1102, the luma value can be selected by multiple methods. Hereinafter, two feasible processing methods are provided.

[0198] Method 1: Determine that the position coordinates of the luma sample corresponding to the i-th preset position within the preset position in the adjacent samples of the luma block are (2×X i , 2×Y i ), the position coordinates of the chroma sample at the i-th preset position are (X i , Y i ), and based on the position coordinates of the luma sample corresponding to the chroma sample at the preset position, obtain the luma value of the luma sample corresponding to the chroma sample at the preset position from the adjacent samples of the luma block corresponding to the chroma block.

[0199] The i-th preset position is any one of the preset positions.

[0200] In a specific implementation, when the chroma block is decoded, the position coordinates of the chroma sample at the i-th preset position within the preset position are (X i , Y i ) (X i is the horizontal coordinate of the i-th preset position in the coordinate system shown in FIG. 12, and Y i(where is the vertical coordinate of the i-th preset position in the coordinate system shown in FIG. 12), from the adjacent samples of the luma block, the position coordinates of the luma sample corresponding to the chroma sample at the i-th preset position are (2×X i , 2×Y i ). Thus, the position coordinates of the luma sample corresponding to the chroma sample at the preset position can be obtained. Then, based on the position coordinates of the luma sample corresponding to the chroma sample at the preset position, the luma value can be obtained from the corresponding luma sample.

[0201] Method 2: Based on the position coordinates of the chroma sample at the j-th preset position within the preset position, determine the position coordinates of a plurality of samples corresponding to the j-th preset position from the adjacent samples of the luma block, and based on the position coordinates of the plurality of samples, determine the position coordinates of the luma sample corresponding to the chroma sample at the j-th preset position. Then, based on the position coordinates of the luma sample corresponding to the chroma sample at the preset position, obtain the luma value of the luma sample corresponding to the chroma sample at the preset position from the adjacent samples of the luma block corresponding to the chroma block.

[0202] The j-th preset position is any one of the preset positions.

[0203] In a specific implementation, when the chroma block is decoded, the position coordinates of the chroma sample at the j-th preset position within the preset position are (X j , Y j ) (where X j is the horizontal coordinate of the j-th preset position in the coordinate system shown in FIG. 12, and Y j is the vertical coordinate of the j-th preset position in the coordinate system shown in FIG. 12). For the chroma sample at the j-th preset position, based on the position coordinates of the chroma sample at the j-th preset position, (X j , Y jThe plurality of sample positions of the adjacent samples of the luma block corresponding to ) can be determined, and then, based on the position coordinates of the plurality of sample positions, the position coordinates of the luma sample corresponding to the chroma sample at the j-th preset position can be obtained.

[0204] For example, a weighting method can be used. The position coordinates of the plurality of sample positions are (2×X j , 2×Y j ), (2×X j , 2×Y j + 1), (2×X j + 1, 2×Y j ), (2×X j + 1, 2×Y j + 1), (2×X j + 2, 2×Y j ), and (2×X j + 2, 2×Y j + 1), and all these sample positions correspond to weight values of 2 / 8, 1 / 8, 1 / 8, 2 / 8, 1 / 8, and 1 / 8 respectively. After weighting, the position coordinates (2×X j , 2×Y j + 0.5) of the luminance sample corresponding to the chroma sample at the j-th preset position can be obtained. Also, the weighting method may not be used. This is not limited in this embodiment of this application. Thus, the position coordinates of the luma sample corresponding to the chroma sample at the preset position can be obtained. And based on the position coordinates of the luma sample corresponding to the chroma sample at the preset position, the luma value can be obtained from the corresponding luma sample.

[0205] Step 1103: Classify the obtained luma values into a first luma set and a second luma set.

[0206] In a specific implementation, after the luma value is obtained, the obtained luma value can be classified into a first luma set and a second luma set.

[0207] Optionally, these luma sets can be obtained by a plurality of methods. This embodiment of this application provides three feasible methods.

[0208] Method 1: Determine the average value of the luma values of the luma samples corresponding to the chroma samples at the preset positions, group the luma values that are among the luma values of the luma samples corresponding to the chroma samples at the preset positions and are less than or equal to the average value of the luma values into the first luma set, and group the luma values that are among the luma values of the luma samples corresponding to the chroma samples at the preset positions and are greater than the average value of the luma values into the second luma set.

[0209] In a specific implementation, the average value of the luma values obtained in step 1102 can be determined. Then, the luma values that are among the luma values obtained in step 1102 and are less than or equal to the average value of the luma values are determined and grouped into the first luma set. Further, the luma values that are among the luma values obtained in step 1102 and are greater than the average value of the luma values are determined and grouped into the second luma set. Thus, the luma values in the first luma set are smaller than the luma values in the second luma set.

[0210] The following formula is used for representation: The average value of the luma values is

Number

[0211] Method 2: Arrange the luma values of the luma samples corresponding to the chroma samples at the preset positions in ascending order to obtain a first luma value queue. When the number of luma samples in the first luma value queue is even, group the luma values in the first half of the first luma value queue into a first luma set, and group the luma values in the second half of the luma value queue into a second luma set.

[0212] In a specific implementation, the luma values obtained in step 1102 can be arranged in ascending order to obtain a first luma value queue. When the number of luma samples in the first luma value queue is even, the luma values in the first half of the first luma value queue can be determined and grouped into a first luma set, and further, the luma values in the second half of the first luma value queue can be determined and grouped into a second luma set.

[0213] For example, in step 1102, four luma values, which are L1, L2, L3, and L4 in sequence, are obtained. If L4 < L2 < L1 < L3, the first luma value queue is L4, L2, L1, and L3. In this case, the first luma set is {L4, L2}, and the second luma set is {L1, L3}.

[0214] Method 3: Arrange the luma values of the luma samples corresponding to the chroma samples at the preset positions in descending order to obtain a second luma value queue. When the number of luma samples in the second luma value queue is even, group the luma values in the second half of the second luma value queue into a first luma set, and group the luma values in the first half of the luma value queue into a second luma set.

[0215] In a particular implementation, the luma values obtained in step 1102 may be sorted in descending order to obtain a second luma value queue. If the number of luma samples in the second luma value queue is even, the luma values in the second half of the second luma value queue may be determined and grouped into a first luma set, and further, the luma values in the first half of the second luma value queue may be determined and grouped into a second luma set.

[0216] For example, in step 1102, if four luma values, which are L1, L2, L3, and L4 in sequence, are obtained and L3 > L1 > L2 > L4, the second luma value queue is L3, L1, L2, and L4. In this case, the first luma set is {L4, L2}, and the second luma set is {L1, L3}.

[0217] Furthermore, when the number of luma samples in the first luma value queue in method 2 is odd, if the first luma value queue contains N luma values, the first (N - 1) / 2 luma values may be grouped into the first luma set, and the last (N + 1) / 2 luma values may be grouped into the second luma set, or alternatively, the first (N + 1) / 2 luma values may be grouped into the first luma set, and the last (N - 1) / 2 luma values may be grouped into the second luma set. Similarly, when the number of luma samples in the second luma value queue in method 3 is odd, if the second luma value queue contains N luma values, the first (N - 1) / 2 luma values may be grouped into the second luma set, and the last (N + 1) / 2 luma values may be grouped into the first luma set, or alternatively, the first (N + 1) / 2 luma values may be grouped into the second luma set, and the last (N - 1) / 2 luma values may be grouped into the first luma set.

[0218] Step 1104: Group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the first luma set into a first chroma set, and group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the second luma set into a second chroma set.

[0219] In a particular implementation, after the first luma set and the second luma set are obtained, the chroma values corresponding to all the luma values in the first luma set can be classified into the first chroma set, and the chroma values corresponding to all the luma values in the second luma set can be classified into the second chroma set. The following equation can be used to represent: The first chroma set is φ L ={C i0 ,C i1 ,…,C ij ,…,C iS}, and The obtained second chroma set is φ R ={C j0 ,C j1 ,…,C ji ,…,C jT}.

[0220] Note that in step 1101, the luma value is determined using the luma sample corresponding to the chroma sample. The chroma value corresponding to the luma value here is the chroma value of the chroma sample corresponding to the luma sample related to the luma value.

[0221] Step 1105: Determine the scaling coefficient in the linear model corresponding to the chroma block based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set.

[0222] In a particular implementation, after the first luma set, the second luma set, the first chroma set, and the second chroma set are determined, the average value of the luma values in the first luma set and the average value of the luma values in the second luma set can be determined, and the average value of the chroma values in the first chroma set and the average value of the chroma values in the second chroma set can be determined. Corresponding to Method 1, the following equation is used to represent: The average value of the luma values in the first luma set is

Number

[0223] Next, based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set, the scaling coefficient in the linear model corresponding to the chroma block is determined.

[0224] Optionally, the scaling coefficient can be determined by using an equation, and the corresponding process can be as follows: [Number] where α is the scaling coefficient in the linear model corresponding to the chroma block, C Lmean is the average value of the chroma values in the first chroma set, C Rmean is the average value of the chroma values in the second chroma set, L Lmean is the average value of the luma values in the first luma set, L Rmean is the average value of the luma values in the second luma set.

[0225] In a specific implementation, a predefined calculation formula for the scaling coefficient can be obtained, and by substituting C Lmean -C Rmean ) / (L Lmean -L Rmean ) with C Lmean , C Rmean , L Lmean , and L Rmean into the calculation formula for the scaling coefficient α = (C

[0226] Step 1106: Determine the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient.

[0227] In a specific implementation, after the scaling coefficient is determined, the offset factor in the linear model corresponding to the chroma block can be determined based on the scaling coefficient. The offset factor can be determined by multiple methods. This embodiment of this application provides two feasible methods.

[0228] Method 1: Determine the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, the average value of the chroma values in the first chroma set, and the average value of the luma values in the first luma set.

[0229] In a specific implementation, the average value of the chroma values in the first chroma set:

Number

Number

[0230] Based on the scaling coefficient, the average value of chroma values in the first chroma set, and the average value of luma values in the first luma set, the offset factor in the linear model corresponding to the chroma block is determined.

[0231] Optionally, the offset factor can be determined by using the following formula, and the corresponding process can be as follows: β = C Lmean -α * L Lmean where α is the scaling coefficient, β is the offset factor in the linear model corresponding to the chroma block, C Lmean is the average value of chroma values in the first chroma set, and L Lmean is the average value of luma values in the first luma set.

[0232] In a specific implementation, a predefined calculation formula for the offset factor can be obtained, and for the calculation formula for the offset factor β = C Lmean -α * L Lmean substitute the scaling coefficient C obtained in step 1105 Lmean and L Lmean to obtain the offset factor in the linear model corresponding to the chroma block.

[0233] Method 2: Determine the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, the average value of chroma values of chroma samples at the preset position, and the average value of luma values of luma samples corresponding to the chroma samples.

[0234] In a specific implementation, the average value of chroma values of chroma samples at the preset position:

Number

Number

[0235] And, by using the scaling factors C mean and L mean the offset factor corresponding to the chroma block is determined.

[0236] Optionally, the offset factor can be determined by using the following equation, and the corresponding process can be as follows: β = C mean - α * L mean where α is a scaling factor, β is the offset factor in the linear model corresponding to the chroma block, C mean is the average value of the chroma values of the chroma samples at the preset position, and L mean is the average value of the luma values of the luma samples corresponding to the chroma samples at the preset position.

[0237] In a specific implementation, a predetermined calculation formula for the offset factor can be obtained, and by substituting the scaling factors C mean and L mean obtained in step 1105 into the calculation formula for the offset factor, the offset factor in the linear model corresponding to the chroma block is obtained.

[0238] Step 1107: Determine the prediction information of the chroma block based on the scaling factor, the offset factor, and the luma reconstruction information corresponding to the chroma block.

[0239] The luma reconstruction information corresponding to the chroma block includes the downsampling information of the luma reconstruction block corresponding to the chroma block.

[0240] In a specific implementation, after obtaining the scaling coefficient and offset factor corresponding to the chroma block, the prediction information of the chroma block can be obtained by using the luma reconstruction information, scaling coefficient, and offset factor corresponding to the chroma block.

[0241] Optionally, to obtain the prediction information of the chroma block, the formula:

Equation

[0242] When this embodiment of this application is applied to the encoder side, there is only a difference in the method of determining the intra prediction mode. On the encoder side, when it is determined that the intra prediction method is used for the chroma block, the chroma block is encoded using each intra prediction mode. Then, using the rate-distortion optimization (RDO) criterion, the intra prediction mode with the best encoding effect is determined, and the intra prediction mode is written into the above-mentioned syntax table, and the intra prediction mode is added to the bitstream. In this way, the decoder side can directly determine the intra prediction mode to be used for decoding from the bitstream.

[0243] Optionally, the method of determining the intra prediction mode by using the RDO criterion can be as follows.

[0244] The prediction information of the chroma block is determined by the method of steps 1101 to 1107, the residual information is obtained by subtracting the prediction information from the original information of the chroma block, the conversion processing is executed on the residual information to obtain the conversion coefficient, the quantization processing is executed on the conversion coefficient to obtain the quantization coefficient, and then the entropy encoding processing is executed on the quantization coefficient to obtain the bit stream. Then, the inverse entropy encoding processing, inverse quantization processing, and inverse conversion processing are sequentially performed on the bit stream to execute intra prediction to obtain the reconstruction information, the reconstruction information is compared with the original information, and the intra prediction mode with the smallest difference between the reconstruction information and the original information is determined as the finally used intra prediction mode, that is, the intra prediction mode written in the above-mentioned syntax table.

[0245] Note that the encoder side can be an encoder side using H.263, H.264, MPEG-2, MPEG-4, VP8, or VP9. Correspondingly, the decoder side can be a decoder side using H.263, H.264, MPEG-2, MPEG-4, VP8, or VP9.

[0246] Note that, furthermore, this embodiment of this application is described by using only the adjacent samples in one row as an example. This embodiment of this application can also be applied to the scenario of the adjacent samples in two rows shown in FIG. 10. The processing method is the same as that in the case of the samples in one row, but the position coordinates of the preset number of position points are different.

[0247] In an embodiment of this application, during encoding or decoding, the chroma value of a chroma sample at a preset position can be obtained from adjacent samples of a chroma block. Then, the luma value of a luma sample corresponding to the chroma sample at the preset position is obtained from adjacent samples of the luma block corresponding to the chroma block, and the obtained luma value is classified into a first luma set and a second luma set. Based on the classification of the luma value, the corresponding chroma value is correspondingly classified into a first chroma set and a second chroma set. Next, based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set, the scaling coefficient in the linear model corresponding to the chroma block can be determined. And after the scaling coefficient is determined, based on the scaling coefficient, the offset factor in the linear model corresponding to the chroma block can be determined. Finally, based on the scaling coefficient, the offset factor, and the luma reconstruction information corresponding to the chroma block, the prediction information of the chroma block is determined. Thus, in encoding or decoding, a preset number of chroma values are selected, and then classified into two chroma sets, and a preset number of luma values are selected and then classified into two luma sets. Based on the average value of the luma values in each luma set and the average value of the chroma values in each chroma set, the scaling coefficient in the linear model corresponding to the chroma block is determined, and further, the offset factor is determined. Since only a smaller number of multiplications are required, the encoding time and the decoding time can be shortened.

[0248] FIG. 17 is a configuration diagram of a chroma block prediction device according to an embodiment of this application. The device can be implemented as part of or the whole of the device by using software, hardware, or a combination thereof. The device provided in this embodiment of this application can implement the procedures of FIGS. 11 to 16(d) in the embodiment of this application. The device includes an acquisition module 1710, a classification module 1720, and a determination module 1730.

[0249] The acquisition module 1710 is configured to obtain the chroma values of the chroma samples at the preset positions from the adjacent samples of the chroma block, and obtain the luma values of the luma samples corresponding to the chroma samples at the preset positions based on the adjacent samples of the luma block corresponding to the chroma block. In particular, it can be configured to implement the acquisition function in FIG. 11 and the implicit steps included in FIG. 11.

[0250] The classification module 1720 is configured to classify the obtained luma values into a first luma set and a second luma set, group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the first luma set into a first chroma set, and group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the second luma set into a second chroma set. In particular, it can be configured to implement the classification function in FIG. 11 and the implicit steps included in FIG. 11.

[0251] The determination module 1730 determines the scaling coefficient in the linear model corresponding to the chroma block based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set. Based on the scaling coefficient, it determines the offset factor in the linear model corresponding to the chroma block, and then determines the prediction information of the chroma block based on the scaling coefficient, the offset factor, and the luma reconstruction information corresponding to the chroma block. The luma reconstruction information corresponding to the chroma block includes the downsampling information of the luma reconstruction block corresponding to the chroma block. In particular, it can be configured to implement the determination function in FIG. 11 and the implicit steps included in FIG. 11.

[0252] Optionally, the acquisition module 1710 further Obtain indication information, and based on the indication information, determine an intra prediction mode corresponding to a chroma block, where the intra prediction mode includes at least one of a linear mode LM, an upper linear mode LMA, and a left linear mode LML. Determine a preset position based on the intra prediction mode corresponding to the chroma block. It is configured as follows.

[0253] Optionally, the acquisition module 1710 Based on a predetermined correspondence between the intra prediction mode and the preset position, and the intra prediction mode corresponding to the chroma block, obtain the chroma value of the chroma sample at the preset position from the adjacent samples of the chroma block. It is configured as follows.

[0254] Optionally, the position coordinates of the upper adjacent samples of the chroma block are (0, -1), (1, -1), …, and (X - 1, -1), and the position coordinates of the left adjacent samples of the chroma block are (-1, 0), (-1, 1), …, and (-1, Y - 1). When the intra prediction mode corresponding to the chroma block is the cross-component linear mode LM, the position coordinates of the chroma samples at the preset position are (0, -1), (X - 1, -1), (-1, -0), and (-1, Y - 1).

[0255] Optionally, the position coordinates of the upper adjacent samples of the chroma block are (0, -1), (1, -1), …, and (X - 1, -1). When the intra prediction mode corresponding to the chroma block is the cross-component linear mode upper LMA, the position coordinates of the preset position are (0, -1), (X / 4, -1), (X - 1 - X / 4, -1), and (X - 1, -1), or (0, -1), (X / 4, -1), (3×X / 4, -1), and (X - 1, -1).

[0256] Optionally, the position coordinates of the left adjacent samples of the chroma block are (-1, 0), (-1, 1), …, and (-1, Y - 1). When the intra prediction mode corresponding to the chroma block is the cross-component linear mode left LML, the position coordinates of the preset positions are (-1, 0), (-1, Y / 4), (-1, Y - 1 - Y / 4), and (-1, Y - 1), or (-1, 0), (-1, Y / 4), (-1, 3×Y / 4), and (-1, Y - 1).

[0257] Optionally, the acquisition module 1710 determines from the adjacent samples of the luma block that the position coordinates of the luma sample corresponding to the chroma sample at the i-th preset position within the preset positions are (2×X i , 2×Y i ), and the position coordinates of the chroma sample at the i-th preset position are (X i , Y i ), and based on the position coordinates of the luma sample corresponding to the chroma sample at the preset position, obtains the luma value of the luma sample corresponding to the chroma sample at the preset position from the adjacent samples of the luma block corresponding to the chroma block, or based on the position coordinates of the chroma sample at the j-th preset position within the preset positions, determines the position coordinates of a plurality of samples corresponding to the j-th preset position from the adjacent samples of the luma block, determines the position coordinates of the luma sample corresponding to the chroma sample at the j-th preset position based on the position coordinates of the plurality of samples, and based on the position coordinates of the luma sample corresponding to the chroma sample at the preset position, obtains the luma value of the luma sample corresponding to the chroma sample at the preset position from the adjacent samples of the luma block corresponding to the chroma block, is configured as such.

[0258] Optionally, the classification module 1720 determines the average value of the luma values of the luma samples corresponding to the chroma samples at the preset position, Group the luma values that are among the luma values of the luma samples corresponding to the chroma samples at the preset positions and are less than or equal to the average value of the luma values into a first luma set, and group the luma values that are among the luma values of the luma samples corresponding to the chroma samples at the preset positions and are greater than the average value of the luma values into a second luma set. It is configured as follows.

[0259] Optionally, classification module 1720 Arrange the luma values of the luma samples corresponding to the chroma samples at the preset positions in ascending order to obtain a first luma value queue, and when the number of luma samples in the first luma value queue is even, group the luma values in the first half of the first luma value queue into the first luma set, and group the luma values in the second half of the luma value queue into the second luma set, or Arrange the luma values of the luma samples corresponding to the chroma samples at the preset positions in descending order to obtain a second luma value queue, and when the number of luma samples in the second luma value queue is even, group the luma values in the second half of the second luma value queue into the first luma set, and group the luma values in the first half of the luma value queue into the second luma set. It is configured as follows.

[0260] Optionally, decision module 1730 α = (C Lmean - C Rmean ) / (L Lmean - L Rmean ) is configured such that α is a scaling coefficient in the linear model corresponding to the chroma block, C Lmean is the average value of the chroma values in the first chroma set, C Rmean is the average value of the chroma values in the second chroma set, L Lmean is the average value of the luma values in the first luma set, and L Rmean is the average value of the luma values in the second luma set.

[0261] Optionally, decision module 1730 Determine an offset factor in a linear model corresponding to a chroma block based on a scaling factor, an average value of chroma values in a first chroma set, and an average value of luma values in a first luma set. It is configured as follows.

[0262] Optionally, determination module 1730 β = C Lmean -α * L Lmean is configured such that, where α is a scaling factor, β is an offset factor in a linear model corresponding to a chroma block, C Lmean is an average value of chroma values in a first chroma set, and L Lmean is an average value of luma values in a first luma set.

[0263] Optionally, determination module 1730 Determine an offset factor in a linear model corresponding to a chroma block based on a scaling factor, an average value of chroma values of chroma samples at a preset position, and an average value of luma values of luma samples corresponding to the chroma samples. It is configured as follows.

[0264] Optionally, determination module 1730 β = C mean -α * L mean is configured such that, where α is a scaling factor, β is an offset factor in a linear model corresponding to a chroma block, C mean is an average value of chroma values of chroma samples at a preset position, and L mean is an average value of luma values of luma samples corresponding to the chroma samples at a preset position.

[0265] In an embodiment of this application, during encoding or decoding, the chroma values of chroma samples at preset positions can be obtained from adjacent samples of a chroma block. Then, the luma values of luma samples corresponding to the chroma samples at the preset positions are obtained from adjacent samples of the luma block corresponding to the chroma block, the obtained luma values are classified into a first luma set and a second luma set, and correspondingly, the chroma values are classified into a first chroma set and a second chroma set based on the classification of the luma values. Next, based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set, the scaling coefficient in the linear model corresponding to the chroma block can be determined, and after the scaling coefficient is determined, the offset factor in the linear model corresponding to the chroma block can be determined based on the scaling coefficient. Finally, based on the scaling coefficient, the offset factor, and the luma reconstruction information corresponding to the chroma block, the prediction information of the chroma block is determined. Thus, in encoding or decoding, a preset number of chroma values are selected and then classified into two chroma sets, and a preset number of luma values are selected and then classified into two luma sets. Based on the average value of the luma values in each luma set and the average value of the chroma values in each chroma set, the scaling coefficient in the linear model corresponding to the chroma block is determined, and further, the offset factor is determined. Since only a smaller number of multiplications are required, the encoding time and the decoding time can be shortened.

[0266] Note that when the chroma block prediction device provided in the above embodiment determines the chroma block prediction information, the division of the above functional modules is merely used as an example for explanation. In actual applications, the above functions can be assigned to different functional modules as needed. In other words, the internal structure of the device is divided into a plurality of different functional modules so as to complete all or part of the described functions. Further, the chroma block prediction device provided in the above embodiment has the same concept as the embodiment of the chroma block prediction method. For details regarding the specific implementation process of the chroma block prediction device, please refer to the method embodiment. Details will not be described again here.

[0267] This application further provides a computer-readable storage medium. The computer-readable storage medium stores instructions, and when the computer-readable storage medium is executed on a computing device, the computing device is enabled to execute the chroma block prediction method.

[0268] This application further provides a computer program product including instructions. When the computer program product is executed on a computing device, the computing device is enabled to execute the chroma block prediction method.

[0269] All or part of the above embodiments can be implemented by using software, hardware, firmware, or any combination thereof. When software is used for implementation, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a server or a terminal, all or part of the procedures or functions according to the embodiments of this application are generated. The computer instructions can be stored in a computer-readable storage medium, or can be sent from a computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be sent from a website, a computer, a server, or a data center to another website, a computer, a server, or a data center in a wired (for example, coaxial optical cable, optical fiber, or digital subscriber line) or wireless (for example, infrared, wireless, or microwave) manner. The computer-readable storage medium can be any available medium accessible by a server or a terminal, or can be a data storage device such as a server or a data center that integrates one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a Digital Video Disk (DVD)), or a semiconductor medium (for example, a solid state drive).

[0270] The above description is only a specific implementation of this application and is not intended to limit this application. Any changes, equivalent substitutions, or improvements made without departing from the spirit and principle of this application should be included in the protection scope of this application.

Claims

1. A method of symbolization, comprising: determining a preset position based on an intra prediction mode for a chroma block; obtaining four chroma values of chroma samples at the preset position from adjacent samples of the chroma block; obtaining four luma values of luma samples corresponding to the four chroma samples at the preset position based on adjacent samples of a luma block corresponding to the chroma block; classifying the four luma values into two smaller luma values and two larger luma values by rearranging the four luma values; grouping the two smaller luma values into a first luma set and grouping the two larger luma values into a second luma set; grouping chroma values of chroma samples corresponding to luma samples related to luma values in the first luma set into a first chroma set, and grouping chroma values of chroma samples corresponding to luma samples related to luma values in the second luma set into a second chroma set; determining a scaling coefficient in a linear model corresponding to the chroma block based on an average value of the luma values in the first luma set, an average value of the luma values in the second luma set, an average value of the chroma values in the first chroma set, and an average value of the chroma values in the second chroma set; determining an offset factor in the linear model corresponding to the chroma block based on the scaling coefficient; determining a prediction block of the chroma block based on the scaling coefficient, the offset factor, and a luma reconstruction block related to the chroma block; obtaining residual information based on the prediction block of the chroma block and the chroma block; encoding the indication information and the residual information into a bit stream, where the indication information is used to indicate the intra prediction mode for the chroma block, and the intra prediction mode is a linear mode left (LML) mode; A method having the above steps.

2. The obtaining of the four chroma values of the chroma samples at the preset position from adjacent samples of the chroma block is: Based on a predetermined correspondence between an intra prediction mode and a preset position, and the intra prediction mode for the chroma block, obtaining the four chroma values of the chroma samples at the preset position from the adjacent samples of the chroma block. The method according to claim 1, comprising this.

3. Classifying the four luma values into two smaller luma values and two larger luma values by rearranging the four luma values, arranging the four luma values of the luma samples corresponding to the chroma samples at the preset position in ascending order to obtain a first luma value queue, wherein the first half part in the first luma value queue has the two smaller luma values of the first luma value queue, and the second half part in the first luma value queue has the two larger luma values of the first luma value queue, or arranging the four luma values of the luma samples corresponding to the chroma samples at the preset position in descending order to obtain a second luma value queue, wherein the first half part in the second luma value queue has the two larger luma values of the second luma value queue, and the second half part in the second luma value queue has the two smaller luma values of the second luma value queue. The method according to claim 1 or 2, comprising this.

4. Determining the offset factor in the linear model corresponding to the chroma block based on the scaling factor, determining the offset factor in the linear model corresponding to the chroma block based on the scaling factor, the average value of the chroma values in the first chroma set, and the average value of the luma values in the first luma set. The method according to any one of claims 1 to 3, comprising this.

5. Determining the offset factor in the linear model corresponding to the chroma block based on the scaling factor, the average value of the chroma values in the first chroma set, and the average value of the luma values in the first luma set, β = C Ｌｍｅａｎ - α * L Ｌｍｅａｎ having that, where α is the scaling coefficient, β is the offset factor in the linear model corresponding to the chroma block, C Ｌｍｅａｎ is the average value of the chroma values in the first chroma set, and L Ｌｍｅａｎ is the average value of the luma values in the first luma set. is the method according to claim 4.

6. An encoding device, a determination module configured to determine a preset position based on an intra prediction mode for a chroma block, An acquisition module configured to obtain four chroma values of the chroma samples at the preset position from adjacent samples of the chroma block, and obtain four luma values of the luma samples corresponding to the four chroma samples at the preset position based on adjacent samples of the luma block corresponding to the chroma block; A classification module configured to classify the four luma values into two smaller luma values and two larger luma values by rearranging the four luma values, group the two smaller luma values into a first luma set, group the two larger luma values into a second luma set, group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the first luma set into a first chroma set, and group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the second luma set into a second chroma set; The determination module is further configured to determine a scaling coefficient in the linear model corresponding to the chroma block based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set, determine an offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, and determine a prediction block of the chroma block based on the scaling coefficient, the offset factor, and the luma reconstruction block related to the chroma block; a classification module; A residual block configured to obtain residual information based on the prediction block of the chroma block and the chroma block; An encoding module configured to encode the indication information and the residual information into a bitstream, where the indication information is used to indicate the intra prediction mode for the chroma block, and the intra prediction mode is a linear mode left (LML) mode; an encoding module; An apparatus having the above.

7. The acquisition module is further configured to Based on a predetermined correspondence between the intra prediction mode and the preset position, and the intra prediction mode for the chroma block, obtaining the four chroma values of the chroma samples at the preset position from the adjacent samples of the chroma block. The apparatus according to claim 6, configured as described above. **Claim 8** The classification module arranges the four luma values of the luma samples corresponding to the chroma samples at the preset position in ascending order to obtain a first luma value queue, where the first half portion within the first luma value queue has the two smaller luma values of the first luma value queue, and the second half portion within the first luma value queue has the two larger luma values of the first luma value queue, or arranges the four luma values of the luma samples corresponding to the chroma samples at the preset position in descending order to obtain a second luma value queue, where the first half portion within the second luma value queue has the two larger luma values of the second luma value queue, and the second half portion within the second luma value queue has the two smaller luma values of the second luma value queue. The apparatus according to claim 6 or 7, configured as described above. **Claim 9** The determination module determines the offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, the average value of the chroma values in the first chroma set, and the average value of the luma values in the first luma set. The apparatus according to any one of claims 6 to 8, configured as described above. **Claim 10** The determination module β = C Ｌｍｅａｎ - α * L Ｌｍｅａｎ where α is the scaling coefficient, β is the offset factor in the linear model corresponding to the chroma block, C Ｌｍｅａｎ is the average value of the chroma values in the first chroma set, and L Ｌｍｅａｎ is the average value of the luma values in the first luma set. The apparatus according to claim 9, configured as described above. **Claim 11** An encoding apparatus having a processing circuit configured to execute the encoding method according to any one of claims 1 to 5. **Claim 12** A computer-readable storage medium storing instructions, wherein when the instructions are executed on a computing device, the computing device is enabled to execute the encoding method according to any one of claims 1 to 5. **Claim 13** A computer program having instructions which, when executed on a computing device, enable the computing device to execute the encoding method according to any one of claims 1 to 5.

14. A decoding method, comprising: analyzing instruction information and residual information from a bitstream, the instruction information being used to indicate an intra prediction mode for a chroma block, the intra prediction mode being a linear mode left (LML) mode; determining a preset position based on the intra prediction mode for the chroma block; obtaining four chroma values of chroma samples at the preset position from adjacent samples of the chroma block; obtaining four luma values of luma samples corresponding to the four chroma samples at the preset position based on adjacent samples of a luma block corresponding to the chroma block; classifying the four luma values into two smaller luma values and two larger luma values by rearranging the four luma values; grouping the two smaller luma values into a first luma set and grouping the two larger luma values into a second luma set; grouping chroma values of chroma samples corresponding to luma samples related to luma values in the first luma set into a first chroma set, and grouping chroma values of chroma samples corresponding to luma samples related to luma values in the second luma set into a second chroma set; determining a scaling coefficient in a linear model corresponding to the chroma block based on an average value of the luma values in the first luma set, an average value of the luma values in the second luma set, an average value of the chroma values in the first chroma set, and an average value of the chroma values in the second chroma set; determining an offset factor in the linear model corresponding to the chroma block based on the scaling coefficient; determining a prediction block of the chroma block based on the scaling coefficient, the offset factor, and a luma reconstruction block related to the chroma block; obtaining a reconstruction block of the chroma block based on the residual information and the prediction block of the chroma block. A method comprising the above.

15. Obtaining the four chroma values of the chroma sample at the preset position from adjacent samples of the chroma block comprises: obtaining the four chroma values of the chroma sample at the preset position from the adjacent samples of the chroma block based on a predefined correspondence between the intra prediction mode and the preset position and the intra prediction mode for the chroma block; The method according to claim 14, comprising: **Claim 16** Classifying the four luma values into two smaller luma values and two larger luma values by rearranging the four luma values comprises: rearranging the four luma values of the luma samples corresponding to the chroma sample at the preset position in ascending order to obtain a first luma value queue, wherein the first half portion in the first luma value queue has the two smaller luma values of the first luma value queue, and the second half portion in the first luma value queue has the two larger luma values of the first luma value queue; or rearranging the four luma values of the luma samples corresponding to the chroma sample at the preset position in descending order to obtain a second luma value queue, wherein the first half portion in the second luma value queue has the two larger luma values of the second luma value queue, and the second half portion in the second luma value queue has the two smaller luma values of the second luma value queue; The method according to claim 14 or 15, comprising: **Claim 17** Determining the offset factor in the linear model corresponding to the chroma block based on the scaling factor comprises: determining the offset factor in the linear model corresponding to the chroma block based on the scaling factor, the average value of the chroma values in the first chroma set, and the average value of the luma values in the first luma set; The method according to any one of claims 14 to 16, comprising: **Claim 18** Determining the offset factor in the linear model corresponding to the chroma block based on the scaling factor, the average value of the chroma values in the first chroma set, and the average value of the luma values in the first luma set comprises: β = C Ｌｍｅａｎ -α * L Ｌｍｅａｎ having that, where α is the scaling coefficient, β is the offset factor in the linear model corresponding to the chroma block, C Ｌｍｅａｎ is the average value of the chroma values in the first chroma set, and L Ｌｍｅａｎ is the average value of the luma values in the first luma set. the method according to claim 17. **Claim 19** A decoding apparatus having a processing circuit, wherein the processing circuit is configured to execute the decoding method according to any one of claims 14 to 18.

20. A computer-readable storage medium storing instructions, wherein when the instructions are executed on a computing device, the computing device is enabled to execute the decoding method according to any one of claims 14 to 18.

21. A computer program having instructions, wherein when the instructions are executed on a computing device, the computing device is enabled to execute the decoding method according to any one of claims 14 to 18.

22. A method for storing a bitstream, receiving or transmitting a bitstream via a communication interface, storing the bitstream in one or more storage media, the bitstream being obtained by performing an encoding process on video data, the bitstream having encoded video data, indication information, and residual information, the indication information being used to indicate an intra prediction mode for a chroma block, the intra prediction mode being a linear mode left (LML) mode, and the encoding process comprising: determining a preset position based on the intra prediction mode for the chroma block, obtaining four chroma values of chroma samples at the preset position from adjacent samples of the chroma block, obtaining four luma values of luma samples corresponding to the four chroma samples at the preset position based on adjacent samples of a luma block corresponding to the chroma block, classifying the four luma values into two smaller luma values and two larger luma values by rearranging the four luma values, grouping the two smaller luma values into a first luma set and grouping the two larger luma values into a second luma set, Group chroma values of chroma samples corresponding to luma samples related to luma values in the first luma set into a first chroma set, and group chroma values of chroma samples corresponding to luma samples related to luma values in the second luma set into a second chroma set. Determine a scaling coefficient in a linear model corresponding to the chroma block based on an average value of the luma values in the first luma set, an average value of the luma values in the second luma set, an average value of the chroma values in the first chroma set, and an average value of the chroma values in the second chroma set. Determine an offset factor in the linear model corresponding to the chroma block based on the scaling coefficient. Determine a prediction block of the chroma block based on the scaling coefficient, the offset factor, and a luma reconstruction block related to the chroma block. Obtain the residual information based on the prediction block of the chroma block and the chroma block. Encode the indication information and the residual information into the bitstream. Having the above. Method.

23. An apparatus for storing a bitstream, having one or more storage media, wherein the one or more storage media are configured to store one or more bitstreams, the bitstream is obtained by performing an encoding process on video data, the bitstream has encoded video data, indication information, and residual information, the indication information is used to indicate an intra prediction mode for a chroma block, the intra prediction mode is a linear mode left (LML) mode, and the encoding process includes determining a preset position based on the intra prediction mode for the chroma block. Obtaining four chroma values of chroma samples at the preset position from adjacent samples of the chroma block. Obtaining four luma values of luma samples corresponding to the four chroma samples at the preset position based on adjacent samples of a luma block corresponding to the chroma block. By rearranging the four luma values, classify the four luma values into two smaller luma values and two larger luma values. Group the two smaller luma values into a first luma set and the two larger luma values into a second luma set, group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the first luma set into a first chroma set, and group the chroma values of the chroma samples corresponding to the luma samples related to the luma values in the second luma set into a second chroma set, determine a scaling coefficient in the linear model corresponding to the chroma block based on the average value of the luma values in the first luma set, the average value of the luma values in the second luma set, the average value of the chroma values in the first chroma set, and the average value of the chroma values in the second chroma set, determine an offset factor in the linear model corresponding to the chroma block based on the scaling coefficient, determine a predicted block of the chroma block based on the scaling coefficient, the offset factor, and the luma reconstruction block related to the chroma block, obtain the residual information based on the predicted block of the chroma block and the chroma block, encode the indication information and the residual information into the bitstream, having, method.

Images