Image component prediction method, encoder, decoder, and storage medium

Abstract

The embodiment of the invention discloses an image component prediction method, an encoder, a decoder and a storage medium, and the method comprises the steps: determining an adjacent reference sampling set of a current block, and determining a preset parameter value corresponding to the current block; wherein the adjacent reference sampling set comprises at least one reference sampling value; caching the adjacent reference sample set and the preset parameter value, and constructing an input reference sample value set; determining an input sampling matrix by using a first preset calculation model according to the reference sample value set; and performing image component prediction on the current block according to the input sampling matrix, and determining a prediction block of the current block.

Description

[0001] Case Analysis This application is a divisional application of patent application No. 201980092311.6, filed on December 19, 2019, entitled "Image Component Prediction Method, Encoder, Decoder and Storage Medium". Technical Field

[0002] This application relates to the field of image processing technology, and in particular to an image component prediction method, an encoder, a decoder, and a storage medium. Background Technology

[0003] As people's demands for video display quality increase, new video application forms such as high-definition and ultra-high-definition video have emerged. H.265 / High Efficiency Video Coding (HEVC) can no longer meet the needs of the rapidly developing video applications. The Joint Video Exploration Team (JVET) proposed the next-generation video coding standard H.266 / Versatile Video Coding (VVC), and its corresponding test model is the VVC Reference Software Test Model (VTM).

[0004] In H.266 / VVC, matrix-based intraprediction (MIP) technology has been adopted. This technology adds different numbers of MIP prediction modes to the intraluminance prediction process depending on the type of intraluminance block. However, the derivation process for MIP prediction is cumbersome and unsuitable for parallel processing due to its dependence on the luma block type, further increasing complexity. Summary of the Invention

[0005] This application provides an image component prediction method, encoder, decoder, and storage medium, which can simplify the derivation process of matrix multiplication input sampling and reduce time complexity.

[0006] The technical solution of this application embodiment can be implemented as follows: In a first aspect, embodiments of this application provide an image component prediction method applied to an encoder, the method comprising: Determine the adjacent reference sample set of the current block, and determine the preset parameter value corresponding to the current block; wherein, the adjacent reference sample set includes at least one reference sample value; The adjacent reference sample set and the preset parameter value are cached to construct the input reference sample set; Based on the input reference sample set, the input sampling matrix is ​​determined using a first preset calculation model; Based on the input sampling matrix, image component prediction is performed on the current block to obtain the predicted block of the current block.

[0007] Secondly, embodiments of this application provide an image component prediction method applied to a decoder, the method comprising: Determine the adjacent reference sample set of the current block, and determine the preset parameter value corresponding to the current block; wherein, the adjacent reference sample set includes at least one reference sample value; The adjacent reference sample set and the preset parameter value are cached to construct the input reference sample set; Based on the input reference sample set, the input sampling matrix is ​​determined using a first preset calculation model; Based on the input sampling matrix, image component prediction is performed on the current block to obtain the predicted block of the current block.

[0008] Thirdly, embodiments of this application provide an encoder, which includes a first determining unit, a first buffering unit, and a first prediction unit; wherein, The first determining unit is configured to determine the adjacent reference sample set of the current block and to determine the preset parameter value corresponding to the current block; wherein the adjacent reference sample set includes at least one reference sample value; The first buffer unit is configured to buffer the adjacent reference sample set and the preset parameter value to construct an input reference sample set; The first determining unit is further configured to determine the input sampling matrix based on the input reference sample set using a first preset calculation model. The first prediction unit is configured to perform image component prediction on the current block based on the input sampling matrix to obtain the prediction block of the current block.

[0009] Fourthly, embodiments of this application provide an encoder, which includes a first memory and a first processor; wherein, A first memory for storing computer programs that can run on a first processor; A first processor is configured to execute the method described in the first aspect when running the computer program.

[0010] Fifthly, embodiments of this application provide a decoder, which includes a second determining unit, a second buffering unit, and a second prediction unit; wherein, The second determining unit is configured to determine the adjacent reference sample set of the current block and to determine the preset parameter value corresponding to the current block; wherein the adjacent reference sample set includes at least one reference sample value; The second buffer unit is configured to buffer the adjacent reference sample set and the preset parameter value to construct an input reference sample set; The second determining unit is further configured to determine the input sampling matrix based on the input reference sample set using a first preset calculation model; The second prediction unit is configured to perform image component prediction on the current block based on the input sampling matrix to obtain the prediction block of the current block.

[0011] Sixthly, embodiments of this application provide a decoder, which includes a second memory and a second processor; wherein, The second memory is used to store computer programs that can run on the second processor; A second processor is configured to execute the method described in the second aspect when running the computer program.

[0012] In a seventh aspect, embodiments of this application provide a computer storage medium storing an image component prediction program, which, when executed by a first processor, implements the method described in the first aspect, or when executed by a second processor, implements the method described in the second aspect.

[0013] This application provides an image component prediction method, encoder, decoder, and storage medium. The method involves determining a neighboring reference sample set for the current block and determining preset parameter values ​​corresponding to the current block; caching the neighboring reference sample set and the preset parameter values ​​to construct an input reference sample set; determining an input sampling matrix using a first preset calculation model based on the input reference sample set; and then performing image component prediction on the current block based on the input sampling matrix to obtain the predicted block for the current block. This approach eliminates the need to determine the type of the current block, reducing time complexity and facilitating hardware implementation. Furthermore, since the input sampling matrix can be determined based on the input reference sample set and the first preset calculation model, the derivation process for matrix multiplication input sampling is simplified, unifying the derivation process of the input sampling matrix. Moreover, this approach is independent of the current block type and enables parallel processing, further reducing computational complexity. Attached Figure Description

[0014] Figure 1 This application provides a schematic block diagram of a video encoding system. Figure 2 This application provides a schematic block diagram of a video decoding system according to an embodiment of the present application. Figure 3 A flowchart illustrating an image component prediction method provided in an embodiment of this application; Figure 4AThis is a structural schematic diagram of a reference pixel position provided in an embodiment of this application; Figure 4B This is a schematic diagram of a reference pixel downsampling process provided in an embodiment of this application; Figure 5A A schematic diagram of a buffer filling structure provided for related technical solutions; Figure 5B A schematic diagram of another buffer filling structure provided for related technical solutions; Figure 5C This application provides a schematic diagram of a buffer filling structure. Figure 6A A schematic diagram of a structure for determining input sampling is provided for related technical solutions; Figure 6B This is a schematic diagram of another structure for determining input sampling provided in an embodiment of this application; Figure 7 A flowchart illustrating another image component prediction method provided in an embodiment of this application; Figure 8 This is a schematic diagram of a structure for generating predicted values ​​provided in an embodiment of this application; Figure 9 A schematic diagram of the composition structure of an encoder provided in an embodiment of this application; Figure 10 This is a schematic diagram of the specific hardware structure of an encoder provided in an embodiment of this application; Figure 11 A schematic diagram of the composition structure of a decoder provided in an embodiment of this application; Figure 12 This is a schematic diagram of the specific hardware structure of a decoder provided in an embodiment of this application. Detailed Implementation

[0015] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0016] In video images, a coding block (CB) is generally represented by a first image component, a second image component, and a third image component. These three image components are a luminance component, a blue chrominance component, and a red chrominance component, respectively. Specifically, the luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V. Thus, video images can be represented in YCbCr format or YUV format.

[0017] In the embodiments of this application, the first image component can be a luminance component, the second image component can be a blue chroma component, and the third image component can be a red chroma component, but the embodiments of this application do not make specific limitations.

[0018] In MIP (Multi-Input Prediction) technology, the input data for MIP prediction can include reference pixels in the previous row and left column of the current block, the MIP prediction mode applied to the current block (represented by modeId), the width and height information of the current block, and whether transposition is required. The output data for MIP prediction can include the predicted value of the current block. The MIP prediction process can be divided into four steps: configuring MIP core parameters, acquiring reference pixels, constructing input samples, and generating predicted values. After these four steps, the predicted value of the current block can be obtained.

[0019] However, in the MIP prediction process, the step of constructing input sampling is quite cumbersome because it is strongly related to the luma block type (which can be represented by mipSizeId). Different luma block types will use different input sampling derivation methods to obtain the input sampling matrix, which makes the derivation process cumbersome. Especially for the case where the luma block type mipSizeId is equal to 0 or 1, the derivation methods of p[0] and p[x] in the input sampling matrix are different, which is not conducive to parallel processing. In addition, for different luma block types, different numbers of MIP prediction modes are added in the intra-frame luma prediction process, and a luma block type judgment needs to be performed once when performing luma prediction for each MIP prediction mode, which further increases the complexity.

[0020] This application provides an image component prediction method. It involves determining a neighboring reference sampling set for the current block and a preset parameter value corresponding to the current block. The neighboring reference sampling set includes at least one reference sampling value. The neighboring reference sampling set and the preset parameter value are cached to construct an input reference sample set. Then, based on the input reference sample set, an input sampling matrix is ​​determined using a first preset calculation model. Finally, image component prediction is performed on the current block based on the input sampling matrix to obtain the predicted block. This approach eliminates the need to determine the type of the current block, reducing time complexity and facilitating hardware implementation. Furthermore, the input sampling matrix can be determined based on the input reference sample set and the first preset calculation model, simplifying the derivation process of matrix multiplication input sampling and unifying the derivation of the input sampling matrix. Moreover, this approach is independent of the current block type and enables parallel processing, further reducing computational complexity.

[0021] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0022] See Figure 1 It illustrates an example block diagram of a video encoding system provided in an embodiment of this application; as shown Figure 1As shown, the video coding system 100 includes a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control and analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image buffer unit 110. The filtering unit 108 can implement deblocking filtering and Sample Adaptive Offset (SAO) filtering, while the encoding unit 109 can implement header information encoding and Context-based Adaptive Binary Arithmetic Coding (CABAC). For the input raw video signal, the system uses coding tree blocks (Coding Tree Blocks) to perform the encoding of the raw video signal. A video coding block is obtained by partitioning the TreeUnit (CTU). Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transform and quantization unit 101. This transformation includes transforming the residual information from the pixel domain to the transform domain and quantizing the resulting transform coefficients to further reduce the bit rate. Intra-frame estimation unit 102 and intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block. Specifically, intra-frame estimation unit 102 and intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video coding block. Motion compensation unit 104 and motion estimation unit 105 are used to perform inter-frame prediction coding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information. The motion estimation performed by motion estimation unit 105 is a process of generating motion vectors, which can estimate the motion of the video coding block. Then, motion compensation unit 104 uses the motion vectors determined by motion estimation unit 105 to generate motion vectors. Motion vectors perform motion compensation; after determining the intra-prediction mode, the intra-prediction unit 103 is also used to provide the selected intra-prediction data to the coding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the coding unit 109; in addition, the inverse transform and inverse quantization unit 106 is used to reconstruct the video coding block, reconstructing the residual block in the pixel domain, the reconstructed residual block is removed by the filter control analysis unit 107 and the filtering unit 108 to remove block artifacts, and then the reconstructed residual block is added to a predictive block in the frame of the decoding image buffer unit 110 to generate the reconstructed video coding block; the coding unit 109 is used to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-prediction mode, outputting the bitstream of the video signal; and the decoding image buffer unit 110 is used to store the reconstructed video coding block for prediction reference.As video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoding image buffer unit 110.

[0023] See Figure 2 It illustrates an example block diagram of a video decoding system provided in an embodiment of this application; as shown Figure 2 As shown, the video decoding system 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoding image buffer unit 206. The decoding unit 201 can perform header information decoding and CABAC decoding, while the filtering unit 205 can perform deblocking filtering and SAO filtering. The input video signal is processed... Figure 1 After encoding, the video signal bitstream is output. This bitstream is input into the video decoding system 200, first passing through the decoding unit 201 to obtain the decoded transform coefficients. These transform coefficients are then processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the pixel domain. The intra-frame prediction unit 203 can generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from previously decoded blocks in the current frame or image. The motion compensation unit 204 determines the prediction information for the video decoding block by analyzing motion vectors and other associated syntax elements, and uses this prediction information... The information is used to generate a predictive block of the video block being decoded; the decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-frame prediction unit 203 or the motion compensation unit 204; the decoded video signal is passed through the filtering unit 205 to remove block artifacts, which can improve video quality; then the decoded video block is stored in the decoding image buffer unit 206, which stores reference images for subsequent intra-frame prediction or motion compensation, and is also used for the output of the video signal, thus obtaining the recovered original video signal.

[0024] The image component prediction method in this application embodiment is mainly applied in, for example, Figure 1 The intra-prediction unit 103 shown and as follows Figure 2The intra-prediction unit 203 shown is an example. In other words, the image component prediction method in this embodiment can be applied to both video coding and video decoding systems, and even simultaneously, but this embodiment does not impose specific limitations. It should also be noted that when the image component prediction method is applied to the intra-prediction unit 103, "current block" specifically refers to the current coding block in intra-prediction; when the image component prediction method is applied to the intra-prediction unit 203, "current block" specifically refers to the current decoding block in intra-prediction.

[0025] Based on the above Figure 1 or Figure 2 For application scenario examples, see Figure 3 This illustrates a flowchart of an image component prediction method provided in an embodiment of this application. Figure 3 As shown, the method may include: S301: Determine the adjacent reference sample set of the current block, and determine the preset parameter value corresponding to the current block; wherein, the adjacent reference sample set includes at least one reference sample value; It should be noted that a video image can be divided into multiple image blocks, and each image block to be encoded can be called a coding block. Here, each coding block may include a first image component, a second image component, and a third image component; and the current block is the coding block in the video image for which the prediction of the first image component, the second image component, or the third image component is currently to be performed.

[0026] In this context, if the current block performs prediction of the first image component, and the first image component is the luminance component, that is, the image component to be predicted is the luminance component, then the current block can also be called the luminance block; or, if the current block performs prediction of the second image component, and the second image component is the chrominance component, that is, the image component to be predicted is the chrominance component, then the current block can also be called the chrominance block.

[0027] It should also be noted that, for the current block, when the left adjacent region, the lower left adjacent region, the upper adjacent region, and the upper right adjacent region are all valid regions, the adjacent reference sampling set can be obtained by filtering the reference pixels in the left and upper adjacent regions of the current block, or by filtering the reference pixels in the left and lower left adjacent regions of the current block, or by filtering the reference pixels in the upper and upper right adjacent regions of the current block. This application embodiment does not make specific limitations.

[0028] In some embodiments, for S301, determining the neighboring reference sample set of the current block may include: Obtain a reference pixel adjacent to at least one edge of the current block; wherein at least one edge of the current block includes at least one of the following: top edge, top right edge, left edge, and bottom left edge; Based on the acquired reference pixels, the adjacent reference sampling set of the current block is determined.

[0029] Here, at least one edge of the current block can be the top edge (also called the top row), the top right edge (also called the top right row), the left edge (also called the left column), or the bottom left edge (also called the bottom left); it can even be a combination of two edges, such as the top edge and the left edge; however, this application embodiment does not limit it.

[0030] Furthermore, in some embodiments, if at least one edge of the current block is a left edge and / or a top edge, for S301, determining the adjacent reference sample set of the current block may include: Obtain reference pixels adjacent to at least one edge of the current block; wherein at least one edge of the current block includes: the top edge and / or the left edge; Based on the acquired reference pixels, the adjacent reference sampling set of the current block is determined.

[0031] Here, at least one edge of the current block may include the left edge and / or the top edge of the current block; that is, at least one edge of the current block may refer to the top edge of the current block, or the left edge of the current block, or even the top edge and the left edge of the current block. This application embodiment does not make specific limitations.

[0032] Furthermore, in some embodiments, for S301, determining the adjacent reference sample set of the current block may include: A first filtering process is performed on the reference pixels adjacent to at least one edge of the current block to determine the reference sample value adjacent to the at least one edge; Based on the determined reference sample values, the adjacent reference sample set of the current block is formed.

[0033] Furthermore, in some embodiments, the method may further include: The first filtering process includes downsampling filtering or low-pass filtering.

[0034] It should be noted that when both the left and top adjacent regions are valid regions, the adjacent reference sampling set can be obtained by filtering the reference pixels adjacent to the left and top edges of the current block. When the left adjacent region is a valid region and the top adjacent region is an invalid region, the adjacent reference sampling set can be obtained by filtering the reference pixels adjacent to the left edge of the current block. When the left adjacent region is an invalid region and the top adjacent region is a valid region, the adjacent reference sampling set can be obtained by filtering the reference pixels adjacent to the top edge of the current block. The filtering here can be downsampling filtering or low-pass filtering; this embodiment does not specifically limit the specific filtering methods.

[0035] In practical applications, when predicting the current block, the blocks above and to the left of the current block have already been encoded. The reference pixels for MIP technology can be the reconstructed values ​​of the reference pixels adjacent to the previous row and the reference pixels adjacent to the left column of the current block. In this case, the reference pixels for the current block can be obtained by acquiring the reference pixels corresponding to the top and left edges of the current block. For example... Figure 4A As shown, it illustrates a schematic diagram of the position of a reference pixel provided in an embodiment of this application. Figure 4A In the current block, the left and top adjacent regions are all valid regions. The reference pixel corresponding to the top edge of the current block is a pixel filled with gray, denoted by refT; the reference pixel corresponding to the left edge of the current block is a pixel filled with diagonal lines, denoted by refL. Therefore, the reference pixels of the current block can include refT and refL, and the adjacent reference sampling set is obtained by filtering refT and refL. It is important to note that for invalid locations (such as image boundaries), the same method used to obtain reference pixels in traditional intra-frame prediction techniques can be used for filling.

[0036] For MIP technology, the current block can be divided into three categories based on its size, and the type of the current block can be recorded using mipSizeId. Specifically, for different types of current blocks, the number of sampling points included in the adjacent reference sampling set and the number of sampling points output by matrix multiplication are different.

[0037] Assuming the current block is a 4×4 image block, then mipSizeId=0. The input reference pixel set selects two pixels for each edge, the matrix multiplication input sampling points are four, and the output is a 4×4 MIP prediction block. Assuming the current block is a 4×N, N×4, or 8×8 image block (where N=8, 16, 32, 64), then mipSizeId=1. The input reference pixel set selects four pixels for each edge, the matrix multiplication input sampling points are eight, and the output is a 4×4 MIP prediction block. Assuming the current block is an image block of other sizes, where mipSizeId=2, then the reference pixel set for the input is four pixels for each side, the number of input sampling points for matrix multiplication is seven, and the output is an 8×8 MIP prediction block.

[0038] Since the width and height of the current block are greater than the number of input sampling points for matrix multiplication, in order to obtain the sample values ​​of the number of input sampling points, it is necessary to downsample the obtained reference pixels (including refT and refL) to obtain the adjacent reference sampling set.

[0039] Furthermore, in some embodiments, determining the neighboring reference sampling set of the current block based on the acquired reference pixels may include: The sampling position of the reference pixel is determined based on at least one edge of the current block; Select reference sample values ​​corresponding to the sampling position from the obtained reference pixels, and form the adjacent reference sample set by combining the selected parameter sample values.

[0040] Further, determining the sampling position of the reference pixel based on at least one edge of the current block may include: At least one edge of the current block is downsampled to determine the sampling position.

[0041] It should be noted that boundarySize reference samples need to be sampled on each reference edge (including the top and left edges) of the current block, and the value of boundarySize is related to the type mipSizeId of the current block.

[0042] Specifically, for a reference pixel refS, the number of reference pixels (i.e., the side length of the current block) is nTbS; the number of reference pixels to be downsampled is boundarySize. The sampling rate of each reference edge can be represented by bDwn, which can be calculated according to equation (1). Furthermore, in the reference pixel refS, an averaging operation is performed on each bDwn reference pixel, and the resulting average value is used as a sampling point in the reference sampling redS, which can be calculated according to equation (2). Here, S can be replaced by W and H respectively, where W represents the top edge and H represents the left edge.

[0043] (1) (2) After the downsampling processing of equations (1) and (2), two parts of reference sampling can be obtained: the upper side reference sampling redT obtained by downsampling the upper side reference pixel refT of the current block, and the left side reference sampling redL obtained by downsampling the left side reference pixel refL of the current block; as shown Figure 4B As shown. In Figure 4B In the example of a 4×4 block, the redL obtained by downsampling on the left side includes two reference pixels, namely reference pixel 1 and reference pixel 2; the redT obtained by downsampling on the top side includes two reference pixels, namely reference pixel 3 and reference pixel 4; thus, the adjacent reference sampling set of the current block includes four reference pixels.

[0044] For MIP technology, in addition to obtaining the current block type mipSizeId, it is also necessary to obtain the bit depth value (which can be represented by BitDepth) corresponding to the image component to be predicted in the current block. Assuming the image component to be predicted is a luma component, the luma bit depth of the current block can be obtained; or, assuming the image component to be predicted is a chroma component, the chroma bit depth of the current block can be obtained, thereby obtaining the preset parameter value of the current block.

[0045] Furthermore, in some embodiments, for S301, determining the preset parameter value corresponding to the current block may include: Obtain the bit depth value corresponding to the image component to be predicted in the current block; The 1 is converted into a binary value, and the binary value is shifted by subtracting 1 from the bit depth value to obtain the preset parameter value.

[0046] In other words, after obtaining the bit depth value BitDepth corresponding to the image component to be predicted in the current block, the preset parameter value can be expressed as 1<<(BitDepth-1).

[0047] In this way, after obtaining the adjacent reference sample set and preset parameter values ​​of the current block, they can be cached to construct the input reference sample set.

[0048] S302: Cache the adjacent reference sample set and the preset parameter value to construct an input reference sample set; It should be noted that after obtaining the adjacent reference sample set, an initial input reference sample set can be constructed first; then, a bit is added to the end of the initial buffer to cache the preset parameter value, thereby obtaining the input reference sample set, which facilitates the subsequent construction of the input sampling matrix.

[0049] In some embodiments, for S302, caching the adjacent reference sample set and the preset parameter value to construct the input reference sample set may include: Cache the adjacent reference sample sets to obtain the initial input reference sample set; Using a data unit following the initial input reference sample set, the preset parameter value is cached to obtain the input reference sample set.

[0050] Furthermore, the construction methods for the initial input reference sample set differ between the encoder and decoder sides, mainly due to the determination of the transpose processing indicator flag (which can be represented by isTransposed).

[0051] When applied to the encoder side, in some embodiments, buffering the adjacent reference sample sets to obtain the initial input reference sample set may include: The value of the transpose processing indicator flag is determined by using a rate-distortion optimization method. When the transpose processing indicator flag is 0, the reference sample value corresponding to the upper edge of the adjacent reference sample set is stored before the reference sample value corresponding to the left edge in the buffer, and the buffer is determined as the initial input reference sample set; or, When the value of the transpose processing indicator flag is 1, in the buffer, after storing the reference sample value corresponding to the upper side of the adjacent reference sample set in the reference sample value corresponding to the left side, the buffer is transposed, and the transposed buffer is determined as the initial input reference sample set.

[0052] It should be noted that the value of the transpose processing indicator can be determined using Rate Distortion Optimization (RDO). For example, by calculating the first-generation value (which requires transpose) and the second-generation value (which does not), if the first-generation value is less than the second-generation value, the transpose processing indicator can be set to 1. In this case, the reference sample value corresponding to the top edge of the adjacent reference sample set can be stored after the reference sample value corresponding to the left edge, or the reference sample value corresponding to the left edge of the adjacent reference sample set can be stored before the reference sample value corresponding to the top edge, i.e., transpose processing is required. If the first-generation value is not less than the second-generation value, the transpose processing indicator can be set to 0. In this case, the reference sample value corresponding to the top edge of the adjacent reference sample set can be stored before the reference sample value corresponding to the left edge, or the reference sample value corresponding to the left edge of the adjacent reference sample set can be stored after the reference sample value corresponding to the top edge, i.e., transpose processing is not required.

[0053] It should also be noted that, on the encoder side, the value of the determined transpose processing indicator flag needs to be written into the bitstream to facilitate subsequent parsing processing on the decoder side.

[0054] When applied to the decoder side, in some embodiments, buffering the adjacent reference sample sets to obtain the initial input reference sample set may include: Parse the bitstream to obtain the value of the transpose processing indicator flag; When the transpose processing indicator flag is 0, the reference sample value corresponding to the upper edge of the adjacent reference sample set is stored before the reference sample value corresponding to the left edge in the buffer, and the buffer is determined as the initial input reference sample set; or, When the value of the transpose processing indicator flag is 1, in the buffer, after storing the reference sample value corresponding to the upper side of the adjacent reference sample set in the reference sample value corresponding to the left side, the buffer is transposed, and the transposed buffer is determined as the initial input reference sample set.

[0055] It should be noted that on the decoder side, the value of the transpose processing indicator flag can be directly obtained by parsing the bitstream; then, based on the value of the transpose processing indicator flag, it can be determined whether the buffer needs to be transposed.

[0056] In other words, after determining the value of the transpose processing indicator flag (isTransposed), assuming the buffer can be represented by pTemp, when isTransposed is 0, pTemp first stores all reference pixels redT corresponding to the top edge, and then, in the next bit after all redT is stored, it continues to store all reference pixels redL corresponding to the left edge; for example... Figure 5A As shown, taking the current 4×4 block as an example, redL includes reference pixel 1 and reference pixel 2, and redT includes reference pixel 3 and reference pixel 4; thus, the buffer order in pTemp is reference pixel 3, reference pixel 4, reference pixel 1 and reference pixel 2; since all the reference pixels corresponding to the upper side are stored before the reference pixels corresponding to the left side, no transpose is needed at this time, and the resulting buffer is the initial input reference sample set.

[0057] When isTransposed is 1, pTemp first stores all reference pixels redL corresponding to the left side, and then in the next position after all redL is stored, it continues to store all reference pixels redT corresponding to the top side; for example... Figure 5BAs shown, taking the current 4×4 block as an example, redL includes reference pixel 1 and reference pixel 2, and redT includes reference pixel 3 and reference pixel 4; thus, the buffer order in pTemp is reference pixel 1, reference pixel 2, reference pixel 3 and reference pixel 4; since all the reference pixels corresponding to the upper side are stored after the reference pixels corresponding to the left side, it is necessary to transpose them at this time, and then determine the transposed buffer as the initial input reference sample set.

[0058] Thus, after obtaining the initial input reference sample set, a data unit can be extended after the initial input reference sample set. This data unit is used to cache the preset parameter value, that is, to store 1 << (BitDepth-1), such as... Figure 5C As shown; in Figure 5C In the example of a 4×4 current block, the initial input reference sample set stores four values, namely the reference sample values ​​after the reference pixels are downsampled; while in the embodiment of this application, the input reference sample set stores five values, in addition to the four reference sample values ​​after the reference pixels are downsampled, an additional preset parameter value is also stored.

[0059] S303: Determine the input sampling matrix using the first preset calculation model based on the input reference sample set; It should be noted that the input sample is a matrix vector that requires matrix multiplication. The current scheme is determined by the initial buffer (represented by pTemp), the current block type (represented by mipSizeId), the bit depth value corresponding to the image component to be predicted (represented by BitDepth), and the number of input samples, and finally obtains the x-th input sample value in the input sampling matrix (represented by P[x]). Among them, when mipSizeId=0 or 1, the sample value at the zeroth position of pTemp needs to be subtracted by 1<<(BitDepth-1) to obtain P[0], and then the sample value at each other position needs to be subtracted from the value at the zeroth position of pTemp to obtain P[x]; as shown below, (3) When mipSizeId=2, the first position of the initial buffer pTemp can be ignored. Then, for each of the other positions, the sample value corresponding to the zeroth position is subtracted and stored in the position preceding it, as shown below. (4) like Figure 6AAs shown, taking the current 4×4 block as an example, the initial buffer stores four values, but the number of input samples is four. At this time, the four input sample values ​​can be determined according to equation (3) or equation (4), denoted by p[x], x=0,1,2,3; thus forming a 1×4 input sampling matrix. However, in the current scheme, the derivation process of matrix multiplication input sampling needs to be related to the current block type mipSizeId, which makes the derivation process cumbersome and has a certain degree of complexity; moreover, for the case of mipSizeId=0 or 1, the calculation formulas of p[0] and p[i] are different, which is not conducive to parallel processing.

[0060] In this embodiment, the initial buffer can be expanded into an input reference sample set and used to store 1 << (BitDepth-1). This makes the derivation process of input sampling no longer related to the current block type mipSizeId, thus unifying the derivation process of matrix multiplication input sampling. Moreover, the input sampling only needs to be determined by the input reference sample set (still denoted by pTemp) and the number of input samples to obtain the i-th input sample value (denoted by p[i]) in the input sampling matrix.

[0061] Specifically, in some embodiments, for S303, determining the input sampling matrix using the first preset calculation model based on the input reference sample set may include: Based on the sample value corresponding to the (i+1)th position and the sample value corresponding to the 0th position in the input reference sample set, the i-th input sample value is calculated using the first preset calculation model; where i is a positive integer greater than or equal to 0 and less than N, and N represents the number of elements contained in the input sampling matrix; The input sampling matrix is ​​formed based on the calculated N input sample values.

[0062] Furthermore, in some embodiments, calculating the i-th input sample value using a first preset calculation model may include: The first preset calculation model is used to perform a subtraction operation to obtain the i-th input sample value.

[0063] Furthermore, in some embodiments, the method may further include: Set the minuend of the subtraction operation to be equal to the sample value corresponding to the (i+1)th position in the reference sample set; The subtrahend of the subtraction operation is set to be equal to the sample value corresponding to the 0th position in the reference sample set.

[0064] In other words, regardless of the current block type, the first position in the input reference sample set, i.e., the sample corresponding to position 0, can be ignored; then, the sample corresponding to each other position is subtracted from the sample corresponding to position 0 and stored in the previous position; here, the first preset calculation model is as follows: (5) Where i = 0, 1, ..., N-1, N is the number of input samples (which can also be represented by inSize), and the number of input samples is the number of elements contained in the input sampling matrix; pTemp[0] represents the sample value corresponding to the 0th position, pTemp[i+1] represents the sample value corresponding to the (i+1)th position, and p[i] represents the i-th input sample value; thus, after obtaining N input sample values, an input sampling matrix can be formed; such as Figure 6B As shown, taking the current block of 4×4 as an example, the input reference sample set stores five values, but the number of input samples is four. At this time, regardless of the type of current block, four input sample values ​​can be determined according to equation (5), denoted by p[i], i=0,1,2,3; thus forming a 1×4 input sampling matrix.

[0065] S304: Perform image component prediction on the current block according to the input sampling matrix to obtain the prediction block of the current block.

[0066] It should be noted that after obtaining the input sampling matrix, the temporary prediction value of at least one pixel in the MIP prediction block can be calculated first; then, after passing through the embedding process, transpose process and upsampling process, the prediction block of the current block is finally obtained.

[0067] It should also be noted that, assuming the image component to be predicted is a luminance component, the current block can be the current luminance block, and a luminance prediction block for the current luminance block can be obtained, which provides a luminance prediction value for at least one pixel; or, assuming the image component to be predicted is a chrominance component, the current block can be the current chrominance block, and a chrominance prediction block for the current chrominance block can be obtained, which provides a chrominance prediction value for at least one pixel; the embodiments of this application do not impose any limitations.

[0068] In some embodiments, for S304, the step of performing image component prediction on the current block based on the input sampling matrix to obtain the prediction block of the current block is as follows: Figure 7 As shown, this step may include: S401: Based on the input sampling matrix, obtain the MIP prediction block of the current block, wherein the MIP prediction block contains the predicted sample values ​​of at least some pixel positions in the current block; It should be noted that after obtaining the input sampling matrix, the weight matrix (denoted by mWeight), shift factor (denoted by sW), and offset factor (denoted by fO) corresponding to the current block can also be obtained; then as follows Figure 8 As shown, a temporary prediction value for at least one pixel in the MIP prediction block (represented by predMip) can be calculated using the second preset calculation model to obtain the MIP prediction block.

[0069] Furthermore, in some embodiments, for S401, obtaining the MIP prediction block of the current block based on the input sampling matrix may include: Obtain the weight matrix, shift factor, and offset factor corresponding to the current block; The input sampling matrix, the weight matrix, the shift factor, and the offset factor are multiplied using a second preset calculation model to calculate the MIP prediction block.

[0070] In other words, a weight matrix table is pre-built in the encoder or decoder, and this weight matrix table is stored in the encoder or decoder. In this way, based on the current block type mipSizeId and MIP prediction mode modeId, the weight matrix mWeight[x][y] to be used for the current block can be determined by looking up the table.

[0071] In addition, a shift factor table, as shown in Table 1, and an offset factor table, as shown in Table 2, are pre-established in the encoder or decoder. These shift factor and offset factor tables are also stored in the encoder or decoder. Thus, based on the current block type mipSizeId and MIP prediction mode modeId, the shift factor sW and offset factor fO required for the current block can be determined by looking up these tables.

[0072] Table 1

[0073] Table 2

[0074] Thus, after obtaining the current block type mipSizeId and the MIP prediction mode modeId, the weight matrix mWeight[x][y], shift factor sW, and offset factor fO can be determined by looking up a table, thereby calculating the MIP prediction block predMip[x][y]. The second preset calculation model is as follows: (6) Where, [x][y] represents the position coordinates of a pixel point, x represents the horizontal direction, and y represents the vertical direction; inSize represents the number of input samples, and predSize represents the side length of the MIP prediction block predMip. Here, predSize is only related to the current block type mipSizeId; when mipSizeId = 0 or 1, the output MIP prediction block is 4×4, then predSize is equal to 4; when mipSizeId = 2, the output MIP prediction block is 8×8, then predSize is equal to 8. Thus, according to the above formula (6), the temporary prediction value of at least one pixel in the MIP prediction block predMip can be calculated to obtain the MIP prediction block.

[0075] S402: Clip the predicted sample values in the MIP prediction block to obtain the MIP prediction block of the current block; It should be noted that after obtaining the temporary prediction value of at least one pixel in the MIP prediction block, the temporary prediction value of at least one pixel in the MIP prediction block can be clipped. Specifically, if the temporary prediction value is less than 0, it can be set to 0; if the temporary prediction value is greater than (1<<BitDepth)-1, it can be set to (1<<BitDepth)-1; thus, the range of the prediction value can be clipped between 0 and (1<<BitDepth)-1.

[0076] In this way, after clipping the MIP prediction block, the prediction value of at least one pixel in the MIP prediction block can be obtained, and the range of the prediction value is between 0 and (1<<BitDepth)-1; then, it is determined whether transposition processing is required according to the transposed sign bit isTransposed to determine the final MIP prediction block.

[0077] S403: Determine whether to perform transposition processing on the MIP prediction block; S404: When the judgment result is "yes", perform transposition processing on the predicted sample values in the MIP prediction block, and determine the transposed MIP prediction block as the MIP prediction block of the current block; S405: When the judgment result is "no", determine the MIP prediction block as the MIP prediction block of the current block; S406: Determine whether the size of the MIP prediction block is the same as the size of the current block; When applied to the encoder side, in some embodiments, for S403, the determination of whether to perform transposition processing on the MIP prediction block may include: Using rate-distortion optimization, the first-generation value of transposing the MIP prediction block and the second-generation value of not transposing the MIP prediction block are calculated respectively. When the value of the first generation is less than the value of the second generation, it is determined that the MIP prediction block should be transposed; or... When the value of the first generation is not less than the value of the second generation, it is determined that the MIP prediction block will not be transposed.

[0078] When applied to the decoder side, in some embodiments, for S403, determining whether to transpose the MIP prediction block may include: Parse the bitstream to obtain the value of the transpose processing indicator flag; Based on the value of the transpose processing indicator flag, determine whether to perform transpose processing on the MIP prediction block.

[0079] It should be noted that the transpose processing indicator flag is denoted by isTransposed. The value of isTransposed can be used to determine whether the MIP prediction block needs transpose processing. Specifically, on the encoder side, if the first-generation value is less than the second-generation value, the value of isTransposed is set to 1, indicating that the MIP prediction block needs to be transposed; or, if the first-generation value is not less than the second-generation value, the value of isTransposed is set to 0, indicating that the MIP prediction block does not need to be transposed. On the decoder side, the value of the transpose processing indicator flag can be obtained by parsing the bitstream; if the parsed value of isTransposed is 1, it indicates that the MIP prediction block needs to be transposed; or, if the parsed value of isTransposed is 0, it indicates that the MIP prediction block does not need to be transposed.

[0080] More specifically, when isTransposed is 0, it indicates that the MIP prediction block does not require transposition. In this case, the MIP prediction block predMip can be used directly for subsequent steps, i.e., step S406 is executed to determine whether the size of the MIP prediction block is the same as the size of the current block. When isTransposed is 0, it indicates that the MIP prediction block requires transposition. In this case, the transposition can be performed using the following formula. (7) Thus, according to equation (7), after transposing the MIP prediction block, the transposed MIP prediction block can be obtained, and the transposed MIP prediction block can be set as the MIP prediction block. Then, step S406 is executed in the same way, that is, it is determined whether the size of the MIP prediction block is the same as the size of the current block.

[0081] S407: When the size of the MIP prediction block is not the same as the size of the current block, perform a second filtering process on the MIP prediction block to obtain the prediction block of the current block; S408: When the size of the MIP prediction block is the same as the size of the current block, the prediction block of the current block is set to be equal to the MIP prediction block; wherein, the prediction block contains the predicted sample values ​​of all pixel positions in the current block.

[0082] Furthermore, the second filtering process may include upsampling filtering or low-pass filtering.

[0083] It should be noted that after obtaining the MIP prediction block, since there are only two sizes of MIP prediction blocks: 4×4 MIP prediction blocks and 8×8 MIP prediction blocks, the size of the current block may be the same as or different from the size of the MIP prediction block. In other words, the sampled values ​​corresponding to the MIP prediction block may not be able to fill the current block, which may require upsampling the MIP prediction block to generate the final prediction value. That is, by judging whether the size of the MIP prediction block is the same as the size of the current block, it is determined whether to perform upsampling on the MIP prediction block.

[0084] Specifically, when the size of the MIP prediction block is the same as the size of the current block (i.e., both the width and height of the MIP prediction block are the same), it indicates that upsampling of the MIP prediction block is not required. In this case, the MIP prediction block can be directly filled into the current block, meaning there are no empty pixels in the filled current block. The predicted value of each pixel in the current block can then be directly set to the predicted value of each pixel in the MIP prediction block, as shown below. (8) Where [x][y] represents the position coordinates of the pixel, x represents the horizontal direction and y represents the vertical direction; predSamples[x][y] represents the predicted value corresponding to the pixel at position coordinates [x][y] in the current block, and predMip[x][y] represents the predicted value corresponding to the pixel at position coordinates [x][y] in the MIP prediction block. Thus, according to equation (8), the MIP prediction block predMip[x][y] can be directly used as the prediction block predSamples[x][y] of the current block.

[0085] When the size of the MIP prediction block is different from the size of the current block, that is, one of the width and height of the MIP prediction block is different from the current block, it indicates that the MIP prediction block needs to be upsampled. After upsampling, the prediction block of the current block can be obtained.

[0086] Specifically, in some embodiments, for S407, when the second filtering process is an upsampling filtering process, the method may further include: Determine the horizontal upsampling factor and the vertical upsampling factor corresponding to the current block; Based on the MIP prediction block, the horizontal upsampling factor, and the vertical upsampling factor, a third preset calculation model is used to determine the predicted value of the pixel position to be filled in the current block, so as to obtain the prediction block of the current block; wherein, the pixel position to be filled is a pixel position in the current block that is different from the pixel position in the MIP prediction block.

[0087] It should be noted that when the size of the MIP prediction block is different from the size of the current block, there are empty pixels in the current block after filling. In this case, the MIP prediction block predMip[x][y] needs to be upsampled using linear interpolation.

[0088] Here, we assume that the width and height of the MIP prediction block are both predSize, the width of the current block is nTbW, and the height of the current block is nTbH. Then, based on the side length predSize of the MIP prediction block and the width nTbW of the current block, the horizontal upsampling factor (denoted by upHor) can be calculated. Similarly, based on the side length predSize of the MIP prediction block and the height nTbH of the current block, the vertical upsampling factor (denoted by upVer) can be calculated. The specific calculation formulas are as follows. (9) Since the MIP prediction block cannot fill the current block, it is necessary to fill the current block according to the horizontal upsampling factor upHor and the vertical upsampling factor upVer, i.e., to perform an upsampling operation. The filling method for the corresponding positions is as follows. (10) Where x = 0, 1, ..., nTbW–1; y = 0, 1, …, nTbH–1.

[0089] Here, the specific method of upsampling is as follows: First, fill the upper side reference pixel refT into the corresponding position predSamples[x][-1] of the previous row of the current block, and then fill the left side reference pixel refL into the corresponding position predSamples[-1][y] of the left column of the current block; then, according to equation (10), for the pixel position to be filled in the current block, such as the empty position between the predicted value of the corresponding position, or the empty position between the reference pixel and the predicted value of the corresponding position, etc., first perform horizontal interpolation, and then perform vertical interpolation, and finally obtain the upsampling result predSamples[x][y] of the current block. This predSamples[x][y] is the predicted value obtained by the current block according to the MIP prediction mode.

[0090] In this embodiment, when the image component prediction method is applied to the encoder side, the predicted value of at least one pixel in the current block can be calculated using the image component prediction method. Then, the residual corresponding to at least one pixel can be calculated based on the difference between the actual value and the predicted value of at least one pixel in the current block, and the calculated residual is written into the bitstream. In addition, after obtaining the transpose processing indicator flag (isTransposed), the value of isTransposed also needs to be written into the bitstream. Then, the bitstream is transmitted from the encoder side to the decoder side. Correspondingly, when the image component prediction method is applied to the decoder side, the value of isTransposed can be determined by parsing the bitstream, and then it can be determined whether transpose processing is required. In addition, the predicted value of at least one pixel in the current block can be calculated using the image component prediction method, and the residual corresponding to at least one pixel can be directly obtained by parsing the bitstream. Based on the predicted value and residual of at least one pixel in the current block, the actual value of at least one pixel in the current block can be obtained.

[0091] This embodiment provides an image component prediction method applied to an encoder or decoder. It involves determining a neighboring reference sample set for the current block and a preset parameter value corresponding to the current block; wherein the neighboring reference sample set includes at least one reference sample value; caching the neighboring reference sample set and the preset parameter value to construct an input reference sample set; determining an input sampling matrix using a first preset calculation model based on the input reference sample set; and performing image component prediction on the current block based on the input sampling matrix to obtain the predicted block of the current block. Thus, the solution of this embodiment eliminates the need to determine the type of the current block, reducing time complexity and facilitating hardware implementation. Furthermore, since the input sampling matrix can be determined based on the input reference sample set and the first preset calculation model, the derivation process of matrix multiplication input sampling is simplified, unifying the derivation process of the input sampling matrix. Moreover, the solution of this embodiment no longer depends on the type of the current block and can achieve parallel processing, reducing computational complexity.

[0092] Based on the same inventive concept as the foregoing embodiments, see [link to previous document]. Figure 9 This illustrates a schematic diagram of the composition structure of an encoder 90 provided in an embodiment of this application. Figure 9 As shown, the encoder 90 may include: a first determining unit 901, a first buffering unit 902, and a first prediction unit 903, wherein, The first determining unit 901 is configured to determine the adjacent reference sampling set of the current block and to determine the preset parameter value corresponding to the current block; wherein, the adjacent reference sampling set includes at least one reference sampling value; The first cache unit 902 is configured to cache the adjacent reference sample set and the preset parameter value to construct an input reference sample set; The first determining unit 901 is further configured to determine the input sampling matrix based on the input reference sample set using a first preset calculation model; The first prediction unit 903 is configured to perform image component prediction on the current block based on the input sampling matrix to obtain the prediction block of the current block.

[0093] In the above scheme, see [reference] Figure 9 The encoder 90 may further include a first acquisition unit 904 configured to acquire a reference pixel adjacent to at least one edge of the current block; wherein at least one edge of the current block includes at least one of the following: top edge, top right edge, left edge, and bottom left edge; The first determining unit 901 is configured to determine the adjacent reference sampling set of the current block based on the acquired reference pixels.

[0094] In the above scheme, see [reference] Figure 9The encoder 90 may further include a first processing unit 905, configured to perform a first filtering process on reference pixels adjacent to at least one edge of the current block to determine reference sample values ​​adjacent to the at least one edge. The first determining unit 901 is configured to form an adjacent reference sample set of the current block based on the determined reference sample values.

[0095] In the above scheme, the first filtering process includes downsampling filtering or low-pass filtering.

[0096] In the above scheme, the first acquisition unit 904 is further configured to acquire the bit depth value corresponding to the image component to be predicted in the current block; The first processing unit 905 is further configured to convert 1 into a binary value and perform a shift operation on the binary value by subtracting 1 from the bit depth value to obtain the preset parameter value.

[0097] In the above scheme, the first cache unit 902 is configured to cache the adjacent reference sample set to obtain the initial input reference sample set; and to use a data unit after the initial input reference sample set to cache the preset parameter value to obtain the input reference sample set.

[0098] In the above scheme, the first determining unit 901 is also configured to determine the value of the transpose processing indicator flag in a way that optimizes utilization distortion. The first buffer unit 902 is specifically configured such that when the transpose processing indicator flag is 0, the reference sample value corresponding to the upper side of the adjacent reference sample set is stored before the reference sample value corresponding to the left side in the buffer, and the buffer is determined as the initial input reference sample set; or, when the transpose processing indicator flag is 1, the reference sample value corresponding to the upper side of the adjacent reference sample set is stored after the reference sample value corresponding to the left side in the buffer, the buffer is transposed, and the transposed buffer is determined as the initial input reference sample set.

[0099] In the above scheme, see [reference] Figure 9 The encoder 90 may further include a first calculation unit 906, configured to calculate the i-th input sample value using a first preset calculation model based on the sample value corresponding to the (i+1)-th position and the sample value corresponding to the 0-th position in the input reference sample set; where i is a positive integer greater than or equal to 0 and less than N, and N represents the number of elements contained in the input sampling matrix; The first determining unit 901 is configured to form the input sampling matrix based on the calculated N input sample values.

[0100] In the above scheme, the first calculation unit 906 is specifically configured to perform subtraction operation using the first preset calculation model to obtain the i-th input sample value.

[0101] In the above scheme, the first calculation unit 906 is specifically configured to set the minuend of the subtraction operation to be equal to the sample value corresponding to the (i+1)th position in the reference sample value set; and to set the subtrahend of the subtraction operation to be equal to the sample value corresponding to the 0th position in the reference sample value set.

[0102] In the above scheme, the first acquisition unit 904 is further configured to obtain the MIP prediction block of the current block according to the input sampling matrix, wherein the MIP prediction block includes the predicted sampling values ​​of at least some pixel positions in the current block; The first processing unit 905 is further configured to perform a second filtering process on the MIP prediction block to obtain a prediction block of the current block when one of the width and height of the MIP prediction block is not the same as the current block; or, when the width and height of the MIP prediction block are both the same as the current block, set the prediction block of the current block to be equal to the MIP prediction block; wherein the prediction block contains the predicted sample values ​​of all pixel positions in the current block.

[0103] In the above scheme, the first acquisition unit 904 is specifically configured to perform embedding processing on the predicted sample values ​​in the MIP prediction block to obtain the MIP prediction block of the current block.

[0104] In the above scheme, see [reference] Figure 9 The encoder 90 may further include a first judgment unit 907, configured to determine whether to perform transpose processing on the MIP prediction block; and when the judgment result is "yes", to perform transpose processing on the prediction sample value in the MIP prediction block, and to determine the transposed MIP prediction block as the MIP prediction block of the current block.

[0105] In the above scheme, the first calculation unit 906 is also configured to calculate the first generation value of the MIP prediction block after transposing and the second generation value of the MIP prediction block without transposing in a utilization distortion optimization manner. The first judgment unit 907 is specifically configured to determine that the MIP prediction block should be transposed when the value of the first generation is less than the value of the second generation; or, when the value of the first generation is not less than the value of the second generation, determine that the MIP prediction block should not be transposed.

[0106] In the above scheme, the second filtering process includes upsampling filtering or low-pass filtering.

[0107] In the above scheme, the first acquisition unit 904 is further configured to acquire the weight matrix, shift factor and offset factor corresponding to the current block; The first calculation unit 906 is further configured to perform matrix multiplication on the input sampling matrix, the weight matrix, the shift factor, and the offset factor using a second preset calculation model to calculate the MIP prediction block.

[0108] In the above scheme, when the second filtering process is an upsampling filtering process, the first determining unit 901 is further configured to determine the horizontal upsampling factor and the vertical upsampling factor corresponding to the current block; The first calculation unit 906 is further configured to determine the predicted value of the pixel position to be filled in the current block using a third preset calculation model based on the MIP prediction block, the horizontal upsampling factor and the vertical upsampling factor, so as to obtain the prediction block of the current block; wherein the pixel position to be filled is a pixel position in the current block that is different from the pixel position in the MIP prediction block.

[0109] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0110] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] Therefore, this application provides a computer storage medium applied to an encoder 90, the computer storage medium storing an image component prediction program, which, when executed by a first processor, implements the method described in any of the foregoing embodiments.

[0112] Based on the composition of the encoder 90 and the computer storage medium described above, see [link to documentation]. Figure 10 This illustrates a specific hardware structure example of the encoder 90 provided in this application embodiment, which may include: a first communication interface 1001, a first memory 1002, and a first processor 1003; the various components are coupled together through a first bus system 1004. It is understood that the first bus system 1004 is used to implement the connection and communication between these components. In addition to a data bus, the first bus system 1004 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 10 The general designated all buses as the first bus system 1004. Among them, The first communication interface 1001 is used for receiving and sending signals during the process of sending and receiving information with other external network elements; The first memory 1002 is used to store computer programs that can run on the first processor 1003; The first processor 1003 is configured to, when running the computer program, execute: Determine the adjacent reference sample set of the current block, and determine the preset parameter value corresponding to the current block; wherein, the adjacent reference sample set includes at least one reference sample value; The adjacent reference sample set and the preset parameter value are cached to construct the input reference sample set; Based on the input reference sample set, the input sampling matrix is ​​determined using a first preset calculation model; Based on the input sampling matrix, image component prediction is performed on the current block to obtain the predicted block of the current block.

[0113] It is understood that the first memory 1002 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The first memory 1002 of the system and method described in this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0114] The first processor 1003 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware or by software instructions within the first processor 1003. The first processor 1003 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the first memory 1002. The first processor 1003 reads the information in the first memory 1002 and completes the steps of the above method in conjunction with its hardware.

[0115] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), DSP devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0116] Alternatively, as another embodiment, the first processor 1003 is further configured to execute the method described in any of the foregoing embodiments when running the computer program.

[0117] This embodiment provides an encoder, which may include a first determining unit, a first buffering unit, and a first prediction unit. The first determining unit is configured to determine the adjacent reference sampling set of the current block and to determine the preset parameter value corresponding to the current block. The first buffering unit is configured to buffer the adjacent reference sampling set and the preset parameter value to construct an input reference sample set. The first determining unit is further configured to determine an input sampling matrix based on the input reference sample set using a first preset calculation model. The first prediction unit is configured to perform image component prediction on the current block based on the input sampling matrix to obtain the predicted block of the current block. This not only simplifies the derivation process of matrix multiplication input sampling and unifies the derivation process of the input sampling matrix, but also eliminates the need to determine the type of the current block, reducing time complexity and facilitating hardware implementation.

[0118] Based on the same inventive concept as the foregoing embodiments, see [link to previous document]. Figure 11 This illustrates a schematic diagram of the composition structure of a decoder 110 provided in an embodiment of this application. Figure 11 As shown, the decoder 110 may include: a second determining unit 1101, a second buffering unit 1102, and a second prediction unit 1103, wherein, The second determining unit 1101 is configured to determine the adjacent reference sampling set of the current block and to determine the preset parameter value corresponding to the current block; wherein, the adjacent reference sampling set includes at least one reference sampling value; The second cache unit 1102 is configured to cache the adjacent reference sample set and the preset parameter value to construct an input reference sample set; The second determining unit 1101 is further configured to determine the input sampling matrix based on the input reference sample set using a first preset calculation model. The second prediction unit 1103 is configured to perform image component prediction on the current block based on the input sampling matrix to obtain the prediction block of the current block.

[0119] In the above scheme, see [reference] Figure 11 The decoder 110 may further include a second acquisition unit 1104, configured to acquire a reference pixel adjacent to at least one edge of the current block; wherein at least one edge of the current block includes at least one of the following: top edge, top right edge, left edge, and bottom left edge; The second determining unit 1101 is configured to determine the adjacent reference sampling set of the current block based on the acquired reference pixels.

[0120] In the above scheme, see [reference] Figure 11 The decoder 110 may further include a second processing unit 1105, configured to perform a first filtering process on reference pixels adjacent to at least one edge of the current block to determine reference sample values ​​adjacent to the at least one edge. The second determining unit 1101 is configured to form an adjacent reference sample set of the current block based on the determined reference sample values.

[0121] In the above scheme, the first filtering process includes downsampling filtering or low-pass filtering.

[0122] In the above scheme, the second acquisition unit 1104 is further configured to acquire the bit depth value corresponding to the image component to be predicted in the current block; The second processing unit 1105 is further configured to convert 1 into a binary value and perform a shift operation on the binary value by subtracting 1 from the bit depth value to obtain the preset parameter value.

[0123] In the above scheme, the second cache unit 1102 is configured to cache the adjacent reference sample set to obtain the initial input reference sample set; and to use a data unit after the initial input reference sample set to cache the preset parameter value to obtain the input reference sample set.

[0124] In the above scheme, see [reference] Figure 11 The decoder 110 may also include a parsing unit 1106, configured to parse the bitstream and obtain the value of the transpose processing indicator flag; The second buffer unit 1102 is specifically configured such that when the transpose processing indicator flag is 0, the reference sample value corresponding to the upper side of the adjacent reference sample set is stored before the reference sample value corresponding to the left side in the buffer, and the buffer is determined as the initial input reference sample set; or, when the transpose processing indicator flag is 1, the reference sample value corresponding to the upper side of the adjacent reference sample set is stored after the reference sample value corresponding to the left side in the buffer, the buffer is transposed, and the transposed buffer is determined as the initial input reference sample set.

[0125] In the above scheme, see [reference] Figure 11 The decoder 110 may further include a second computing unit 1107, configured to calculate the i-th input sample value using a first preset computing model based on the sample value corresponding to the (i+1)-th position and the sample value corresponding to the 0-th position in the input reference sample set; where i is a positive integer greater than or equal to 0 and less than N, and N represents the number of elements contained in the input sampling matrix; The second determining unit 1101 is configured to form the input sampling matrix based on the calculated N input sample values.

[0126] In the above scheme, the second calculation unit 1107 is specifically configured to perform subtraction operation using the first preset calculation model to obtain the i-th input sample value.

[0127] In the above scheme, the second calculation unit 1107 is specifically configured to set the minuend of the subtraction operation to be equal to the sample value corresponding to the (i+1)th position in the reference sample value set; and to set the subtrahend of the subtraction operation to be equal to the sample value corresponding to the 0th position in the reference sample value set.

[0128] In the above scheme, the second acquisition unit 1104 is further configured to obtain the MIP prediction block of the current block according to the input sampling matrix, wherein the MIP prediction block includes the predicted sampling values ​​of at least some pixel positions in the current block; The second processing unit 1105 is further configured to perform a second filtering process on the MIP prediction block to obtain a prediction block of the current block when one of the width and height of the MIP prediction block is not the same as the current block; or, when the width and height of the MIP prediction block are both the same as the current block, set the prediction block of the current block to be equal to the MIP prediction block; wherein the prediction block contains the predicted sample values ​​of all pixel positions in the current block.

[0129] In the above scheme, the second acquisition unit 1104 is specifically configured to perform embedding processing on the predicted sample values ​​in the MIP prediction block to obtain the MIP prediction block of the current block.

[0130] In the above scheme, see [reference] Figure 11 The decoder 110 may further include a second judgment unit 1108, configured to determine whether to perform transpose processing on the MIP prediction block; and when the judgment result is "yes", to perform transpose processing on the prediction sample value in the MIP prediction block, and to determine the transposed MIP prediction block as the MIP prediction block of the current block.

[0131] In the above scheme, the parsing unit 1106 is specifically configured to parse the bitstream and obtain the value of the transpose processing indicator flag; The second judgment unit 1108 is specifically configured to determine whether to perform transpose processing on the MIP prediction block based on the value of the transpose processing indicator flag.

[0132] In the above scheme, the second filtering process includes upsampling filtering or low-pass filtering.

[0133] In the above scheme, the second acquisition unit 1104 is further configured to acquire the weight matrix, shift factor and offset factor corresponding to the current block; The second calculation unit 1107 is further configured to perform matrix multiplication on the input sampling matrix, the weight matrix, the shift factor, and the offset factor using a second preset calculation model to calculate the MIP prediction block.

[0134] In the above scheme, the second determining unit 1101 is further configured to determine the horizontal upsampling factor and the vertical upsampling factor corresponding to the current block; The second calculation unit 1107 is further configured to determine the predicted value of the pixel position to be filled in the current block using a third preset calculation model based on the MIP prediction block, the horizontal upsampling factor and the vertical upsampling factor, so as to obtain the prediction block of the current block; wherein the pixel position to be filled is a pixel position in the current block that is different from the pixel position in the MIP prediction block.

[0135] Understandably, in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular component. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0136] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer storage medium applied to the decoder 110, which stores an image component prediction program. When the image component prediction program is executed by a second processor, it implements the method described in any of the foregoing embodiments.

[0137] Based on the composition of the decoder 110 and the computer storage medium described above, see [link to documentation]. Figure 12 This illustrates a specific hardware structure example of the decoder 110 provided in this application embodiment, which may include: a second communication interface 1201, a second memory 1202, and a second processor 1203; the various components are coupled together through a second bus system 1204. It is understood that the second bus system 1204 is used to implement the connection and communication between these components. In addition to a data bus, the second bus system 1204 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 12 The various buses are all labeled as the second bus system 1204. Among them, The second communication interface 1201 is used for receiving and sending signals during the process of sending and receiving information with other external network elements; The second memory 1202 is used to store computer programs that can run on the second processor 1203; The second processor 1203 is configured to, when running the computer program, perform: Determine the adjacent reference sample set of the current block, and determine the preset parameter value corresponding to the current block; wherein, the adjacent reference sample set includes at least one reference sample value; The adjacent reference sample set and the preset parameter value are cached to construct the input reference sample set; Based on the input reference sample set, the input sampling matrix is ​​determined using a first preset calculation model; Based on the input sampling matrix, image component prediction is performed on the current block to obtain the predicted block of the current block.

[0138] Alternatively, as another embodiment, the second processor 1203 is also configured to perform the method described in any of the foregoing embodiments when running the computer program.

[0139] It is understood that the second memory 1202 has similar hardware functions to the first memory 1002, and the second processor 1203 has similar hardware functions to the first processor 1003; these will not be described in detail here.

[0140] This embodiment provides a decoder, which may include a second determining unit, a second buffering unit, and a second prediction unit. The second determining unit is configured to determine the adjacent reference sampling set of the current block and to determine the preset parameter value corresponding to the current block. The second buffering unit is configured to buffer the adjacent reference sampling set and the preset parameter value to construct an input reference sample set. The second determining unit is further configured to determine an input sampling matrix based on the input reference sample set using a first preset calculation model. The second prediction unit is configured to perform image component prediction on the current block based on the input sampling matrix to obtain the predicted block of the current block. This not only simplifies the derivation process of matrix multiplication input sampling and unifies the derivation process of the input sampling matrix, but also eliminates the need to determine the type of the current block, reducing time complexity and facilitating hardware implementation.

[0141] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0142] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0143] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0144] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0145] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0146] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0147] Industrial applicability In this embodiment, the adjacent reference sampling set of the current block and the preset parameter value corresponding to the current block are determined; the adjacent reference sampling set and the preset parameter value are cached to construct an input reference sample set; then, based on the input reference sample set, the input sampling matrix is ​​determined using a first preset calculation model; and finally, image component prediction is performed on the current block based on the input sampling matrix to obtain the predicted block of the current block. Thus, the solution in this embodiment does not require determining the type of the current block, reducing time complexity and facilitating hardware implementation. Furthermore, since the input sampling matrix can be determined based on the input reference sample set and the first preset calculation model, the derivation process of matrix multiplication input sampling is simplified, unifying the derivation process of the input sampling matrix. Moreover, the solution in this embodiment no longer depends on the type of the current block and can achieve parallel processing, reducing computational complexity.

Claims

1. An image component prediction method, applied to a decoder, the method comprising: Determine the adjacent reference sample set of the current block, wherein the adjacent reference sample set includes at least one reference sample value; Determining the preset parameter value corresponding to the current block includes: performing bit shifting on the binary value corresponding to the number 1 to obtain the preset parameter value, wherein the number of bits shifted is equal to the bit depth value corresponding to the image component to be predicted of the current block minus 1; The input sampling matrix is ​​determined based on the adjacent reference sampling set and the preset parameter values; Based on the input sampling matrix, determine the MIP prediction block of the current block; The predicted sample values ​​in the MIP prediction block are clamped. When the width or height of the MIP prediction block is not the same as that of the current block, the MIP prediction block is upsampled to determine the prediction block of the current block; When the width and height of the MIP prediction block are the same as those of the current block, the prediction block of the current block is set to be equal to that of the MIP prediction block.

2. The method according to claim 1, wherein, The determination of the adjacent reference sample set of the current block includes: Obtain the reference sample of the top edge adjacent to the top edge of the current block and the reference sample of the left edge adjacent to the left edge of the current block; The downsampled upper side reference sample value is determined by downsampling the upper side reference sample, and the downsampled left side reference sample value is determined by downsampling the left side reference sample. The adjacent reference sample set of the current block is determined based on the downsampled upper side reference sample value and the downsampled left side reference sample value.

3. The method according to claim 2, wherein, The step of determining the adjacent reference sample set of the current block based on the downsampled upper side reference sample value and the downsampled left side reference sample value includes: When the transpose processing indicator flag is set to 1, the downsampled left side reference sample value is stored before the downsampled upper side reference sample value in the adjacent reference sample set. When the value of the transpose processing indicator flag is 0, the downsampled upper side reference sample value is stored before the downsampled left side reference sample value in the adjacent reference sample set.

4. The method according to claim 1, wherein, The step of determining the input sampling matrix based on the adjacent reference sampling set and the preset parameter value includes: Determine the size parameters of the current block; When the size parameter of the current block is equal to the first value or the second value, the i-th input sample value is calculated based on the sample value corresponding to the i-th position and the sample value corresponding to the 0-th position in the adjacent reference sample set, and the 0-th input sample value is calculated based on the preset parameter value and the sample value corresponding to the 0-th position in the adjacent reference sample set. When the size parameter value of the current block is equal to the third value, the i-th input sample value is calculated based on the sample value corresponding to the (i+1)-th position and the sample value corresponding to the 0-th position in the adjacent reference sample set.

5. The method according to claim 4, wherein, Determining the size parameters of the current block includes: When the width and height of the current block are both equal to 4, the value of the size parameter is equal to the first value; When the width or height of the current block is equal to 4, or when both the width and height of the current block are equal to 8, the value of the size parameter is equal to the second value; Otherwise, the value of the size parameter is equal to the third value.

6. The method according to claim 1, wherein, The step of determining the MIP prediction block of the current block based on the input sampling matrix includes: Determine the weight matrix corresponding to the current block; The MIP prediction block of the current block is determined based on the input sampling matrix, the weight matrix, the shift factor, and the offset factor.

7. An image component prediction method applied to an encoder, the method comprising: Determine the adjacent reference sample set of the current block, wherein the adjacent reference sample set includes at least one reference sample value; Determining the preset parameter value corresponding to the current block includes: performing bit shifting on the binary value corresponding to the number 1 to obtain the preset parameter value, wherein the number of bits shifted is equal to the bit depth value corresponding to the image component to be predicted of the current block minus 1; The input sampling matrix is ​​determined based on the adjacent reference sampling set and the preset parameter values; Based on the input sampling matrix, determine the MIP prediction block of the current block; The predicted sample values ​​in the MIP prediction block are clamped. When the width or height of the MIP prediction block is not the same as that of the current block, the MIP prediction block is upsampled to determine the prediction block of the current block; When the width and height of the MIP prediction block are the same as those of the current block, the prediction block of the current block is set to be equal to that of the MIP prediction block.

8. The method according to claim 7, wherein, The determination of the adjacent reference sample set of the current block includes: Obtain the reference sample of the top edge adjacent to the top edge of the current block and the reference sample of the left edge adjacent to the left edge of the current block; The downsampled upper side reference sample value is determined by downsampling the upper side reference sample, and the downsampled left side reference sample value is determined by downsampling the left side reference sample. The adjacent reference sample set of the current block is determined based on the downsampled upper side reference sample value and the downsampled left side reference sample value.

9. The method according to claim 8, wherein, The step of determining the adjacent reference sample set of the current block based on the downsampled upper side reference sample value and the downsampled left side reference sample value includes: When the transpose processing indicator flag is set to 1, the downsampled left side reference sample value is stored before the downsampled upper side reference sample value in the adjacent reference sample set. When the value of the transpose processing indicator flag is 0, the downsampled upper side reference sample value is stored before the downsampled left side reference sample value in the adjacent reference sample set.

10. The method according to claim 7, wherein, The step of determining the input sampling matrix based on the adjacent reference sampling set and the preset parameter value includes: Determine the size parameters of the current block; When the size parameter of the current block is equal to the first value or the second value, the i-th input sample value is calculated based on the sample value corresponding to the i-th position and the sample value corresponding to the 0-th position in the adjacent reference sample set, and the 0-th input sample value is calculated based on the preset parameter value and the sample value corresponding to the 0-th position in the adjacent reference sample set. When the size parameter value of the current block is equal to the third value, the i-th input sample value is calculated based on the sample value corresponding to the (i+1)-th position and the sample value corresponding to the 0-th position in the adjacent reference sample set.

11. The method according to claim 10, wherein, Determining the size parameters of the current block includes: When the width and height of the current block are both equal to 4, the value of the size parameter is equal to the first value; When the width or height of the current block is equal to 4, or when both the width and height of the current block are equal to 8, the value of the size parameter is equal to the second value; Otherwise, the value of the size parameter is equal to the third value.

12. The method according to claim 7, wherein, The step of determining the MIP prediction block of the current block based on the input sampling matrix includes: Determine the weight matrix corresponding to the current block; The MIP prediction block of the current block is determined based on the input sampling matrix, the weight matrix, the shift factor, and the offset factor.

13. A decoder, the encoder comprising a first memory and a first processor; wherein, A first memory for storing computer programs that can run on a first processor; A first processor is configured to perform the method as described in any one of claims 1 to 6 when running the computer program.

14. An encoder, the decoder comprising a second memory and a second processor; wherein, The second memory is used to store computer programs that can run on the second processor; A second processor is configured to perform the method as described in any one of claims 7 to 12 when running the computer program.

15. A computer-readable storage medium having a computer program and a bitstream stored thereon, characterized in that, When executed by a processor, the computer program implements the method of any one of claims 7 to 12 to generate the bitstream.

Images