Encoding and decoding method and device, encoding equipment, decoding equipment and storage medium
Patent Information
- Application Number
- CN202280094511.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-12
- Publication Date
- 2024-11-22
AI Technical Summary
There is a large deviation between the prediction value of the coding block calculated by the inter-color component prediction technology of H.266/VVC and the original value, resulting in low prediction accuracy and affecting coding performance.
By filtering the reference information of the current block, the target information is determined, and the weighting coefficient is calculated based on the color component reference sample value in the target information, and then the prediction block of the color component is determined to improve the accuracy of chroma prediction and encoding and decoding performance.
The accuracy of chroma prediction is improved, the code rate is saved, the encoding and decoding performance is improved, and the problem of low prediction accuracy in the existing technology is solved.
Smart Images

Figure CN119013986A_ABST
Abstract
Description
Coding and decoding method, device, encoding device, decoding device and storage medium Technical Field
[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular to a coding and decoding method, apparatus, encoding device, decoding device, and storage medium. Background Art
[0002] As demand for video display quality increases, new video applications such as high-definition and ultra-high-definition video are emerging. The Joint Video Exploration Team (JVET) of the international standards organizations ISO / IEC and ITU-T has developed the next-generation video coding standard H.266 / Versatile Video Coding (VVC).
[0003] H.266 / VVC includes inter-color prediction technology. However, the predicted values of coding blocks calculated by this technology deviate significantly from the original values. This results in low prediction accuracy, degraded decoded video quality, and reduced encoding performance.
[0004] Summary of the Invention
[0005] The embodiments of the present application provide a coding and decoding method, apparatus, coding device, decoding device, and storage medium, which can not only improve the accuracy of chrominance prediction and save bit rate, but also improve coding and decoding performance.
[0006] The technical solution of the embodiment of the present application can be implemented as follows:
[0007] In a first aspect, an embodiment of the present application provides a decoding method, comprising:
[0008] Determine the reference information of the current block;
[0009] Filtering part of the reference information of the current block to determine the target information of the current block;
[0010] Determining a weighting coefficient according to a reference sample value of the first color component in the target information;
[0011] Determining a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information;
[0012] Determine a reconstructed value of a second color component sampling point of the current block according to the prediction block.
[0013] In a second aspect, an embodiment of the present application provides an encoding method, including:
[0014] Determine the reference information of the current block;
[0015] Filtering part of the reference information of the current block to determine the target information of the current block;
[0016] Determining a weighting coefficient according to a reference sample value of the first color component in the target information;
[0017] Determining a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information;
[0018] Determine a predicted difference value of a second color component sampling point of the current block according to the predicted block.
[0019] In a third aspect, an embodiment of the present application provides an encoding device, comprising a first determining unit, a first filtering unit, and a first predicting unit; wherein,
[0020] a first determining unit, configured to determine reference information of a current block;
[0021] a first filtering unit configured to filter part of the reference information of the current block to determine target information of the current block;
[0022] The first determining unit is further configured to determine a weighting coefficient according to a reference sample value of the first color component in the target information;
[0023] a first prediction unit configured to determine a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information;
[0024] The first determining unit is further configured to determine, according to the prediction block, a predicted difference value of a second color component sampling point of the current block.
[0025] In a fourth aspect, an embodiment of the present application provides an encoding device, comprising a first memory and a first processor; wherein,
[0026] a first memory for storing a computer program capable of running on the first processor;
[0027] The first processor is configured to execute the method according to the second aspect when running a computer program.
[0028] In a fifth aspect, an embodiment of the present application provides a decoding device, comprising a second determining unit, a second filtering unit, and a second prediction unit; wherein,
[0029] a second determining unit, configured to determine reference information of a current block;
[0030] a second filtering unit configured to filter part of the reference information of the current block to determine target information of the current block;
[0031] The second determining unit is further configured to determine a weighting coefficient according to a reference sample value of the first color component in the target information;
[0032] a second prediction unit configured to determine a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information;
[0033] The second determining unit is further configured to determine the reconstructed value of the second color component sampling point of the current block according to the predicted block.
[0034] In a sixth aspect, an embodiment of the present application provides a decoding device, including a second memory and a second processor; wherein,
[0035] a second memory for storing a computer program capable of running on the second processor;
[0036] The second processor is configured to execute the method according to the first aspect when running a computer program.
[0037] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, it implements the method described in the first aspect or the method described in the second aspect.
[0038] Embodiments of the present application provide a coding and decoding method, apparatus, coding device, decoding device, and storage medium. At either the coding or decoding end, the method determines reference information for a current block; filters a portion of the reference information for the current block to determine target information for the current block; determines a weighting coefficient based on reference samples of a first color component in the target information; and determines a prediction block for a second color component of the current block based on the weighting coefficient and reference samples of a second color component in the target information. Thus, at the coding end, the prediction difference of a sampling point of the second color component of the current block can be determined based on the prediction block; and at the decoding end, the reconstructed value of a sampling point of the second color component of the current block can be determined based on the prediction block. This improves the accuracy of the reference information for color component information in adjacent areas of the current block and color component information within the current block by fully considering the characteristics of color format information. Furthermore, filtering such as upsampling and downsampling the reference information can enhance the continuity of its spatial correlation to a certain extent or remove certain information with weak spatial correlation. The target information obtained after filtering can be used to better perform chroma prediction, thereby improving the accuracy of chroma prediction, saving bit rate, and enhancing coding and decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] FIG1 is a schematic diagram showing the distribution of effective adjacent areas;
[0040] FIG2 is a schematic diagram showing the distribution of selected areas under different prediction modes;
[0041] FIG3 is a flow chart of a model parameter derivation scheme;
[0042] FIG4A is a schematic block diagram of an encoder provided in an embodiment of the present application;
[0043] FIG4B is a schematic block diagram of a decoder according to an embodiment of the present application;
[0044] FIG5 is a flowchart of a decoding method according to an embodiment of the present application;
[0045] FIG6 is a first schematic diagram of a reference area of a current block provided by an embodiment of the present application;
[0046] FIG7A is a schematic diagram of downsampling positions of reference brightness information provided in an embodiment of the present application;
[0047] FIG7B is a schematic diagram of upsampling positions of reference chrominance information provided in an embodiment of the present application;
[0048] FIG8 is a schematic diagram of rearranged positions of reference brightness information provided by an embodiment of the present application;
[0049] FIG9 is a schematic diagram of weighted prediction of a WCP mode and other prediction modes provided in an embodiment of the present application;
[0050] FIG10 is a schematic diagram of a framework for weighted chrominance prediction provided by an embodiment of the present application;
[0051] FIG11 is a second flow chart of a decoding method provided in an embodiment of the present application;
[0052] FIG12 is a first schematic diagram of an upsampling interpolation processing process provided by an embodiment of the present application;
[0053] FIG13 is a second schematic diagram of an upsampling interpolation processing process provided by an embodiment of the present application;
[0054] FIG14 is a schematic diagram of weight values for upsampling interpolation provided in an embodiment of the present application;
[0055] FIG15 is a third schematic diagram of an upsampling interpolation processing process provided by an embodiment of the present application;
[0056] FIG16 is a third flow chart of a decoding method provided in an embodiment of the present application;
[0057] FIG17 is a second schematic diagram of a reference area of a current block provided by an embodiment of the present application;
[0058] FIG18 is a flowchart diagram 1 of an encoding method provided in an embodiment of the present application;
[0059] FIG19 is a second flow chart of an encoding method provided in an embodiment of the present application;
[0060] FIG20 is a schematic diagram of the structure of an encoding device provided in an embodiment of the present application;
[0061] FIG21 is a schematic diagram of a specific hardware structure of an encoding device provided in an embodiment of the present application;
[0062] FIG22 is a schematic diagram of the structure of a decoding device provided in an embodiment of the present application;
[0063] FIG23 is a schematic diagram of a specific hardware structure of a decoding device provided in an embodiment of the present application;
[0064] FIG24 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0065] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.
[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.
[0067] In the following description, reference is made to "some embodiments," which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first, second, and third" in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first, second, and third" may be interchanged in a specific order or sequential order where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0068] In a video image, a first color component, a second color component, and a third color component are generally used to represent a coding block (CB); wherein the three color components are a luminance component, a blue chrominance component, and a red chrominance component. For example, the luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V; thus, the video image can be represented in YCbCr format or YUV format. In addition, the video image can also be in RGB format, YCgCo format, etc., which is not limited in the embodiments of the present application.
[0069] It can be understood that in the current video image or video encoding and decoding process, the cross-component prediction technology mainly includes the cross-component linear model (CCLM) prediction mode and the multi-directional linear model (MDLM) prediction mode. Regardless of the model parameters derived according to the CCLM prediction mode or the model parameters derived according to the MDLM prediction mode, the corresponding prediction models can realize the prediction between color components such as the first color component to the second color component, the second color component to the first color component, the first color component to the third color component, the third color component to the first color component, the second color component to the third color component, or the third color component to the second color component.
[0070] Taking the prediction from the first color component to the second color component as an example, assuming that the first color component is the luminance component and the second color component is the chrominance component, in order to reduce the redundancy between the luminance component and the chrominance component, the CCLM prediction mode is used in H.266 / Versatile Video Coding (VVC), that is, the chrominance prediction value is constructed according to the luminance reconstruction value of the same coding block, such as: Pred C (i, j) = α·Rec L (i,j)+β.
[0071] Among them, i, j represent the position coordinates of the pixel to be predicted in the coding block, i represents the horizontal direction, and j represents the vertical direction; Pred C (i, j) represents the chroma prediction value corresponding to the pixel to be predicted at the position coordinate (i, j) in the coding block, Rec L (i, j) represents the reconstructed brightness value of the pixel to be predicted at position (i, j) in the same coding block (after downsampling). In addition, α and β represent model parameters, which can be derived from the reference pixels.
[0072] For a coding block, its adjacent areas can be divided into five parts: the left adjacent area, the upper adjacent area, the lower left adjacent area, the upper left adjacent area, and the upper right adjacent area. H.266 / VVC includes three cross-component linear model prediction modes, namely: the left and upper adjacent intra-frame CCLM prediction mode (which can be represented by INTRA_LT_CCLM), the left and lower left adjacent intra-frame CCLM prediction mode (which can be represented by INTRA_L_CCLM), and the upper and upper right adjacent intra-frame CCLM prediction mode (which can be represented by INTRA_T_CCLM). In these three prediction modes, each prediction mode can select a preset number (for example, 4) of reference pixels for the derivation of model parameters α and β. The biggest difference between these three prediction modes is that the selected areas corresponding to the reference pixels used to derive the model parameters α and β are different.
[0073] Specifically, for the coding block size corresponding to the chrominance component is W×H, assuming that the upper selection area corresponding to the reference pixel is W′, and the left selection area corresponding to the reference pixel is H′; thus,
[0074] For INTRA_LT_CCLM mode, the reference pixels can be selected from the upper adjacent area and the left adjacent area, that is, W′=W, H′=H;
[0075] For INTRA_L_CCLM mode, the reference pixel can be selected from the left adjacent area and the lower left adjacent area, that is, H′=W+H, and W′=0;
[0076] For the INTRA_T_CCLM mode, reference pixels may be selected from the upper adjacent area and the upper right adjacent area, ie, W′=W+H, and H′=0.
[0077] It should be noted that in VTM, only pixels in the W range are stored in the upper right adjacent area, and only pixels in the H range are stored in the lower left adjacent area. Although the selection area of INTRA_L_CCLM mode and INTRA_T_CCLM mode is defined as W+H, in actual application, the selection area of INTRA_L_CCLM mode will be limited to H+H, and the selection area of INTRA_T_CCLM mode will be limited to W+W; thus,
[0078] For INTRA_L_CCLM mode, the reference pixels can be selected from the left adjacent area and the lower left adjacent area, H′=min{W+H,H+H};
[0079] For the INTRA_T_CCLM mode, reference pixels can be selected from the upper adjacent area and the upper right adjacent area, where W′=min{W+H,W+W}.
[0080] See Figure 1, which shows a schematic diagram of the distribution of valid adjacent regions. In Figure 1, the left adjacent region, the lower left adjacent region, the upper adjacent region, and the upper right adjacent region are all valid; in addition, the gray-filled block is the pixel to be predicted at position (i, j) in the coding block.
[0081] Thus, based on Figure 1, the selection areas for the three prediction modes are shown in Figure 2. In Figure 2, (a) shows the selection area for the INTRA_LT_CCLM mode, including the left adjacent area and the upper adjacent area; (b) shows the selection area for the INTRA_L_CCLM mode, including the left adjacent area and the lower left adjacent area; and (c) shows the selection area for the INTRA_T_CCLM mode, including the upper adjacent area and the upper right adjacent area. Thus, after determining the selection areas for the three prediction modes, pixels for model parameter derivation can be selected within the selection areas. The pixels selected in this way can be called reference pixels, and the number of reference pixels is usually 4. Moreover, for a W×H coding block of a certain size, the positions of the reference pixels are generally fixed.
[0082] After obtaining a preset number of reference pixels, chrominance prediction is currently performed according to the flow diagram of the model parameter derivation scheme shown in Figure 3. According to the process shown in Figure 3, assuming that the preset number is 4, the process may include:
[0083] S301: Acquire reference pixels in a selected area.
[0084] S302: Determine the number of valid reference pixels.
[0085] S303: If the number of valid reference pixels is 0, the model parameter α is set to 0 and β is set to a default value.
[0086] S304: The chroma prediction value is filled with a default value.
[0087] S305: If the number of valid reference pixels is 4, two reference pixels with larger values and two reference pixels with smaller values in the luminance component are obtained through comparison.
[0088] S306: Calculate the mean point corresponding to the larger value and the mean point corresponding to the smaller value.
[0089] S307: Derivation of model parameters α and β based on the two mean points.
[0090] S308: Perform chroma prediction using the prediction model constructed using α and β.
[0091] It should be noted that in VVC, the step of valid reference pixels being 0 is judged based on the validity of adjacent areas.
[0092] It should also be noted that the prediction model is constructed using the principle of "two points determine a straight line". The two points here can be called fitting points. In the current technical solution, after obtaining 4 reference pixels, the two reference pixels with larger values and the two reference pixels with smaller values in the brightness component are obtained through comparison; then, a mean point (which can be used as mean) is obtained based on the two reference pixels with larger values. max According to the two reference pixels with smaller values, another mean point is obtained (mean can be used min Represents), we can get two mean points mean max and mean min ; Then mean max and mean min As two fitting points, model parameters (represented by α and β) can be derived; finally, a prediction model is constructed based on α and β, and prediction processing of the chrominance component is performed based on the prediction model.
[0093] However, in the related art, a simple linear model Pred is used for each coding block. C (i, j) = α·Rec L (i, j) + β to predict the chrominance component, and the same model parameters α and β are used for prediction for all pixels at any position in each coding block. This leads to the following drawbacks: First, coding blocks with different content characteristics all use a simple linear model to map luminance to chrominance to achieve chrominance prediction. However, not every luminance-to-chrominance mapping function within a coding block can be accurately fitted by this simple linear model, resulting in inaccurate prediction for some coding blocks. Second, during the prediction process, the same model parameters α and β are used for pixels at different positions within the coding block, resulting in significant variation in prediction accuracy across different locations within the coding block. Furthermore, the CCLM prediction process fails to fully account for the high correlation between the reconstructed luminance information of the current block and the reference information of adjacent regions. This results in inaccurate prediction for some coding blocks, thus reducing the gain of the technique. Simply put, the current CCLM technique results in significant deviations between the predicted and original values for some coding blocks, resulting in low prediction accuracy, reduced quality, and lower encoding and decoding efficiency.
[0094] Based on this, an embodiment of the present application provides an encoding method, which determines the reference information of the current block; filters part of the information in the reference information of the current block to determine the target information of the current block; determines the weighting coefficient based on the reference sample value of the first color component in the target information; determines the prediction block of the second color component of the current block based on the weighting coefficient and the reference sample value of the second color component in the target information; and determines the prediction difference of the sampling point of the second color component of the current block based on the prediction block.
[0095] An embodiment of the present application also provides a decoding method, which determines the reference information of the current block; filters part of the information in the reference information of the current block to determine the target information of the current block; determines a weighting coefficient based on the reference sample value of the first color component in the target information; determines a prediction block of the second color component of the current block based on the weighting coefficient and the reference sample value of the second color component in the target information; and determines a reconstructed value of the sampling point of the second color component of the current block based on the prediction block.
[0096] In this way, for the color component information in the adjacent areas of the current block and the color component information in the current block, the accuracy of these reference information can be improved because the characteristics of the color format information are fully considered; and by performing filtering processing such as upsampling and downsampling on these reference information, the continuity of its spatial correlation can be enhanced to a certain extent or some information with weak spatial correlation can be removed. The target information obtained after filtering processing can be used to better perform chroma prediction, thereby improving the accuracy of chroma prediction, saving bit rate, and improving encoding and decoding performance.
[0097] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0098] Referring to FIG4A , which shows a schematic block diagram of the composition of an encoder provided in an embodiment of the present application. As shown in FIG4A , the encoder (specifically, a “video encoder”) 100 may include a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image cache unit 110, etc., wherein the filtering unit 108 can implement deblocking filtering and sample adaptive offset (SAO) filtering, and the encoding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (CABAC).For the input original video signal, a video coding block can be obtained by dividing it into coding tree units (CTUs). Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transform and quantization unit 101, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block; specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video coding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-frame prediction coding on the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is the process of generating a motion vector, which can estimate the motion of the video coding block, and the motion compensation unit 104 then calculates the motion vector based on the motion vector determined by the motion estimation unit 105. After determining the intra-frame prediction mode, the intra-frame prediction unit 103 is further configured to provide the selected intra-frame prediction data to the encoding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the encoding unit 109. In addition, the inverse transform and inverse quantization unit 106 is configured to reconstruct the video coding block and reconstruct a residual block in the pixel domain. The reconstructed residual block is subjected to the filter control analysis unit 107 and the filtering unit 108 to remove the block effect artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video coding block. The encoding unit 109 is configured to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-frame prediction mode, and output the code stream of the video signal. The decoded image buffer unit 110 is configured to store the reconstructed video coding block for prediction reference. As the video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoded image buffer unit 110 .
[0099] Referring to FIG4B , which shows a block diagram of a decoder provided in an embodiment of the present application, as shown in FIG4B , the decoder (specifically, a "video decoder") 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image buffer unit 206. The decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering. After the input video signal is encoded as shown in FIG4A , a code stream of the video signal is output; the code stream is input to the decoder 200 and first passes through the decoding unit 201 to obtain decoded transform coefficients; the transform coefficients are processed by the inverse transform and inverse quantization unit 202 to generate a residual block in the pixel domain; the intra-frame prediction unit 203 can be used to generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from the previously decoded block of the current frame or picture; the motion compensation unit 204 determines the prediction information for the video decoding block by analyzing the motion vector and other associated syntax elements, and uses The prediction information is used to generate a predictive block for the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-frame prediction unit 203 or the motion compensation unit 204; the decoded video signal passes through the filtering unit 205 to remove blocking artifacts, thereby improving video quality; the decoded video block is then stored in the decoded image buffer unit 206, which stores reference images used for subsequent intra-frame prediction or motion compensation, and is also used for outputting the video signal, thereby obtaining the restored original video signal.
[0100] It should be noted that the method of the embodiment of the present application is mainly applied to the intra-frame prediction unit 103 shown in Figure 4A and the intra-frame prediction unit 203 shown in Figure 4B. In other words, the embodiment of the present application can be applied to both the encoder and the decoder, or even to both the encoder and the decoder at the same time, but the embodiment of the present application is not specifically limited thereto.
[0101] It should also be noted that, when applied to the intra-frame prediction unit 103, the "current block" specifically refers to the coding block currently to be intra-frame predicted; when applied to the intra-frame prediction unit 203, the "current block" specifically refers to the decoding block currently to be intra-frame predicted.
[0102] In one embodiment of the present application, referring to FIG5 , a flowchart of a decoding method provided by an embodiment of the present application is shown. As shown in FIG5 , the method may include:
[0103] S501: Determine reference information of the current block.
[0104] It should be noted that the decoding method of the embodiment of the present application is applied to a decoding device, or a decoding device integrated with the decoding device (also referred to as a "decoder"). In addition, the decoding method of the embodiment of the present application can specifically refer to an intra-frame prediction method, more specifically, a pre-processing operation method of weighted chroma prediction (WCP).
[0105] In an embodiment of the present application, a video image can be divided into a plurality of decoding blocks, each of which can include a first color component, a second color component, and a third color component. The current block herein refers to a decoding block in the video image that is currently to be intra-frame predicted. In addition, assuming that the current block predicts the first color component, and the first color component is a luminance component, that is, the component to be predicted is a luminance component, then the current block can also be referred to as a luminance prediction block; or, assuming that the current block predicts the second color component, and the second color component is a chrominance component, that is, the component to be predicted is a chrominance component, then the current block can also be referred to as a chrominance prediction block.
[0106] It should also be noted that, in embodiments of the present application, the reference information of the current block may include the values of the first color component sampling points in the adjacent areas of the current block and the values of the second color component sampling points in the adjacent areas of the current block. These sampling points (Samples) may be determined based on decoded pixels in the adjacent areas of the current block. In some embodiments, the adjacent areas of the current block may include at least one of the following: an upper adjacent area, an upper right adjacent area, a left adjacent area, and a lower left adjacent area.
[0107] Here, the upper adjacent area and the upper right adjacent area can be considered as the upper area, and the left adjacent area and the lower left adjacent area can be considered as the left area. In addition, the adjacent area can also include the upper left area, as shown in Figure 6. When predicting the second color component of the current block, the upper area, left area, and upper left area of the current block as adjacent areas can all be called reference areas of the current block, and the pixels in the reference areas are all decoded reference pixels.
[0108] In addition, the reference information of the current block may also include the reconstructed value of the first reference color component sampling point in the current block. Therefore, in some embodiments, determining the reference information of the current block may include:
[0109] Reference information of the current block is determined according to the value of the first color component sampling point in the adjacent area of the current block, the value of the second color component sampling point in the adjacent area of the current block, and the reconstructed value of the first reference color component sampling point in the current block.
[0110] It should be noted that, in the embodiment of the present application, the reference pixel of the current block may refer to a reference pixel point adjacent to the current block, or may be referred to as a first color component sampling point or a second color component sampling point in an adjacent area of the current block, and may be represented by Neighboring Sample or Reference Sample. The term "adjacent" here may refer to spatial proximity, but is not limited thereto. For example, "adjacent" may refer to temporal proximity, spatial and temporal proximity, or even a reference pixel of the current block obtained by performing some processing on spatially adjacent, temporally adjacent, or spatially and temporally adjacent reference pixels, and the embodiment of the present application does not impose any limitation thereto.
[0111] It should also be noted that in the embodiment of the present application, the first color component is the luminance component, and the second color component is the chrominance component. Therefore, the value of the first color component sampling point in the adjacent area of the current block represents the reference luminance information corresponding to the reference pixel of the current block, and the value of the second color component sampling point in the adjacent area of the current block represents the reference chrominance information corresponding to the reference pixel of the current block.
[0112] In some embodiments, determining the reference pixel of the current block may include: performing a screening process on pixels in an adjacent area of the current block to determine the reference pixel.
[0113] Specifically, in an embodiment of the present application, a first reference pixel set is formed based on pixels in an adjacent area of the current block. This first reference pixel set can then be screened to determine the reference pixels. Here, the number of reference pixels can be M, where M is a positive integer. In other words, M reference pixels can be selected from the pixels in the adjacent area. The value of M can generally be 4, but this is not a specific limitation.
[0114] It should also be noted that among the pixels in the adjacent area of the current block, there may be some unimportant pixels (for example, these pixels have poor correlation) or some abnormal pixels. In order to ensure the accuracy of the prediction, these pixels need to be eliminated in order to obtain valid reference pixels.
[0115] In a specific embodiment, screening pixels in the adjacent area to determine the reference pixel may include:
[0116] Determining a pixel position to be selected based on positions and / or color component intensities of pixels in a neighboring area;
[0117] According to the position of the pixel to be selected, a reference pixel is determined from pixels in an adjacent area.
[0118] It should be noted that in the embodiment of the present application, the color component intensity can be represented by color component information, such as reference luminance information, reference chrominance information, etc. Here, the larger the value of the color component information, the higher the color component intensity. In this way, the pixels in the adjacent area can be screened based on the pixel position or the color component intensity, thereby determining the reference pixel of the current block based on the screened pixels, and further determining the value of the first color component sampling point in the adjacent area of the current block and the value of the second color component sampling point in the adjacent area of the current block.
[0119] S502: Filter part of the reference information of the current block to determine target information of the current block.
[0120] In an embodiment of the present application, because the number of values of the first color component sampling points and the number of values of the second color component sampling points in the adjacent areas of the current block are different, it may be necessary to filter the values of the first color component sampling points or the values of the second color component sampling points to make the numbers of the two equal. In addition, because the number of reconstructed values of the first reference color component sampling points in the current block is different from a preset size, it may also be necessary to filter the reconstructed values of the first reference color component sampling points in the current block to convert the number of reconstructed values of the first reference color component sampling points obtained from the current block to the preset size for input. In other words, for the reference information of the current block, filtering can be performed only on the values of the first color component sampling points in the adjacent areas of the current block, or only on the values of the second color component sampling points in the adjacent areas of the current block, or only on the reconstructed values of the first reference color component sampling points in the current block, or on any two of these, or even all three, without limitation in this embodiment. These situations will be described below.
[0121] In a possible implementation, filtering part of the reference information of the current block to determine the target information of the current block may include:
[0122] Performing a first filtering process on the values of the first color component sampling points in the adjacent area of the current block to obtain filtered sample values of the first color component in the adjacent area of the current block;
[0123] Target information of the current block is determined according to filtered samples of the first color component in the adjacent area of the current block.
[0124] In an embodiment of the present application, the number of filtered sample values (Sample) of the first color component in the adjacent area of the current block is equal to L, where L represents the number of values (Sample) of the second color component sampling points in the adjacent area of the current block, and L is a positive integer.
[0125] In a specific embodiment, the first filtering process may be a downsampling filtering process. For example, the amount of reference luminance information of the current block is generally greater than the amount of reference chrominance information, and thus the reference luminance information of the current block may be downsampled and filtered so that the amount of reference luminance information is the same as the amount of reference chrominance information.
[0126] It should also be noted that since the amount of reference luminance information and the amount of reference chrominance information of the current block will also be affected by the color format information; therefore, in some embodiments, the method may also include: based on the color format information, performing a first filtering process on the value of the first color component sampling point in the adjacent area of the current block to obtain the filtered sample value of the first color component in the adjacent area of the current block.
[0127] In a specific embodiment, the first filtering process may further include:
[0128] If the color format information indicates 4:4:4 sampling, downsampling filtering is not performed on the values of the first color component sampling points in the adjacent areas of the current block;
[0129] If the color format information indicates 4:2:2 sampling, downsampling filtering is performed on the values of the first color component sampling points in the adjacent areas of the current block;
[0130] If the color format information indicates 4:2:0 sampling, downsampling filtering is performed on the values of the first color component sampling points in the adjacent areas of the current block.
[0131] It should be noted that, in the embodiment of the present application, when the color format information indicates 4:2:0 sampling, the downsampling rate is the reciprocal of a positive integer multiple of 2.
[0132] It should also be noted that, in the embodiment of the present application, the color format information may include 4:4:4 sampling, 4:2:2 sampling, 4:2:0 sampling, etc. Among them, if the color format information indicates 4:4:4 sampling (also expressed as YUV444), that is, the ratio of luminance to chrominance samples is 4:4:4, then the number of reference luminance information obtained from the adjacent area is the same as the number of reference chrominance information, and no operation is required; if the color format information indicates 4:2:2 sampling (also expressed as YUV422), that is, the ratio of luminance to chrominance samples is 4:2:2, or the color format information indicates 4:2:0 sampling (also expressed as YUV420), that is, the ratio of luminance to chrominance samples is 4:1:1, then the number of reference luminance information obtained from the adjacent area is different from the number of reference chrominance information. In this case, the reference luminance information obtained from the adjacent area needs to be downsampled and filtered to match the number of reference chrominance information.
[0133] In this way, for the downsampling filtering of the reference luminance information obtained from the adjacent area, a multi-tap filter (such as a four-tap filter, a five-tap filter, a six-tap filter, etc.) can be used for downsampling, or a pixel of multiple reference luminance pixels at the corresponding position of the reference chrominance pixel can be obtained, or even other downsampling methods can be used, which are not specifically limited in the embodiments of the present application.
[0134] For example, as shown in FIG7A , when the color format information indicates 4:2:0 sampling, taking an adjacent area of 8×4 luminance pixels as an example, it needs to be downsampled to an area of 4×2 pixels. (a) represents the reference luminance information refY, the bold frame represents the four luminance pixels corresponding to the same chrominance pixel, and (k, l) represents the corresponding upper left corner pixel position; (b) represents the downsampled reference luminance information refY', the bold frame represents a luminance pixel after downsampling, and the same chrominance pixel corresponding to the four luminance pixels in the bold frame in (a) is the luminance and chrominance pixel pair at the same position; (i, j) represents the position of the downsampled luminance pixel in the downsampled reference luminance information; wherein i = 0,…,3, j = 0,1.
[0135] Furthermore, when the number of reference luminance information after downsampling and filtering is L and is the same as the number of reference chrominance information, it is also necessary to match L with a preset number. Therefore, in some embodiments, the method may further include:
[0136] Compare L with the preset number;
[0137] Based on the comparison result of L and the preset number, performing a second filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block;
[0138] Target information of the current block is determined according to filtered adjacent samples of the first color component of the current block and filtered adjacent samples of the second color component of the current block.
[0139] In a specific embodiment, performing a second filtering process on the filtered samples of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block based on the comparison result of L with the preset number to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block may include:
[0140] If L is greater than a preset number, performing a first sub-filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block;
[0141] If L is less than a preset number, a second sub-filtering process is performed on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0142] It should be noted that in the embodiment of the present application, the preset number can be represented by inSize. Here, inSize represents the number of inputs when making predictions based on weighted coefficients, and the value of inSize is related to the type of the current block. For example, assuming that the width of the current block is nTbW and the height of the current block is nTbH, then the value of inSize can be 2×nTbH+2×nTbW, or 1.5×nTbH+1.5×nTbW, or even nTbH+nTbW, which is not specifically limited here.
[0143] It should also be noted that in the embodiment of the present application, if L is equal to the preset number, then no operation is required; if L is greater than the preset number, then the reference luminance information and the reference chrominance information need to be subjected to a first sub-filtering process (i.e., reducing the size); if L is less than the preset number, then the reference luminance information and the reference chrominance information need to be subjected to a second sub-filtering process (i.e., enlarging the size).
[0144] In another possible implementation, filtering part of the reference information of the current block to determine the target information of the current block may include:
[0145] Performing a third filtering process on the values of the second color component sampling points in the adjacent area of the current block to obtain filtered sample values of the second color component in the adjacent area of the current block;
[0146] The target information of the current block is determined according to the filtered sample values of the second color component in the adjacent area of the current block.
[0147] In an embodiment of the present application, the number of filtered sample values of the second color component in the adjacent area of the current block is equal to K, where K represents the number of values of the first color component sampling points in the adjacent area of the current block, and K is a positive integer.
[0148] In a specific embodiment, the third filtering process is an upsampling filtering process. For example, the amount of reference luminance information of the current block is generally greater than the amount of reference chrominance information. Therefore, the upsampling filtering process may also be performed on the reference chrominance information of the current block to make the amount of reference luminance information equal to the amount of reference chrominance information.
[0149] Since the amount of reference luminance information and the amount of reference chrominance information of the current block will also be affected by the color format information; therefore, in some embodiments, the method may further include: based on the color format information, performing a third filtering process on the values of the second color component sampling points in the adjacent area of the current block to obtain the filtered sample values of the second color component in the adjacent area of the current block.
[0150] In a specific embodiment, the third filtering process may further include:
[0151] If the color format information indicates 4:4:4 sampling, upsampling filtering is not performed on the values of the second color component sampling points in the adjacent areas of the current block;
[0152] If the color format information indicates 4:2:2 sampling, upsampling filtering is performed on the values of the second color component sampling points in the adjacent area of the current block;
[0153] If the color format information indicates 4:2:0 sampling, up-sampling filtering is performed on the values of the second color component sampling points in the adjacent area of the current block.
[0154] It should be noted that, in the embodiment of the present application, when the color format information indicates 4:2:0 sampling, the upsampling rate is a positive integer multiple of 2. In this way, when the amount of reference luminance information obtained from the adjacent area is different from the amount of reference chrominance information, the reference chrominance information obtained from the adjacent area can be upsampled and filtered to match the amount of reference luminance information.
[0155] In this way, for the upsampling filtering of the reference chromaticity information obtained from the adjacent area, upsampling can be performed by linear interpolation or nonlinear interpolation, or a reference chromaticity pixel can be copied, or even other upsampling methods can be used, which are not specifically limited in the embodiments of the present application.
[0156] For example, as shown in FIG7B , when the color format information indicates 4:2:0 sampling, taking an adjacent area of 4×2 chroma pixels as an example, it is necessary to upsample it to an area of 8×4 pixels. In FIG7B , (a) represents the reference chroma information refC, and (b) represents the upsampled reference chroma information refC'; wherein, the bold frame in (a) represents the chroma pixels corresponding to the four upsampled chroma pixels in the bold frame in (b), k = 0, ..., 3, l = 0, 1. The four chroma pixels in the bold frame in (b) represent the upsampled chroma pixels corresponding to the bold frame in (a), and (i, j) represents the pixel position in the upper left corner.
[0157] Furthermore, when the number of reference chrominance information after upsampling and filtering is K and is the same as the number of reference luminance information, it is also necessary to match K with a preset number. Therefore, in some embodiments, the method may further include:
[0158] Compare K with the preset number;
[0159] performing a fourth filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block according to a comparison result of K with a preset number, to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block;
[0160] Target information of the current block is determined according to filtered adjacent samples of the first color component of the current block and filtered adjacent samples of the second color component of the current block.
[0161] In a specific embodiment, performing a fourth filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block based on the comparison result of K with the preset number to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block may include:
[0162] If K is greater than a preset number, performing a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block;
[0163] If K is less than a preset number, a second sub-filtering process is performed on the values of the first color component sampling points in the adjacent area of the current block and the filtered sample values of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0164] It should be noted that in the embodiment of the present application, the preset number can be represented by inSize. Specifically, if K is equal to inSize, no operation is required; if K is greater than inSize, a first sub-filtering process is performed on the reference luminance information and the reference chrominance information (i.e., the size is reduced); if K is less than inSize, a second sub-filtering process is performed on the reference luminance information and the reference chrominance information (i.e., the size is increased).
[0165] It can also be understood that in the embodiment of the present application, if the color format information is not considered, the number of reference luminance information and the number of reference chrominance information obtained from the adjacent area can be directly matched with inSize so that the two numbers are the same.
[0166] In a possible implementation, filtering part of the reference information of the current block to determine the target information of the current block may include:
[0167] Determine that the number of values of the first color component sampling points in the adjacent area of the current block is a first number;
[0168] performing a fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block according to a comparison result of the first number and the preset number, to determine filtered adjacent sample values of the first color component of the current block;
[0169] Target information of the current block is determined according to filtered adjacent samples of the first color component of the current block.
[0170] In a specific embodiment, performing the fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block may include:
[0171] If the first number is greater than the preset number, performing a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the first color component of the current block;
[0172] If the first number is smaller than the preset number, a second sub-filtering process is performed on the values of the first color component sampling points in the adjacent areas of the current block to determine filtered adjacent sample values of the first color component of the current block.
[0173] It should be noted that in an embodiment of the present application, the amount of reference brightness information obtained from adjacent areas can be matched with a preset number. The amount of reference brightness information obtained from adjacent areas (i.e., the first number) can be represented by refYnum; the preset number can be represented by inSize. Specifically, if refYnum is equal to inSize, then no operation is required; if refYnum is greater than inSize, then the reference brightness information needs to be subjected to a first sub-filtering process (i.e., reduced in size); if refYnum is less than inSize, then the reference brightness information needs to be subjected to a second sub-filtering process (i.e., enlarged in size).
[0174] In another possible implementation, filtering part of the reference information of the current block to determine the target information of the current block may further include:
[0175] Determine that the number of values of the second color component sampling points in the adjacent area of the current block is a second number;
[0176] performing a sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block according to a comparison result between the second number and the preset number, to determine filtered adjacent sample values of the second color component of the current block;
[0177] Target information of the current block is determined according to filtered adjacent samples of the second color component of the current block.
[0178] In a specific embodiment, performing the sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block may include:
[0179] If the second number is greater than the preset number, performing a first sub-filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the second color component of the current block;
[0180] If the second number is smaller than the preset number, a second sub-filtering process is performed on the values of the second color component sampling points in the adjacent areas of the current block to determine filtered adjacent sample values of the second color component of the current block.
[0181] It should also be noted that in the embodiment of the present application, the amount of reference chromaticity information obtained from the adjacent area can be matched with a preset number. The amount of reference chromaticity information obtained from the adjacent area (i.e., the second number) can be represented by refCnum; the preset number can be represented by inSize. Specifically, if refCnum is equal to inSize, then no operation is required; if refCnum is greater than inSize, then the reference chromaticity information needs to be subjected to a first sub-filtering process (i.e., reduced in size); if refCnum is less than inSize, then the reference chromaticity information needs to be subjected to a second sub-filtering process (i.e., enlarged in size).
[0182] Here, the first sub-filtering process may be a downsampling filtering process. In addition, in some embodiments, the first sub-filtering process may further include at least one of the following:
[0183] The sub-filtering process is performed by selecting fixed interval positions;
[0184] The sub-filtering process is performed by selecting continuous positions.
[0185] It should be noted that, taking reference luminance information as an example, the number of reference luminance information obtained from adjacent regions is refYnum. If a fixed-interval selection method is used for size reduction, that is, refYnum reference luminance pixels are arranged into a one-dimensional vector according to a certain positional relationship, and then fixed-interval selection is performed based on the size relationship between inSize and refYnum, the size of the reference luminance information can be reduced. As shown in Figure 8, the reference luminance information refY obtained from the "L"-shaped reference region in a row and column can be rearranged into a one-dimensional vector refY' according to the positional relationship from the lower left to the upper left, and then from the upper left to the upper right. Here, the arrangement method into a one-dimensional vector and the starting position of the fixed-interval selection can be freely set and are not limited in any way. Thus, assuming that the fixed interval used for size reduction is subDist, starting from the first pixel position of the one-dimensional vector, a pixel is selected every (subDist-1) pixel position, and ultimately, inSize amount of reference luminance information can be obtained.
[0186] It should also be noted that, still using the reference luminance information as an example, if the size is reduced using a continuous position selection method, that is, refYnum reference luminance pixels are arranged into a one-dimensional vector according to a certain positional relationship, and then inSize pixels are continuously selected from a preset position of this one-dimensional vector, the inSize amount of reference luminance information can be obtained. The preset position can be the first pixel position or the posth pixel position, and there is no limitation here.
[0187] In addition, the second sub-filtering process may be an up-sampling filtering process. In addition, in some embodiments, the second sub-filtering process may further include: performing sub-filtering process by interpolation at fixed intervals.
[0188] It should be noted that if a fixed-interval interpolation method is used to enlarge the size, that is, the refCnum reference chromaticity pixels are arranged into a one-dimensional vector according to a certain positional relationship, and then fixed-interval points are selected based on the size relationship between inSize and refCnum to enlarge the size of the reference chromaticity information. Among them, the method of arranging into a one-dimensional vector and the starting position of the fixed-interval interpolation can be freely set and are not limited here. In this way, after the reference luminance information is arranged in a manner similar to the size reduction, a one-dimensional vector is obtained; then, based on the fixed interval used for the enlargement size, starting from the first pixel position of the one-dimensional vector, the reference luminance information of inSize can be finally interpolated.
[0189] In addition, it should be noted that after performing a first filtering process on the values of the first color component sampling points in the adjacent areas of the current block, the target information of the current block is determined based on the filtered adjacent sample values of the first color component of the current block. The target information at this time may include: the filtered adjacent sample values of the first color component of the current block, the values of the second color component sampling points in the adjacent areas of the current block, and the reconstructed values of the first reference color component sampling points in the current block. Similarly, after performing a second filtering process on the values of the second color component sampling points in the adjacent areas of the current block, the target information of the current block is determined based on the filtered adjacent sample values of the second color component of the current block. The target information at this time may include: the filtered adjacent sample values of the second color component of the current block, the values of the first color component sampling points in the adjacent areas of the current block, and the reconstructed values of the first reference color component sampling points in the current block. Alternatively, after performing a first filtering process on the values of the first color component sampling points in the adjacent area of the current block, and performing a second filtering process on the values of the second color component sampling points in the adjacent area of the current block, the target information at this time may include: filtered adjacent sample values of the first color component of the current block, filtered adjacent sample values of the second color component of the current block, and reconstructed values of the first reference color component sampling points in the current block.
[0190] It is also understandable that if the chrominance component of the current block is predicted, the reference information of the current block may also include reconstructed luminance information of the current block. Therefore, in some embodiments, determining the reference information of the current block may further include: determining the reference information of the current block based on the reconstructed value of the first reference color component sampling point in the current block.
[0191] It should be noted that, in the embodiment of the present application, the first reference color component may be a brightness component; then, the reconstructed value of the first reference color component sampling point in the current block is the reconstructed brightness information of the current block.
[0192] It should also be noted that in this embodiment of the present application, since the amount of reconstructed luminance information of the current block is different from the preset size, it is necessary to filter the reconstructed luminance information of the current block to convert the amount of reconstructed luminance information obtained from the current block into the preset size for input. Here, the preset size is predSizeW×predSizeH; where predSizeW represents the width of the prediction block and predSizeH represents the height of the prediction block.
[0193] In some embodiments, filtering part of the reference information of the current block to determine the target information of the current block may include:
[0194] Performing a seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine a filtered sample value of the first reference color component sampling point in the current block;
[0195] The target information of the current block is determined according to the filtered sample value of the first reference color component sampling point in the current block.
[0196] In a specific embodiment, performing the seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block may include:
[0197] Determine a first width and a first height according to a reconstructed value of a first reference color component sampling point in the current block;
[0198] If the first width is greater than the width of the prediction block, or the first height is greater than the height of the prediction block, performing a third sub-filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine a filtered sample value of the first reference color component sampling point in the current block;
[0199] If the first width is smaller than the width of the prediction block, or the first height is smaller than the height of the prediction block, a fourth sub-filtering process is performed on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block.
[0200] It should be noted that, in the embodiment of the present application, assuming that the amount of reconstructed luminance information is recSizeW×recSizeH, then recSizeW represents the first width, and recSizeH represents the first height. Specifically, if recSizeW is equal to predSizeW, and recSizeH is equal to predSizeH, then no operation is required; if recSizeW is greater than predSizeW, or recSizeH is greater than predSizeH, then it is necessary to perform a first sub-filtering process on the reconstructed luminance information (i.e., reduce the size); if recSizeW is less than predSizeW, or recSizeH is less than predSizeH, then it is necessary to perform a second sub-filtering process on the reconstructed luminance information (i.e., enlarge the size).
[0201] It should also be noted that, in the embodiment of the present application, the third sub-filtering process may be a downsampling filtering process. In addition, in some embodiments, the third sub-filtering process may also include at least one of the following:
[0202] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction;
[0203] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction;
[0204] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction;
[0205] The reconstructed value of the first reference color component sampling point in the current block is downsampled and filtered in the vertical direction and then downsampled and filtered in the horizontal direction.
[0206] Here, first, the horizontal downsampling factor can be calculated based on recSizeW and predSizeW, and the vertical downsampling factor can be calculated based on recSizeH and predSizeH; then, the reconstructed luminance information is downsampled and filtered according to the horizontal downsampling factor and the vertical downsampling factor. Specifically, if the horizontal downsampling factor is greater than 1 and the vertical downsampling factor is equal to 1, then the reconstructed luminance information only needs to be downsampled in the horizontal direction; if the horizontal downsampling factor is equal to 1 and the vertical downsampling factor is greater than 1, then the reconstructed luminance information only needs to be downsampled in the vertical direction; if the horizontal downsampling factor is greater than 1 and the vertical downsampling factor is greater than 1, then the reconstructed luminance information needs to be downsampled in both the horizontal and vertical directions, wherein downsampling can be performed first in the horizontal direction and then in the vertical direction, or first in the vertical direction and then in the horizontal direction, and even the convolution operation in the neural network structure can be used to replace the downsampling operation here, and the embodiments of the present application do not impose any restrictions.
[0207] It should also be noted that, in the embodiment of the present application, the fourth sub-filtering process is an upsampling filtering process. In addition, in some embodiments, the fourth sub-filtering process may also include at least one of the following:
[0208] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction;
[0209] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction;
[0210] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction;
[0211] The reconstructed value of the first reference color component sampling point in the current block is subjected to up-sampling filtering in the vertical direction and then to up-sampling filtering in the horizontal direction.
[0212] Here, first, the horizontal upsampling factor is calculated according to recSizeW and predSizeW, and the vertical upsampling factor is calculated according to recSizeH and predSizeH; then, the reconstructed luminance information is upsampled and filtered according to the horizontal upsampling factor and the vertical upsampling factor. Specifically, if the horizontal upsampling factor is greater than 1 and the vertical upsampling factor is equal to 1, then the reconstructed luminance information only needs to be upsampled in the horizontal direction; if the horizontal upsampling factor is equal to 1 and the vertical upsampling factor is greater than 1, then the reconstructed luminance information only needs to be upsampled in the vertical direction; if the horizontal upsampling factor is greater than 1 and the vertical upsampling factor is greater than 1, then the reconstructed luminance information needs to be upsampled in both the horizontal and vertical directions, wherein upsampling can be performed first in the horizontal direction and then in the vertical direction, or first in the vertical direction and then in the horizontal direction, and even the upsampling results of "first in the horizontal direction and then in the vertical direction" and the upsampling results of "first in the vertical direction and then in the horizontal direction" can be weighted averaged to determine the final upsampling result, and even the convolution operation in the neural network structure can be used to replace the upsampling operation here, and the embodiments of the present application are not limited in any way.
[0213] That is, in an embodiment of the present application, the reference information of the current block may include: the value of the first color component sampling point in the adjacent area of the current block, the value of the second color component sampling point in the adjacent area of the current block, and the reconstructed value of the first reference color component sampling point in the current block; the target information may include: the filtered adjacent sample value of the first color component of the current block, the filtered adjacent sample value of the second color component of the current block, and the filtered sample value of the first reference color component sampling point in the current block. If the value of the first color component sampling point in the adjacent area of the current block does not require filtering, then the filtered adjacent sample value of the first color component of the current block is equal to the value of the first color component sampling point in the adjacent area of the current block; similarly, if the value of the second color component sampling point in the adjacent area of the current block does not require filtering, then the filtered adjacent sample value of the second color component of the current block is equal to the value of the second color component sampling point in the adjacent area of the current block; if the reconstructed value of the first reference color component sampling point in the current block does not require filtering, then the filtered sample value of the first reference color component sampling point in the current block is equal to the reconstructed value of the first reference color component sampling point in the current block.
[0214] In this way, the obtained target information may include: the filtered adjacent samples of the first color component of the current block, the filtered adjacent samples of the second color component of the current block, and the filtered samples of the first reference color component sampling point in the current block, that is, inSize amount of reference chromaticity information, inSize amount of reference luminance information, and reconstructed luminance information that meets the preset size.
[0215] S503: Determine a weighting coefficient according to the reference sample value of the first color component in the target information.
[0216] It should be noted that after determining the filtered adjacent samples of the first color component of the current block, the filtered adjacent samples of the second color component of the current block, and the filtered samples of the first reference color component sampling point in the current block, in some embodiments, the method may further include: setting the reference sample of the first color component in the target information to the absolute value of the difference between the filtered adjacent samples of the first color component of the current block and the filtered sample of the first reference color component sampling point in the current block.
[0217] In some embodiments, the method may further include: setting the reference sample value of the second color component in the target information as the filtered adjacent sample value of the second color component of the current block.
[0218] In addition, considering that the filtering of the reference information may be performed on only the reference luminance information, only the reconstructed luminance information, both the reference luminance information and the reconstructed luminance information, or even all the reference chrominance information, the reference luminance information, and the reconstructed luminance information, without any limitation, the calculation of the luminance difference may be the absolute value of the difference between the reference luminance information and the reconstructed luminance information, the absolute value of the difference between the filtered reference luminance information and the reconstructed luminance information, the absolute value of the difference between the reference luminance information and the filtered reconstructed luminance information, or even the absolute value of the difference between the filtered reference luminance information and the filtered reconstructed luminance information. Therefore, in an embodiment of the present application, for the reference sample value of the first color component in the target information, the method may further include: setting the reference sample value of the first color component in the target information to the absolute value of the difference between the value of the first color component sampling point in the adjacent area of the current block and the reconstructed value of the first reference color component sampling point in the current block; or, setting the reference sample value of the first color component in the target information to the absolute value of the difference between the filtered adjacent sample value of the first color component of the current block and the reconstructed value of the first reference color component sampling point in the current block; or, setting the reference sample value of the first color component in the target information to the absolute value of the difference between the value of the first color component sampling point in the adjacent area of the current block and the filtered sample value of the first reference color component sampling point in the current block, which is not specifically limited here.
[0219] In some embodiments, determining the weighting coefficient according to the reference sample value of the first color component in the target information may include:
[0220] Determining a value corresponding to the reference sample value of the first color component under a preset mapping relationship;
[0221] Set the weighting factor equal to the value.
[0222] It should be noted that, in this embodiment of the present application, the reference sample value of the first color component may be the absolute value of the difference between the filtered adjacent sample value of the first color component of the current block and the filtered sample value of the first reference color component sampling point in the current block. The first reference color component is the first color component, and the first color component is a color component different from the second color component to be predicted in this embodiment of the present application.
[0223] For example, assuming that the first color component is the chrominance component and the second color component is the luminance component, the embodiment of the present application mainly predicts the chrominance component of the pixel to be predicted in the current block. First, at least one pixel to be predicted in the current block is selected, and the luminance difference between its reconstructed chrominance and the reference chrominance in the adjacent area is calculated (using |ΔC k | indicates); the chroma difference of the pixels to be predicted at different positions in adjacent areas is different. The reference pixel position with the smallest chroma difference will follow the changes of the pixels to be predicted in the current block. Usually, the size of the chroma difference represents the degree of similarity between the chroma. If |ΔC k | is small, indicating that the similarity of chromaticity values is relatively strong, and the corresponding weighting coefficient (using w k ) can be given a larger weight; on the contrary, if |ΔC k | is larger, indicating that the similarity of chromaticity values is weaker, w k A smaller weight can be assigned. That is, w k and |ΔC k | is approximately inversely proportional. Thus, according to |ΔC k |You can create a preset mapping relationship as shown below,
[0224] w k =f(|ΔC k |) (1)
[0225] Here, taking formula (1) as an example, |ΔC k | represents the reference sample value of the first color component, f(|ΔC k |) represents the value corresponding to the reference sample value of the first color component under the preset mapping relationship, w k Represents the weighting coefficient, that is, w k Set to be equal to f(|ΔC k |).
[0226] It should also be noted that, in the embodiment of the present application, if the number of reference samples of the second color component is N, then the number of weighting coefficients is also N; wherein the sum of the N weighting coefficients is equal to 1, and each weighting coefficient is a value greater than or equal to 0 and less than or equal to 1, that is, 0≤w k ≤1. However, it should be noted that "the sum of N weighting coefficients equals 1" is only a theoretical concept; in actual fixed-point implementation, the absolute value of the weighting coefficient can be greater than 1.
[0227] Understandably, in probability theory and related fields, the normalized exponential function, or Softmax function, is a generalization of the logistic function. It can "compress" an N-dimensional vector z containing arbitrary real numbers into another N-dimensional vector σ(z), so that each element ranges between (0, 1) and the sum of all elements is 1. It is often used as a nonlinear activation function for multi-class neural networks. The Softmax function is shown below.
[0228]
[0229] However, the Softmax function can satisfy w k However, its function value increases with the increase of vector elements, which does not meet the condition of w k and |ΔC k | Approximately inversely proportional relationship. Therefore, in order to adapt to w k The design and meet the requirements of w k For nonlinear expectations, we can propose a Softmax function with parameter γ to implement the f function in formula (1), as shown below:
[0230]
[0231] Among them, in order to achieve w k and |ΔC k | is approximately inversely proportional, γ can be limited to a negative number. That is, when the first reference color component parameter is |ΔC k |ΔC k |with w k The mapping relationship between them can be expressed as:
[0232]
[0233] According to formula (4), in some embodiments, determining the value corresponding to the reference sample value of the first color component under the preset mapping relationship may include:
[0234] Determine the first factor;
[0235] determining a first product value according to the first factor and the reference sample value of the first color component;
[0236] Determine the value of the first product value corresponding to the preset mapping relationship.
[0237] It should be noted that, in the embodiment of the present application, the first factor is a constant value less than zero. Taking the Softmax function as an example, in formula (4), γ represents the first factor, |ΔC k | represents the first reference color component parameter, γ|ΔC k | represents the first product value, w k Represents the weighting coefficient; k = 1, 2,…, N.
[0238] In a specific embodiment, determining the first factor may include: the first factor is a preset constant value.
[0239] It should be noted that in this case, for γ, the weighted coefficient distribution of adjacent chromaticities can be adjusted according to the relatively flat characteristics of chromaticity, so as to capture the weighted coefficient distribution suitable for natural image chromaticity prediction. In order to determine the parameter γ suitable for natural image chromaticity prediction, the given γ set is traversed, and the suitability of γ is measured by the difference between the predicted chromaticity and the original chromaticity under different γ. For example, γ can take -2ε, where ε∈{1,0,-1,-2,-3}; after experiments, it was found that in this γ set, the best value of γ is -0.25. Therefore, in a specific embodiment, γ can be set to -0.25, but the embodiment of the present application is not specifically limited.
[0240] In another specific embodiment, determining the first factor may include: determining a value of the first factor according to a size parameter of the current block.
[0241] Furthermore, in some embodiments, the method may further include: determining the value of the first factor according to a preset mapping lookup table between the size parameter of the current block and the value of the first factor.
[0242] Here, the size parameter of the current block may include at least one of the following parameters: the width of the current block, the height of the current block, and the product of the width and the height of the current block.
[0243] It should be noted that the embodiment of the present application can fix the value of the first factor in a classification manner. For example, the size parameters of the current block are divided into three categories, and the value of the first factor corresponding to each category is determined. In view of this situation, the embodiment of the present application can also pre-store a lookup table mapping the size parameters of the current block and the values of the first factor, and then determine the value of the first factor based on the lookup table. For example, Table 1 shows a correspondence between a first factor and the size parameters of the current block provided by the embodiment of the present application. It should be noted that Table 1 is only an exemplary lookup table and is not a specific limitation.
[0244] Table 1
[0245] The first factor of the size parameter of the current block Min(W,H)<=4-0.125Min(W,H)>4&&Min(W,H)<=16-0.0833Min(W,H)>16-0.0625
[0246] In another specific embodiment, determining the first factor may include: determining a value of the first factor according to the number of reference pixels of the current block.
[0247] Furthermore, in some embodiments, the method may further include: determining the value of the first factor according to a preset mapping lookup table between the number of reference pixels of the current block and the value of the first factor.
[0248] It should be noted that the embodiment of the present application can divide the number of reference pixels into three categories, and still use the classification method to fix the value of the first factor. For example, the number of reference pixels of the current block can be divided into three categories, and the value of the first factor corresponding to each category is determined. In view of this situation, the embodiment of the present application can also pre-store a lookup table mapping the number of reference pixels of the current block and the value of the first factor, and then determine the value of the first factor based on the lookup table. For example, Table 2 shows a correspondence between a first factor and the number of reference pixels of the current block provided by the embodiment of the present application. It should be noted that Table 2 is only an exemplary lookup table and is not specifically limited.
[0249] Table 2
[0250] The first factor of the number of reference pixels is 0 <R<16-0.111316<=R<32-0.0773R> =32-0.0507
[0251] Furthermore, for the first product value, determining the first product value according to the first factor and the reference sample value of the first color component may include:
[0252] Setting the first product value equal to the product of the first factor and the reference sample value of the first color component; or,
[0253] The first product value is set to be equal to a value obtained by right-shifting the reference sample value of the first color component, wherein the number of bits of the right-shift is equal to the first factor; or
[0254] The first product value is set to a value obtained by performing addition and bit shift operations on the reference sample value of the first color component according to the first factor.
[0255] For example, assuming the first factor is 0.25 and the reference sample value of the first color component is denoted by Ref, the first product value can be equal to 0.25×Ref. 0.25×Ref can be expressed as Ref / 4, that is, Ref>>2. Furthermore, during the fixed-point calculation process, floating-point numbers may be converted to addition and shift operations; that is, there is no restriction on the calculation method for the first product value.
[0256] It can also be understood that in the embodiment of the present application, the first color component can also be a luminance component, and the second color component can be a chrominance component. In this case, at least one pixel to be predicted in the current block is selected, and the luminance difference between its reconstructed luminance and the reference luminance in the adjacent area is calculated (using |ΔY k | represents). Where, if |ΔY k | is small, indicating that the similarity of the brightness values is relatively strong, and the corresponding weighting coefficient (using w k ) can be given a larger weight; on the contrary, if |ΔY k | is larger, indicating that the similarity of brightness values is weaker, w k A smaller weight can be assigned, that is, when calculating the weighting coefficient, the reference sample value of the first color component can also be |ΔY k |, and then calculate the weighting coefficient.
[0257] It should be noted that when predicting the chrominance component of the pixel to be predicted in the current block, since the chrominance component value of the pixel to be predicted cannot be directly determined, the chrominance difference |ΔC between the reference pixel and the pixel to be predicted is k | cannot be obtained directly; however, there is a strong correlation between the components in the local area of the current block. At this time, the brightness difference |ΔY between the reference pixel and the pixel to be predicted can be used to calculate the value of |ΔY. k | to derive |ΔC k |; That is, according to |ΔY k | and the product of the second factor, we can get |ΔC k |; Thus, the first product value is equal to the first factor and |ΔY k |'s product.
[0258] That is, in the embodiment of the present application, for the reference sample value of the first color component, it can be |ΔCk |, the absolute value of the chromaticity difference; or |ΔY k |, that is, the absolute value of the brightness difference; or it can be |αΔY k |, that is, the product of the absolute value of the brightness difference and a preset multiplier, etc. The preset multiplier here is the second factor described in the embodiment of the present application.
[0259] Furthermore, with respect to the second factor, in a specific embodiment, the method may further include: performing a least squares calculation based on the first color component value of the reference pixel and the second color component value of the reference pixel to determine the second factor.
[0260] That is, assuming that the number of reference pixels is N, the first color component value of the reference pixel is the reference brightness information of the current block, and the second color component value of the reference pixel is the reference chrominance information of the current block, then the least squares method can be used to calculate the chrominance component values and the brightness component values of the N reference pixels to obtain the second factor. For example, the least squares regression calculation is as follows:
[0261]
[0262] Among them, L k represents the brightness component value of the kth reference pixel, C k represents the chrominance component value of the kth reference pixel, N represents the number of reference pixels, and α represents the second factor, which can be calculated using least squares regression. It should be noted that the second factor can also be a fixed value or fine-tuned based on a fixed value, etc., and this embodiment of the application is not specifically limited to this.
[0263] In addition, regarding the preset mapping relationship, in some embodiments, the preset mapping relationship can be a preset function relationship. In some embodiments, the preset mapping relationship can be a Softmax function. The Softmax function is a normalized exponential function, but in embodiments of the present application, normalization is not required, and its value is not limited to the range of [0, 1].
[0264] For example, |ΔY k Taking the reference sample value of the first color component as an example, the weighting coefficient w corresponding to the kth reference pixel is k It can be calculated by formula (4) or replaced as follows:
[0265]
[0266]
[0267] Where S represents the control parameter, and the relationship between S and the first factor (γ) is:
[0268] Exemplarily, the value of S is related to the size parameters of the current block. The size parameters of the current block include the width and height of the current block. In one possible implementation, if the minimum value of the width and height is less than or equal to 4, the value of S is equal to 8; if the minimum value of the width and height is greater than 4 and less than or equal to 16, the value of S is equal to 12; if the minimum value of the width and height is greater than 16, the value of S is equal to 16. In another possible implementation, if the minimum value of the width and height is less than or equal to 4, the value of S is equal to 7; if the minimum value of the width and height is greater than 4 and less than or equal to 16, the value of S is equal to 11; if the minimum value of the width and height is greater than 16, the value of S is equal to 15. Alternatively, the value of S is related to the number of reference pixels (R) of the current block. In one possible implementation, if R is less than 16, then the value of S is equal to 8; if R is greater than or equal to 16 and less than 32, then the value of S is equal to 12; if R is greater than or equal to 16, then the value of S is equal to 16; the embodiments of the present application do not impose any limitations on this.
[0269] It should also be noted that, in addition to the Softmax function, in other embodiments, the preset mapping relationship may be a weighting function that has an inversely proportional relationship with the reference sample value of the first color component.
[0270] For example, still with |ΔY k Taking the reference sample value of the first color component as an example, the Softmax function in equation (4) can also be replaced as follows:
[0271]
[0272]
[0273] Wherein, k=1, 2, ... N, offset=1 or 2 or 0.5 or 0.25.
[0274] In this way, when the preset mapping relationship is a preset functional relationship, it can be as shown in formula (4), or as shown in formula (6) or formula (7), or as shown in formula (8) or formula (9), or even other functional models that can fit the trend that the closer the reference luminance value of the reference pixel is to the luminance reconstruction value of the pixel to be predicted in the current block, the higher the importance of the reference chrominance value of the reference pixel to the pixel to be predicted in the current block, and the weighted coefficient is constructed, etc., and the embodiments of the present application do not make specific limitations.
[0275] In addition, in some embodiments, the preset mapping relationship may be a preset lookup table. That is, embodiments of the present application may further simplify operations, for example, by using an array element lookup table to reduce some computational operations. The preset mapping relationship may include: determining an array element value based on a preset lookup table mapping a reference sample value of the first color component, a first factor, and array elements; then determining the value corresponding to the array element value under the preset mapping relationship; and then setting the weighting coefficient equal to the value.
[0276] Specifically, the f model for calculating the weighted coefficients can be implemented by simplifying operations, such as table lookup. k |, you can also use |ΔY kij | indicates that; in this way, the weighted coefficient w is calculated by the f model k ij=f(|ΔY kij |). Here, the numerator in the f model can be expressed as the independent variable |ΔY kij | and the number of S types are stored as array indices. Subsequent operations in the f model that are the same as the numerator calculation are obtained by looking up the table, thus avoiding the calculation operations of the numerator and denominator.
[0277] It should be noted that, in the embodiment of the present application, the storage here can be divided into complete storage or partial storage.
[0278] In a specific embodiment, full storage is to store |ΔY kij |The range of values and the number of S types of molecules are all stored. Under full storage, it is necessary to open (|ΔY kij |value range multiplied by the number of S types) of array space. Among them, the required storage array size is |ΔY kij |The value range size, the performance remains unchanged.
[0279] For example, taking 10-bit pixels and formula (7) as an example, the variable |ΔY kij The value range is an integer between 0 and 1023, the number of S types is 3, and the value of |ΔY kij | and the number of species S as indices into the two-dimensional array storMole to fully store the molecule:
[0280]
[0281] At this time, the calculation of the weighted coefficient of formula (7) can be simplified as follows:
[0282]
[0283] Thus, the elements of the two-dimensional array storMole[index 1][index 2] are shown in Table 3.
[0284] Table 3
[0285]
[0286] Furthermore, the complete storage can also set the corresponding array offset according to the classification of S, and then according to (|ΔY kij | plus offset) as the index of the one-dimensional array ostorMole to fully store the molecule, as shown below:
[0287]
[0288] At this time, the calculation of the weighted coefficient of formula (7) can be simplified as follows:
[0289]
[0290] Thus, the elements of the one-dimensional array ostorMole[index] are shown in Table 4.
[0291] Table 4
[0292]
[0293]
[0294] In another specific embodiment, partial storage is to select and store part of all numerator values in the calculation weight coefficients, and |ΔY kij |Part of the range and / or the molecular values of some species of S are stored. The f model value outside the selected range defaults to 0. The required storage array size is (selected |ΔY kij |The value range is multiplied by the number of selected types of S).
[0295] For example, taking 10-bit pixels and formula (7) as an example, assuming that |ΔY kij The range of the value is 0 to 99 integers, and the numerator value of three S is partially stored, then |ΔY kij The value range is an integer between 0 and 99, and the number of S types is 3. kij | and the number of species S as indices into the two-dimensional array partstorMole to store molecules:
[0296]
[0297] At this time, the calculation of the weighted coefficient of formula (7) can be simplified as follows:
[0298]
[0299] Thus, the elements of the two-dimensional array partstorMole[index 1][index 2] are shown in Table 5.
[0300] Table 5
[0301]
[0302] That is to say, the storage range of some storage can be stored according to actual needs, not limited to the |ΔY in the example kij Within 100 integers, three S types; partial storage can also set corresponding offsets for selected S types to create one-dimensional storage space, which is not limited in this embodiment of the present application. Furthermore, for Tables 3-5, for fixed-point calculations, amplified integer values can also be stored; this allows subsequent reduction operations after the predicted value is determined.
[0303] Thus, according to the reference sample value of the first color component (eg |ΔY kij |), the weighting coefficients can be determined, which can include: w1, w2, ..., w N N weighting coefficients are used, where N represents the number of reference samples of the second color component. Theoretically, the sum of these N weighting coefficients is equal to 1, and each weighting coefficient is a value greater than or equal to 0 and less than or equal to 1. However, it should be noted that "the sum of the N weighting coefficients is equal to 1" is only a theoretical concept; in actual fixed-point implementation, the absolute value of the weighting coefficient can be greater than 1.
[0304] S504: Determine a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information.
[0305] S505: Determine, according to the prediction block, a reconstructed value of the second color component sampling point of the current block.
[0306] It should be noted that after the weighting coefficient is determined, the reference sample value of the second color component can be weighted calculated using the weighting coefficient to obtain the predicted value of the second color component sampling point in the prediction block.
[0307] In some embodiments, determining the predicted block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information may include:
[0308] Determining a weighted value of a reference sample value of a second color component and a corresponding weighting coefficient;
[0309] The predicted value of the second color component sampling point in the prediction block is set to be equal to the sum of N weighted values; wherein N represents the number of reference sample values of the second color component, and N is a positive integer.
[0310] That is, if the number of reference samples of the second color component is N, then first determine the weighted value of each reference sample of the second color component and the corresponding weighting coefficient (ie, w k C k ), and then the sum of these N weighted values is used as the predicted value of the second color component sampling point in the prediction block. Specifically, its calculation formula is as follows:
[0311]
[0312] For example, the absolute luminance difference is first calculated based on the reconstructed luminance information at position (i, j) in the current block and the reference luminance information in the adjacent area. The weighting coefficient is then calculated using the Softmax function, and the predicted value of the chrominance component at position (i, j) in the prediction block can be obtained using Equation (16). In particular, it should be noted that this method facilitates parallel processing and can speed up the calculation.
[0313] It should also be noted that, in the embodiment of the present application, the first color component is the luminance component, and the second color component is the chrominance component. Since the resolution of the luminance component is different from the resolution of the chrominance component, when filtering the reference information, it includes downsampling the luminance component to align the resolution of the chrominance component, or upsampling the chrominance component to align the resolution of the luminance component.
[0314] In an embodiment of the present application, the prediction block includes the predicted values of at least some of the second color component sampling points in the current block. If the prediction block only includes the predicted values of some of the second color component sampling points in the current block or does not include the predicted values of the second color component sampling points (the predicted values need to be filtered and enhanced to obtain the final predicted values), then the prediction block can also be filtered. In this case, the prediction block can be called a first prediction block, and then the first prediction block is upsampled and filtered to determine the second prediction block of the second color component of the current block. At this time, the second prediction block is the final prediction block of the second color component of the current block, and includes the predicted values of all the second color component sampling points in the current block. Specifically, in some embodiments, the first prediction block is upsampled and filtered to determine the second prediction block of the second color component of the current block, which can include: the input of the upsampled filtering process is the first upsampled input block; the output of the upsampled filtering process is the first upsampled output block.
[0315] Furthermore, in some embodiments, the upsampling filtering process may include:
[0316] Determine an upsampling factor, wherein the upsampling factor includes at least one of the following: a horizontal upsampling factor and a vertical upsampling factor;
[0317] An upsampling filtering process is performed on the first upsampling input block according to the upsampling factor to obtain a first upsampling output block.
[0318] In one possible implementation, performing upsampling filtering on the first upsampling input block according to the upsampling factor to obtain the first upsampling output block may include: if the horizontal upsampling factor is greater than 1, or the vertical upsampling factor is greater than 1, performing upsampling filtering on the first upsampling input block to obtain the first upsampling output block.
[0319] In an embodiment of the present application, performing upsampling filtering on the first upsampling input block includes at least one of the following:
[0320] Performing upsampling filtering on the first upsampled input block in the horizontal direction;
[0321] Performing upsampling filtering on the first upsampled input block in a vertical direction;
[0322] Performing upsampling filtering processing on the first upsampled input block in the horizontal direction and then performing upsampling filtering processing in the vertical direction;
[0323] The first up-sampled input block is subjected to up-sampling filtering in the vertical direction and then to up-sampling filtering in the horizontal direction.
[0324] Here, first, a horizontal upsampling factor can be calculated based on the width of the first upsampling input block and the width of the first upsampling output block, and a vertical upsampling factor can be calculated based on the height of the first upsampling input block and the height of the first upsampling output block; then, the first upsampling input block is upsampled and filtered according to the horizontal upsampling factor and the vertical upsampling factor. Specifically, if the horizontal upsampling factor is greater than 1 and the vertical upsampling factor is equal to 1, then only the first upsampling input block needs to be upsampled in the horizontal direction; if the horizontal upsampling factor is equal to 1 and the vertical upsampling factor is greater than 1, then only the first upsampling input block needs to be upsampled in the vertical direction; if the horizontal upsampling factor is greater than 1 and the vertical upsampling factor is greater than 1, then the first upsampling input block needs to be upsampled in both the horizontal and vertical directions, wherein upsampling can be performed first in the horizontal direction and then in the vertical direction, or first in the vertical direction and then in the horizontal direction, and even the convolution operation in the neural network structure can be used to replace the upsampling filtering operation here, and the embodiments of the present application do not impose any restrictions.
[0325] In another possible implementation, perform upsampling filtering on the first prediction block to determine the second prediction block. At this time, the method may further include: using the first prediction block as the first upsampling input block; using the first upsampling output block as the second prediction block of the second color component of the current block.
[0326] In another possible implementation, first perform enhancement filtering on the first prediction block, and then perform upsampling filtering to determine the second prediction block. At this time, the method may further include: performing filtering enhancement processing on the first prediction block to determine the first enhanced prediction block; using the first enhanced prediction block as the first upsampling input block; using the first upsampling output block as the second prediction block of the second color component of the current block.
[0327] In yet another possible implementation, first perform upsampling filtering on the first prediction block, and then perform enhancement filtering to determine the second prediction block. At this time, the method may further include: using the first prediction block as the first upsampling input block; using the first upsampling output block as the first upsampling filtering prediction block; performing filtering enhancement processing on the first upsampling filtering prediction block to determine the second prediction block of the second color component of the current block.
[0328] In still another possible implementation, first perform enhancement filtering on the first prediction block, then perform upsampling filtering, and then perform enhancement filtering again to determine the second prediction block. At this time, the method may further include: performing first filtering enhancement processing on the first prediction block to determine the second enhanced prediction block; using the second enhanced prediction block as the first upsampling input block; using the first upsampling output block as the second upsampling filtering prediction block; performing second filtering enhancement processing on the second upsampling filtering prediction block to determine the second prediction block of the second color component of the current block.
[0329] Further, for the predicted value of the sampling points of the second color component in the prediction block, it is usually also necessary to limit it within a preset range. Therefore, in some embodiments, the method may further include: performing a correction operation on the predicted value of the sampling points of the second color component in the prediction block to determine the prediction block of the second color component of the current block.
[0330] [[ID=IS15]]It should be noted that in the embodiments of the present application, the preset range may be: between 0 and (1 << BitDepth) - 1; where BitDepth is the bit depth required for the chrominance component. If the predicted value exceeds the value range of this preset range, then a corresponding correction operation needs to be performed on the predicted value. Exemplarily, a clamping operation may be performed, specifically as follows:
[0331] When the value is less than 0, set it to 0;
[0332] When When its value is greater than or equal to 0 and less than or equal to (1 << BitDepth) - 1, it is equal to
[0333] When its value is greater than (1 << BitDepth) - 1, it is set to (1 << BitDepth) - 1.
[0334] In this way, after the prediction value is corrected, it can be ensured that the prediction values of all the second color component sampling points in the prediction block are between 0 and (1 << BitDepth) - 1.
[0335] Further, after the prediction block is determined, the prediction block needs to be post - processed under certain conditions as the final prediction block. Therefore, in some embodiments, the method may further include:
[0336] Perform related processing on the prediction block, and use the processed prediction block as the prediction block of the second color component of the current block.
[0337] In a possible implementation manner, performing related processing on the prediction block may include: performing filtering and enhancement processing on the prediction block, and using the processed prediction block as the prediction block of the second color component of the current block.
[0338] It should be noted that in the WCP mode, in order to reduce the instability caused by pixel - by - pixel independent and parallel prediction, for example, the prediction block can be smoothed and filtered, and then the processed prediction block can be used as the final prediction block.
[0339] In another possible implementation manner, performing related processing on the prediction block may include: determining the compensation value of the second color component of the prediction block according to the reference samples in the adjacent region of the current block; correcting the prediction value of the second color component sampling point in the prediction block according to the compensation value, and determining the prediction block of the second color component of the current block.
[0340] It should be noted that in order to further improve the prediction accuracy in the WCP mode, a position - related correction process can be performed on the prediction block. For example, chromaticity compensation values are calculated for each second color component sampling point to be predicted using reference pixels with close spatial positions, and the second color component sampling points in the prediction block are corrected with these chromaticity compensation values. The final prediction value (predSamples) of the second color component sampling point is determined according to the corrected prediction value, and then the final prediction block is obtained.
[0341] In another possible implementation, performing related processing on the prediction block may include: performing prediction processing on the second color component sampling point in the prediction block according to at least one prediction mode to determine at least one initial prediction value of the second color component sampling point in the prediction block; and performing weighted fusion processing based on the at least one initial prediction value and the prediction value of the second color component sampling point in the prediction block to determine the prediction block of the second color component of the current block.
[0342] It should be noted that, in order to further improve the prediction accuracy in the WCP mode, the chroma prediction values calculated in other prediction modes can be weighted and fused with the chroma prediction values calculated in the WCP mode, and the final chroma prediction block can be determined based on this fusion result. For example, as shown in Figure 9, other prediction modes may include: planar mode, direct current (DC) mode, vertical mode, horizontal mode, and CCLM mode, and each prediction mode is connected to a switch that controls whether the chroma prediction values in this prediction mode participate in the weighted fusion process. Assume that the weight value corresponding to the Planar mode is W_Planar, the weight value corresponding to the DC mode is W_DC, the weight value corresponding to the vertical mode is W_Ver, the weight value corresponding to the horizontal mode is W_Hor, the weight value corresponding to the CCLM mode is W_CCLM, and the weight value corresponding to the WCP mode is W_Wcp; for the chroma prediction value in the Planar mode, the chroma prediction value in the DC mode, the chroma prediction value in the vertical mode, the chroma prediction value in the horizontal mode, and the chroma prediction value in the CCLM mode, if only the switch connected to the CCLM mode is in the closed state, then the chroma prediction value in the CCLM mode and the chroma prediction value in the WCP mode can be weightedly fused according to W_CCLM and W_Wcp, and the values of W_CCLM and W_Wcp can determine whether to perform equal-weighted or unequal-weighted weighted fusion, and the weighted result is the final chroma prediction value of the second color component sampling point, and then the final prediction block is obtained.
[0343] In some embodiments, after determining the prediction block, determining the reconstructed value of the second color component sampling point of the current block according to the prediction block may include:
[0344] Determine a predicted difference value of a second color component sampling point of the current block;
[0345] Determine, according to the prediction block, a predicted value of a second color component sampling point of the current block;
[0346] Determine a reconstructed value of the second color component sampling point of the current block according to the predicted difference value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.
[0347] It should be noted that, in the embodiment of the present application, the predicted residual value of the second color component sampling point of the current block may be determined by parsing the bitstream.
[0348] It should also be noted that, in an embodiment of the present application, the prediction block may include predicted values of at least some of the second color component sampling points in the current block. If the prediction block includes predicted values of all of the second color component sampling points in the current block, the predicted values of the second color component sampling points in the current block may be set equal to the values of the prediction block. If the prediction block includes predicted values of some of the second color component sampling points in the current block, upsampling filtering may be performed on the values of the prediction block, and the predicted values of the second color component sampling points in the current block may be set equal to the output values after the upsampling filtering.
[0349] That is to say, in an embodiment of the present application, the predicted value of the second color component sampling point of the current block is determined based on the prediction block, which may be to set the predicted value of the second color component sampling point of the current block equal to the value of the prediction block; or, it may be to filter the value of the prediction block and then set the predicted value of the second color component sampling point of the current block equal to the output value of the filtering.
[0350] In this way, taking the chroma component as an example, by parsing the code stream, the chroma prediction difference value of the current block can be determined; then, based on the prediction block, the chroma prediction value of the current block can be determined; then, the chroma prediction value and the chroma prediction difference value are added together to obtain the chroma reconstruction value of the current block.
[0351] It can be understood that the embodiments of the present application optimize the process of obtaining input information in the prediction process of the WCP mode, which mainly includes three aspects: on the one hand, fully considering the characteristics of the input color format information to improve the accuracy of the reference luminance information and the reference chrominance information; on the other hand, making full use of spatial correlation to perform filtering processing such as upsampling, downsampling, scaling up or downsizing on the reference luminance information, reference chrominance information, and the reconstructed luminance information of the current block; on the other hand, fully considering the relevant information of the prediction block, different filtering processing methods are used for the reference luminance information, reference chrominance information, and the reconstructed luminance information of the current block to improve the accuracy of the WCP prediction technology.
[0352] In short, the embodiment of the present application also provides a framework for weighted chrominance prediction, as shown in FIG10 . For each sampling point recY[i][j] in the reconstructed luminance information (i.e., the downsampled luminance block) in the current block, the luminance difference vector diffY[i][j] is first obtained based on the absolute value of the difference between recY[i][j] and the adjacent luminance vector refY[k]. Secondly, the normalized weight vector cWeight[i][j] is derived based on the nonlinear mapping model related to diffY[i][j]. Thirdly, the weight vector is used to perform vector multiplication on the weight vector and the adjacent chrominance vector of the current block to obtain the predicted chrominance prediction value.
[0353] The embodiment of the present application also provides a decoding method, which determines the reference information of the current block; filters part of the information in the reference information of the current block to determine the target information of the current block; determines the weighting coefficient based on the reference sample value of the first color component in the target information; determines the prediction block of the second color component of the current block based on the weighting coefficient and the reference sample value of the second color component in the target information; and determines the reconstruction value of the second color component sampling point of the current block based on the prediction block. In this way, for the color component information in the adjacent areas of the current block and the color component information in the current block, the accuracy of these reference information can be improved because the characteristics of the color format information are fully considered; and by performing filtering processing such as upsampling and downsampling on these reference information, the continuity of its spatial correlation can be enhanced to a certain extent or some information with weak spatial correlation can be removed. The target information obtained after filtering can be used to better perform chroma prediction, thereby improving the accuracy of chroma prediction, saving bit rate, and improving encoding and decoding performance.
[0354] In another embodiment of the present application, based on the decoding method described in the aforementioned embodiment, taking the chroma prediction of the current block as an example, in the embodiment of the present application, the reconstructed luminance information of the current block, the reference luminance information of the adjacent area, and the reference chroma information are all decoded reference information. Therefore, the embodiment of the present application proposes a weighted chroma prediction technology that utilizes the above information, and in the process of determining the target information of the current block based on this information, the information of the adjacent areas of the current block, the information of the current block, and the current color format information can be analyzed and processed. Among them, the information of the adjacent areas mainly refers to the position of the sampling points in the adjacent areas, the reference luminance information, and the reference chroma information, the information of the current block mainly refers to the reconstructed luminance information within the current block, and the current color format information mainly refers to the InputChromaFormat parameter in the configuration file (*.cfg format) during the decoding process, that is, the ratio of the luminance to chroma samples of the input video.
[0355] It can be understood that in the embodiments of the present application, the following are mainly proposed: (1) whether to downsample the reference luminance information of the adjacent area or whether to upsample the reference chrominance information of the adjacent area according to the current color format information; (2) the reference luminance information of the adjacent area, the reference chrominance information and the reconstructed luminance information in the current block can be matched with the preset size of the core parameter through different enlargement / reduction operations.
[0356] Here, the detailed steps of the chromaticity prediction process of the WCP mode are as follows:
[0357] Input of WCP mode: the position of the current block (xTbCmp, yTbCmp), the width of the current block nTbW and the height of the current block nTbH.
[0358] Output of the WCP mode: predicted value predSamples[x][y] of the current block, where the upper left corner of the current block is the coordinate origin, x=0,...,nTbW-1, y=0,...,nTbH-1.
[0359] The prediction process in the WCP mode includes steps such as determining core parameters, obtaining target information, weighted chrominance prediction, and post-processing. Furthermore, a pre-processing step is added to the target information acquisition process. After these steps, the chrominance prediction value for the current block can be obtained.
[0360] In a specific embodiment, referring to FIG11 , which shows a second flow chart of a decoding method provided by an embodiment of the present application, the method may include:
[0361] S1101: Determine the core parameters of the WCP mode.
[0362] It should be noted that, for S1101, the core parameters involved in the WCP mode are determined, that is, the core parameters of the WCP mode can be obtained or inferred through configuration or in some way, such as obtaining the core parameters from the bitstream at the decoding end.
[0363] Here, the core parameters of the WCP mode include but are not limited to a control parameter (S), the number of weighted chroma prediction inputs (inSize), and the number of weighted chroma prediction outputs (arranged as predSizeW×predSizeH). The prediction block of the weighted chroma prediction output can be represented by predWcp, where the number of weighted chroma prediction outputs can be set to the same value (e.g., predSizeW=predSizeH=T / 4) or related to the size parameter of the current block (e.g., predSizeW=nTbW, predSizeH=nTbH). The control parameter (S) can be used to adjust the nonlinear function in subsequent steps or to adjust the data involved in subsequent steps.
[0364] The determination of core parameters is affected by block size, block content, or the number of pixels in a block under certain conditions. For example:
[0365] If the WCP mode is applied to a wide variety of block sizes, or the differences between block sizes are large, or the block contents are very different, or the number of pixels within a block is large, the current block can be classified according to its block size, block content, or number of pixels within the block, and the same or different core parameters can be determined for different categories. That is, the control parameter (S) corresponding to different categories, or the number of weighted chroma prediction inputs inSize, or the number of weighted chroma prediction outputs (arranged as predSizeW×predSizeH), can be the same or different. Note that predSizeW and predSizeH can also be the same or different.
[0366] To better illustrate the determination of core parameters, two simple classifications are used as examples:
[0367] Classification Example 1: WCP mode can classify the current block according to its width and height, using wcpSizeId to indicate the block type. For different block types, the control parameter (S), the number of weighted chroma prediction inputs inSize, and the number of weighted chroma prediction outputs (arranged as predSizeW × predSizeH) can be the same or different. Here is an example of three categories:
[0368] The current block is divided into three categories based on its width and height. The control parameter (S) of different categories can be set to different values. The inSize and the number of weighted chroma prediction outputs (arranged as predSizeW × predSizeH) of different categories can be set to the same. nTbW is the width of the current block, nTbH is the height of the current block, and the block type wcpSizeId is defined as follows:
[0369] wcpSizeId = 0: represents the current block where min(nTbW, nTbH) <= 4. Among them, the control parameter (S) is 8, inSize is (2×nTbH + 2×nTbW), and the weighted chrominance prediction outputs nTbH×nTbW chrominance prediction values;
[0370] wcpSizeId = 1: represents the current block where 4 < min(nTbW, nTbH) <= 16. Among them, the control parameter (S) is 12, inSize is (2×nTbH + 2×nTbW), and the weighted chrominance prediction outputs nTbH×nTbW chrominance prediction values;
[0371] wcpSizeId = 2: represents the current block where min(nTbW, nTbH) > 16. Among them, the control parameter (S) is 16, inSize is (2×nTbH + 2×nTbW), and the weighted chrominance prediction outputs nTbH×nTbW chrominance prediction values;
[0372] The quantitative relationships of the above core parameters are shown in Table 6 in tabular form.
[0373] Table 6
[0374] wcpSizeId S inSize predSizeH predSizeW 0 8 2×nTbH + 2×nTbW nTbH nTbW 1 12 2×nTbH + 2×nTbW nTbH nTbW 2 16 2×nTbH + 2×nTbW nTbH nTbW
[0375] Another example can be used to illustrate the classification into 3 categories:
[0376] The current block is divided into 3 categories according to the width and height of the current block. The control parameter (S) of different categories can be set differently, and the inSize and the number of outputs of the weighted chrominance prediction (arranged as predSizeW×predSizeH) of different categories can be set the same. nTbW is the width of the current block, nTbH is the height of the current block, and the definition of the block type wcpSizeId is as follows:
[0377] wcpSizeId = 0: represents the current block where min(nTbW, nTbH) <= 4. Among them, the control parameter (S) is 8, inSize is (2×nTbH + 2×nTbW), and the weighted chrominance prediction outputs nTbH×nTbW chrominance prediction values; [[ID=wcpSizeId = 1: Represents the current block where 4 < min(nTbW, nTbH) <= 16. Among them, the control parameter (S) is 12, inSize is (1.5×nTbH + 1.5×nTbW), and the weighted chrominance prediction outputs nTbH / 2×nTbW / 2 chrominance prediction values;
[0379] wcpSizeId = 2: Represents the current block where min(nTbW, nTbH) > 16. Among them, the WCP control parameter (S) is 16, inSize is (nTbH + nTbW), and the weighted chrominance prediction outputs nTbH / 4×nTbW / 4 chrominance prediction values;
[0380] The quantitative relationships of the above core parameters are shown in Table 7 in tabular form.
[0381] Table 7
[0382] wcpSizeId Sin Size predSizeH predSizeW 0 8 2×nTbH + 2×nTbW nTbH nTbW 1 12 1.5×nTbH + 1.5×nTbW nTbH / 2 nTbW / 2 2 16 nTbH + nTbW nTbH / 4 nTbW / 4
[0383] Classification Example 2: The WCP mode can also classify the current block according to the width and height of the current block, and wcpSizeId is used to represent the type of the block. For blocks of different types, the control parameter (S), the number of inputs of the weighted chrominance prediction inSize, and the number of outputs of the weighted chrominance prediction (arranged as predSizeW×predSizeH) can be the same or different. Here, an example of dividing into 3 categories is used for illustration: [[ID=~17]]
[0384] The current block is divided into 3 categories according to the width and height of the current block. The control parameter (S) of different categories can be set differently, and the inSize and the number of outputs of the weighted chrominance prediction (arranged as predSizeW×predSizeH) of different categories can be set to be the same. nTbW is the width of the current block, nTbH is the height of the current block, and nTbW×nTbH represents the number of pixels of the current block. The definition of the block type wcpSizeId is as follows:
[0385] wcpSizeId = 0: Represents the current block where (nTbW×nTbH) < 128. Among them, the control parameter (S) is 10, inSize is (2×nTbH + 2×nTbW), and the weighted chrominance prediction outputs nTbH×nTbW chrominance prediction values;
[0386] wcpSizeId=1: indicates a current block with size 128<=(nTbW×nTbH)<=256. The control parameter (S) is 8, inSize is (2×nTbH+2×nTbW), and weighted chroma prediction outputs nTbH×nTbW chroma prediction values.
[0387] wcpSizeId=2: indicates that (nTbW×nTbH)>256 current blocks. In this case, the control parameter (S) is 1, inSize is (2×nTbH+2×nTbW), and the weighted chroma prediction outputs nTbH×nTbW chroma prediction values.
[0388] The quantitative relationship of the above core parameters is expressed in tabular form, as shown in Table 8.
[0389] Table 8
[0390] wcpSizeIdSinSizepredSizeHpredSizeW0102×nTbH+2×nTbWnTbHnTbW182×nTbH+2×nTbWnTbHnTbW212×nTbH+2×nTbWnTbHnTbW
[0391] The three categories can also be explained with another example:
[0392] The current block is divided into three categories based on its width and height. The control parameter (S) of different categories can be set to different values. The inSize and the number of weighted chroma prediction outputs (arranged as predSizeW×predSizeH) of different categories can be set to the same. nTbW is the width of the current block, nTbH is the height of the current block, and nTbW×nTbH represents the number of pixels in the current block. The block type wcpSizeId is defined as follows:
[0393] wcpSizeId=0: indicates a current block with (nTbW×nTbH)<64. Here, the control parameter (S) is 16, inSize is (2×nTbH+2×nTbW), and the weighted chroma prediction outputs nTbH×nTbW chroma prediction values.
[0394] wcpSizeId=1: indicates a current block of 64<=(nTbW×nTbH)<=512. Here, the control parameter (S) is 4, inSize is (1.5×nTbH+1.5×nTbW), and the weighted chroma prediction outputs nTbH / 2×nTbW / 2 chroma prediction values.
[0395] wcpSizeId=2: indicates a current block with (nTbW×nTbH)>512. Here, the control parameter (S) is 1, inSize is (nTbH+nTbW), and the weighted chroma prediction outputs nTbH / 4×nTbW / 4 chroma prediction values.
[0396] The quantitative relationship of the above core parameters is expressed in tabular form, as shown in Table 9.
[0397] Table 9
[0398] wcpSizeIdSinSizepredSizeHpredSizeW0162×nTbH+2×nTbWnTbHnTbW141.5×nTbH+1.5×nTbWnTbH / 2nTbW / 221nTbH+nTbWnTbH / 4nTbW / 4
[0399] S1102: Determine target information after pre-processing operations based on core parameters.
[0400] It should be noted that, for S1102, the target information may include reference chrominance information (refC), reference luminance information (refY), and reconstructed luminance information (recY). In acquiring the target information (i.e., input information), when predicting the current block, the upper area, the upper left area, and the left area of the current block are used as adjacent areas of the current block (also referred to as "reference areas"). As shown in FIG6 above, the pixels in the adjacent areas are all decoded reference pixels.
[0401] It should also be noted that reference chromaticity information refC and reference luminance information refY are obtained from adjacent regions. The reference chromaticity information obtained includes, but is not limited to, selecting a reference chromaticity reconstruction value for the region above the current block and / or a reference chromaticity reconstruction value for the region to the left. The reference luminance information obtained includes, but is not limited to, obtaining corresponding reference luminance information based on the position of the reference chromaticity information.
[0402] The reconstructed luminance information recY of the current block is obtained, and the obtaining method includes but is not limited to: obtaining corresponding reconstructed luminance information according to the position of the chrominance information in the current block as the reconstructed luminance information of the current block.
[0403] It can be understood that when the quantity of the reference chromaticity information refC and the reference luminance information refY obtained from the adjacent areas mentioned above is partially different from or completely different from the inSize specified in the core parameter determination step, or the quantity of the reconstructed luminance information recY obtained in the current block is different from the preset size (predSizeW×predSizeH) specified in the core parameter determination step, then the three obtained information need to be processed in a related manner. This step occurs before the prediction process, so it is named the pre-processing operation process.
[0404] Here, the embodiment of the present application mainly focuses on the case where the inSize specified in the step of determining the core parameters is different from the number of reference chrominance information refC and the number of reference luminance information refY obtained from the adjacent area, or the case where the number of reconstructed luminance information recY obtained in the current block is different from the preset size (predSizeW×predSizeH) specified in the step of determining the core parameters. These two cases will be described in detail below:
[0405] Case 1: If the inSize specified in the core parameter determination step is different from the number of reference chromaticity information refC and the number of reference luminance information refY obtained from the adjacent area, the number of pixels of the reference chromaticity information refC and the reference luminance information refY obtained from the adjacent area needs to be converted into the inSize specified in the core parameter determination step for input. The current color format information also determines the number of reference chromaticity information and reference luminance information. The specific process of the pre-processing operation in the first case is as follows:
[0406] (1) If the current color format information is taken into account, the reference luminance information and the reference chrominance information obtained from the adjacent areas are usually unified and then matched with the inSize in the core parameter. The processing process is as follows:
[0407] a. The number of reference luminance information and reference chrominance information obtained from adjacent areas is unified as refNum.
[0408] ●If the current color format information indicates 4:4:4 sampling (i.e., YUV444), and the ratio of luminance to chrominance samples is 4:4:4, the number of reference luminance information refY obtained from the adjacent area is the same as the number of reference chrominance information refC, and no operation is required.
[0409] ●If the current color format information indicates 4:2:2 sampling (i.e., YUV422), and the ratio of luminance to chrominance samples is 4:2:2, or if the current color format information indicates 4:2:0 sampling (i.e., YUV420), and the ratio of luminance to chrominance samples is 4:1:1, the reference luminance information refY obtained from the adjacent area needs to be downsampled to match the number of reference chrominance information refC; or the reference chrominance information refC obtained from the adjacent area needs to be upsampled to match the number of reference luminance information refY.
[0410] ●The current color format information includes but is not limited to the color format information proposed above. The processing principle is: when the ratio of brightness and chrominance samples is different, the reference brightness information needs to be downsampled or the reference chrominance information needs to be upsampled.
[0411] b. Match the number of reference information refNum with inSize in the core parameter.
[0412] ●If inSize is equal to refNum, no action is required.
[0413] ●If inSize is less than refNum, the reference luminance information and reference chrominance information need to be reduced in size.
[0414] ●If inSize is greater than refNum, the reference luminance information and reference chrominance information need to be enlarged.
[0415] It should be noted that the downsampling process mentioned in the processing of the first case is specifically as follows: for the reference chromaticity information refC of the number refNum obtained from the adjacent area, the downsampling of the reference luminance information refY can be downsampling using a multi-tap filter, or obtaining a pixel of multiple reference luminance pixels at the corresponding position of the reference chromaticity pixel, as well as other downsampling methods, which are not limited in any way in the embodiments of the present application.
[0416] For example, when the color format information indicates 4:2:0 sampling, taking an adjacent area of 8×4 luminance pixels as an example, it needs to be downsampled to an area of 4×2 pixels, as shown in FIG7A above. In FIG7A, (a) represents the reference luminance information refY, and (b) represents the downsampled reference luminance information refY'. The bold frame in (a) represents the four luminance pixels corresponding to the same chrominance pixel, and (k, l) represents the corresponding upper left corner pixel position. The bold frame in (b) represents a luminance pixel after downsampling, and the same chrominance pixel corresponding to the four luminance pixels in the bold frame in (a) is a luminance and chrominance pixel pair at the same position, and (i, j) represents the position of the downsampled luminance pixel in the downsampled reference luminance information refY'; wherein i = 0,…,3, j = 0,1.
[0417] If a 4-tap filter is used for downsampling, for example, the filter coefficients are [1,1,1,1], then the calculation process of the pixel points in refY' is as follows:
[0418] for i=0,…,3 and j=0,1
[0419]
[0420] If a 5-tap filter is used for downsampling, for example, the filter coefficients are [1,1,4,1,1], then the calculation process of the pixel points in refY' is as follows:
[0421] for i=0,…,3 and j=0,1
[0422]
[0423] If a 6-tap filter is used for downsampling, for example, the filter coefficients are [1, 2, 1, 1, 2, 1], then the calculation process of the pixel points in refY' is as follows:
[0424] for i=0,…,3 and j=0,1
[0425]
[0426] If downsampling is performed by obtaining a pixel of multiple reference luminance pixels corresponding to the reference chrominance pixel, such as the upper left corner pixel position, the calculation process of the pixel point in refY' is as follows:
[0427] for i=0,…,3 and j=0,1
[0428] refY′[i][j]=refY[k][l] (20)
[0429] The upsampling process mentioned in the processing of the first case is specifically as follows: for the reference luminance information refY of the number refNum obtained from the adjacent area, the upsampling of the reference chrominance information refC can be performed by linear interpolation or nonlinear interpolation, or by copying a certain reference chrominance pixel, as well as other upsampling methods, which are not limited in the embodiments of the present application.
[0430] For example, when the color format information indicates 4:2:0 sampling, taking an adjacent area of 4×2 chroma pixels as an example, it needs to be upsampled to an area of 8×4 pixels, as shown in FIG7B . In FIG7B , (a) represents the reference chroma information refC, and (b) represents the upsampled reference chroma information refC'. The bold frame in (a) represents the chroma pixels corresponding to the four upsampled chroma pixels in the bold frame in (b), k = 0,…, 3, l = 0, 1. The four chroma pixels in the bold frame in (b) represent the upsampled chroma pixels corresponding to the bold frame in (a), and (i, j) represents the pixel position in the upper left corner.
[0431] If horizontal linear interpolation is used for upsampling, the pixels in refC' need to be linearly interpolated through adjacent horizontal points, for example:
[0432] for k=0,…,3 and l=0,1
[0433] refC′[i][j]=(w0×refC[k][l]+w1×refC[k+1][l]+offset)>>shift (21)
[0434] refC′[i+1][j]=(w2×refC[k][l]+w3×refC[k+1][l]+offset)>>shift (22)
[0435] refC′[i][j+1]=(w4×refC[k][l]+w5×refC[k+1][l]+offset)>>shift (23)
[0436] refC′[i+1][j+1]=(w6×refC[k][l]+w7×refC[k+1][l]+offset)>>shift (24)
[0437] Where, offset = 1 << (shift - 1). For {w0, w1, w2, w3, w4, w5, w6, w7} and shift, different options can be made according to the block size or block content. Here is an example of a commonly used coefficient combination:
[0438]
[0439] If vertical linear interpolation is used for upsampling, the pixels in refC' need to be linearly interpolated through the vertically adjacent points, for example:
[0440] for k=0,…,3 and l=0,1
[0441] refC′[i][j]=(w0×refC[k][l]+w1×refC[k][l+1]+offset)>>shift (25)
[0442] refC′[i+1][j]=(w2×refC[k][l]+w3×refC[k][l+1]+offset)>>shift (26)
[0443] refC′[i][j+1]=(w4×refC[k][l]+w5×refC[k][l+1]+offset)>>shift (27)
[0444] refC′[i+1][j+1]=(w6×refC[k][l]+w7×refC[k][l+1]+offset)>>shift (28)
[0445] Where, offset = 1 < < < (shift - 1). For {w0, w1, w2, w3, w4, w5, w6, w7} and shift, different options can be made according to the block size or block content. Here is an example of a commonly used coefficient combination:
[0446]
[0447] If upsampling is performed by copying a reference chroma pixel, the calculation process of the pixel points in refC' is as follows:
[0448] for k=0,…,3 and l=0,1
[0449] refC′[i][j]=refC[k][l] (29)
[0450] refC′[i+1][j]=refC[k][l] (30)
[0451] refC′[i][j+1]=refC[k][l] (31)
[0452] refC′[i+1][j+1]=refC[k][l] (32)
[0453] (2) If the current color format information is not considered, it is only necessary to match it with the inSize in the core parameter. The processing process is as follows:
[0454] a. Match the number of reference luminance information obtained from the adjacent reference area, refYnum, with inSize in the core parameter.
[0455] ●If inSize is equal to refYnum, no action is required.
[0456] ●If inSize is smaller than refYnum, the reference brightness information needs to be reduced in size.
[0457] ●If inSize is greater than refYnum, the reference brightness information needs to be enlarged.
[0458] b. Match the number of reference chromaticity information obtained from the reference area, refCnum, with the inSize of the WCP core parameter.
[0459] ●If inSize is equal to refYnum, no action is required.
[0460] ●If inSize is less than refYnum, the reference chromaticity information needs to be reduced in size.
[0461] ●If inSize is greater than refYnum, the reference chromaticity information needs to be enlarged.
[0462] The specific operation process of reducing the size mentioned in the processing of the first case is as follows: taking the reference brightness information as an example, the number of reference brightness information obtained from the adjacent areas is refYnum.
[0463] If a fixed-interval point selection method is used for size reduction, the reference brightness information can be reduced in size by arranging the refYnum reference brightness pixels into a one-dimensional vector according to a certain positional relationship. Then, fixed-interval point selection is performed based on the size relationship between inSize and refYnum. Both the one-dimensional vector arrangement method and the starting position of the fixed-interval selection can be freely set and are not limited in this embodiment.
[0464] Exemplarily, as shown in FIG8 , the reference brightness information refY obtained in an “L”-shaped row and column of reference areas can be rearranged into a one-dimensional vector refY′ in a positional relationship from the lower left to the upper left and then from the upper left to the upper right.
[0465] Here, the fixed interval used for size reduction is subDist = (refYnum + inSize-1) / inSize. Starting from the first pixel position of refY' obtained above, a pixel is selected every (subDist-1) pixel position, and finally the reference brightness information in_refY of inSize is obtained. The detailed process is as follows:
[0466] for i=0,…,inSize-1
[0467] in_refY[i]=refY′[i×subDist] (33)
[0468] If the size is reduced by selecting a continuous position, the reference brightness pixels refYnum are arranged into a one-dimensional vector according to a certain position relationship, and then inSize pixels are continuously selected from a certain position of the one-dimensional vector, that is, in_refY. For example, the selection starts from the posth position of refY'. The detailed process is as follows:
[0469] for i=0,…,inSize-1
[0470] in_refY[i]=refY′[pos+i] (34)
[0471] The specific operation process of enlarging the size mentioned in the processing of the first case is as follows: taking the reference chromaticity information as an example, the number of reference chromaticity information obtained from the adjacent areas is refCnum.
[0472] ●If a fixed-interval interpolation method is used to enlarge the size, that is, the refCnum reference chromaticity pixels are arranged into a one-dimensional vector according to a certain positional relationship, and then fixed-interval points are selected based on the size relationship between inSize and refCnum, the size of the reference chromaticity information can be enlarged. Among them, the method of arranging into a one-dimensional vector and the starting position of the fixed-interval interpolation can be freely set, and the embodiments of this application do not impose any restrictions. After refY is arranged in a manner similar to the size reduction, a one-dimensional vector refY' is obtained. Then, the fixed interval subDist = (refCnum + inSize-1) / refCnum is used to enlarge the size. Starting from the first pixel position of refY', the reference brightness information in_refY of inSize is finally obtained. If linear interpolation is used, the detailed process is as follows:
[0473] for i=0,…,inSize-1
[0474] if (i% subDist == 0)
[0475] in_refY[i]=refY′[i×subDist] (35)
[0476] else
[0477]
[0478] Among them, {w0, w1} changes with the position according to the value of the fixed interval, offset = 1 < < (shift-1), and shift can be selected differently according to the block size or block content. If subDist = 3, here is an example of a commonly used coefficient combination:
[0479]
[0480] The second case: If the amount of reconstructed luminance information recY obtained from the current block (recSizeW×recSizeH) is different from the preset size (predSizeW×predSizeH) specified in the step of determining the core parameters, it is necessary to convert the amount of reconstructed luminance information recY obtained from the current block into the preset size (predSizeW×predSizeH) specified in the step of determining the WCP core parameters for input. The specific process is as follows:
[0481] If recSizeW equals predSizeW, and recSizeH equals predSizeH, no action is required.
[0482] ●If recSizeW is larger than predSizeW, or recSizeH is larger than predSizeH, the reconstructed luminance information recY needs to be reduced in size.
[0483] ●If recSizeW is smaller than predSizeW, or recSizeH is smaller than predSizeH, the reconstructed luminance information recY needs to be scaled up.
[0484] The downsizing operation mentioned in the second case is as follows: First, the horizontal downsampling factor downHor is calculated based on the width predSizeW determined by the core parameters and the width recSizeW of the reconstructed luminance information recY; similarly, the vertical downsampling factor downVer is calculated based on the height predSizeH determined by the core parameters and the height recSizeH of the reconstructed luminance information recY. The calculation method is as follows:
[0485] downHor=recSizeW / predSizeW (37)
[0486] downVer=recSizeH / predSizeH (38)
[0487] Then downsample as follows to generate the input reconstructed brightness information in_recY:
[0488] (1) If downHor is greater than 1 and downVer is equal to 1, only horizontal downsampling is required. The specific formula is as follows:
[0489] in_recY[x][y]=(recY[downHor×x-1][y]+2×recY[downHor×x][y]+recY[downHor×x+1][y]+2)>>2 (39)
[0490] (2) If downHor is equal to 1 and downVer is greater than 1, downsampling is only required in the vertical direction. The specific formula is as follows:
[0491] in_recY[x][y]=(recY[x][downVer×y-1]+2×recY[x][downVer×y]+recY[x][downVer×y+1]+2)>>2 (40)
[0492] (3) If downHor and downVer are both greater than 1, downsampling is required in both horizontal and vertical directions. The specific formula is as follows:
[0493] in_recY[x][y]=(recY[downHor×x-1][downVer×y]+recY[downHor×x-1][downVer×y+1]+2*recY[downHor×x][downVer ×y]+2*recY[downHor×x][downVer×y+1]+recY[downHor×x+1][downVer×y]+recY[downHor×x+1][downVer×y+1]+4)>>3 (41)
[0494] The specific operation process of the enlargement size mentioned in the processing of the second case is as follows: First, the horizontal upsampling factor upHor is calculated based on the width predSizeW determined by the core parameters and the width recSizeW of the reconstructed brightness information recY; similarly, the vertical upsampling factor upVer is calculated based on the height predSizeH determined by the core parameters and the height recSizeH of the reconstructed brightness information recY. The calculation method is as follows:
[0495] upHor=predSizeW / recSizeW (42)
[0496] upVer=predSizeH / recSizeH (43)
[0497] Then up-sample as follows to generate the input reconstructed brightness information in_recY:
[0498] First, fill the upper reference reconstructed luminance pixel refY_T into the upper row in_recY[x][-1] of the reconstructed luminance information to be input, and fill the left reference reconstructed luminance pixel refY_L into the left column in_recY[-1][y] of the reconstructed luminance information to be input. Where x = 0, ..., predSizeW-1, y = 0, ..., predSizeH-1. At this time, recY is placed in the corresponding position in in_recY ((x+1)×upHor-1, (y+1)×upVer-1) using the following formula:
[0499] in_recY[(x+1)×upHor-1][(y+1)×upVer-1]=recY[x][y] (44)
[0500] It can be understood as dividing in_recY into equal parts recSizeW×recSizeH sub-blocks, and the placement position is the lower right corner of each sub-block. Referring to Figure 12, taking predSizeW=8 and predSizeH=8, and a current block size of 4×4 as an example, the diagonal fill is the top reference reconstructed luminance pixel refY_T, the vertical bar fill is the left reference reconstructed luminance pixel refY_L, and the grid fill is the location where the reconstructed luminance information recY of the current block is filled.
[0501] If the horizontal upsampling factor upHor is greater than 1, upsampling in the horizontal direction is performed first. The upsampling process is as follows:
[0502] sum=(upHor-dX)×rec_Y[xHor][yHor]+dX×rec_Y[xHor+upHor][yHor] (45)
[0503]
[0504] This process is shown in Figure 13. For example, when predSizeW = 8 and predSizeH = 8, and the current block size is 4×4, the point in the left reference luminance pixel refY_L corresponding to the horizontal position filled in rec_Y during horizontal upsampling will also be used as the reference point for upsampling, and is also marked with a grid fill. In this case, all grid fill points are reference points for upsampling, and the predicted points obtained by linear interpolation between every two grid fill points in the horizontal direction are marked with a horizontal bar fill.
[0505] The upsampling method uses linear interpolation upsampling, that is, the value of each interpolated point (bar filling) between two upsampled luminance reference points (grid filling) is the weighted average of the two upsampled reference luminance points. According to equations (45) and (46), the weight of the left upsampled reference chrominance point is (upHor-dX) / upHor, and the weight of the right upsampled reference chrominance point is dX / upHor, where dX=1,…,upHor-1 (i.e., the distance between the current interpolation point and the left reference point). In this way, when interpolating in the horizontal direction, the weight is only related to the horizontal upsampling factor upHor. Figure 14 shows an example of weights. In this example, upHor=4. Among them, for the first interpolation point in Figure 14, the weight of the left upsampled reference chroma point is 3 / 4, and the weight of the right upsampled reference chroma point is 1 / 4; for the second interpolation point in Figure 14, the weight of the left upsampled reference chroma point is 2 / 4, and the weight of the right upsampled reference chroma point is 2 / 4; for the third interpolation point in Figure 14, the weight of the left upsampled reference chroma point is 1 / 4, and the weight of the right upsampled reference chroma point is 3 / 4.
[0506] ●If the vertical upsampling factor upVer is greater than 1, vertical upsampling is required. The process is similar to horizontal upsampling. The specific process is shown in the following formula:
[0507] sum=(upVer-dY)×rec_Y[xVer][yVer]+dY×rec_Y[xVer][yVer+upVer] (47)
[0508]
[0509] This process is shown in Figure 15. For example, with predSizeW = 8 and predSizeH = 8, and a current block size of 4×4, after completing the horizontal upsampling shown in Figure 13, all existing points of the upper reference luminance pixels refY_T and in_recY will be used as reference points for vertical upsampling, and are also marked with grid fills. At this point, the grid fill points are all upsampling reference points, and the prediction points between every two grid fill points in the vertical direction are obtained by linear interpolation and marked with horizontal bars.
[0510] The upsampling method also uses linear interpolation. According to equations (47) and (48), the weight of the upper upsampling reference point is (upVer-dY) / upVer, and the weight of the lower reference point is dY / upVer, where dY = 1, ..., upVer-1 (i.e., the distance between the current interpolation point and the upper reference point). In this way, during vertical interpolation, the weight is only related to the vertical upsampling factor upVer.
[0511] In this way, the final in_recY can be obtained according to the above process. For the above-mentioned enlargement process, in addition to adopting the "horizontal first then vertical" upsampling method, the "vertical first then horizontal" upsampling method can also be adopted. The reconstructed brightness value after the "horizontal first then vertical" upsampling and the reconstructed brightness value after the "vertical first then horizontal" upsampling can also be averaged to obtain the final input reconstructed brightness value in_recY. The convolution operation in the neural network can also be used instead of the upsampling operation. The embodiments of the present application do not impose any restrictions on this.
[0512] In this way, the target information finally obtained as input may include: inSize amount of reference chrominance information refC, inSize amount of reference luminance information refY, and reconstructed luminance information recY of the current prediction block.
[0513] S1103: Perform weighted chrominance prediction calculation according to the target information to determine a prediction block for the current block.
[0514] It should be noted that, for S1103, the chromaticity prediction value within the preset size specified by the core parameter is expressed as C pred [i][j] represents the number of pixels to be predicted, where i = 0, ..., predSizeW - 1, and j = 0, ..., predSizeH - 1. These are obtained one by one. Note that predSizeH and predSizeW are core parameters and can be the same as or different from the height nTbH or width nTbW of the current block. Under certain conditions, the following calculations can be performed on only a portion of the pixels to be predicted within the current block.
[0515] The chroma prediction calculation in the WCP mode can include the following operations: obtaining weights and performing weighted prediction based on the weights to obtain weighted chroma prediction values. The weight acquisition process includes constructing a luminance difference vector and calculating the weights. The detailed calculation process is as follows:
[0516] For i=0,…,predSizeW-1, j=0,…,predSizeH-1
[0517] For k = 0, 1..., inSize-1
[0518] Construct each element diffY[i][j][k] in the brightness difference vector
[0519] Calculate each element cWeight[i][j][k] (or cWeightFloat[i][j][k]) in the weight vector, and then calculate the chrominance prediction value C by cWeight[i][j] (or cWeightFloat[i][j][k]) and refC pred [i][j].
[0520] In a specific embodiment, referring to FIG16 , which shows a flowchart diagram 3 of a decoding method provided by an embodiment of the present application. As shown in FIG16 , the method may include:
[0521] S1601: For each pixel to be predicted, a luminance difference vector is constructed using the reference chrominance information, the reference luminance information included in the target information, and the reconstructed luminance information of the current block.
[0522] It should be noted that for each pixel to be predicted C within the size specified by the core parameter pred [i][j], subtract the corresponding reconstructed brightness information recY[i][j] from the reference brightness information refY of inSize and take the absolute value to get the brightness difference vector diffY[i][j][k]. The specific calculation formula is as follows:
[0523] diffY[i][j][k]=abs(refY[k]-recY[i][j]) (49)
[0524] in, abs(x) represents the absolute value of x.
[0525] It should also be noted that under certain conditions, the brightness difference vector of the pixel to be predicted can be subjected to linear or nonlinear numerical processing. For example, the value of the brightness difference vector of the pixel to be predicted can be scaled according to the control parameter S in the core parameters.
[0526] S1602: For each pixel to be predicted, a weight vector is calculated using a nonlinear function according to the brightness difference vector.
[0527] It should be noted that, in the embodiment of the present application, a weight model is used to predict each pixel C pred The corresponding luminance difference vector diffY[i][j] is processed to obtain the corresponding floating-point weight vector cWeightFloat[i][j]. The weight model here includes but is not limited to nonlinear normalization functions, nonlinear exponential normalization functions, and other nonlinear functions, and is not limited to any other nonlinear functions.
[0528] For example, a nonlinear Softmax function can be used as a weight model, and the brightness difference vector diffY[i][j] corresponding to each pixel to be predicted is used as the input of the weight model. The weight model outputs a floating-point weight vector cWeightFloat[i][j] corresponding to each pixel to be predicted. The calculation formula is as follows:
[0529]
[0530] Under certain conditions, the weight model can also be adjusted according to the control parameter (S) in the core parameters. For example, if the size of the current block is flexible, the weight model can be adjusted according to the control parameter (S). Taking the nonlinear Softmax function as an example, different control parameters can be selected to adjust the function according to the block classification category to which the current block belongs. In this case, the weight vector calculation formula corresponding to each pixel to be predicted is as follows:
[0531]
[0532] In the embodiment of the present application, the weighting coefficients may also be processed as fixed-point. Therefore, in some embodiments, the method further includes: if the weighting coefficients are floating-point weighting coefficients, then the floating-point weighting coefficients are processed as fixed-point to obtain fixed-point weighting coefficients.
[0533] Thus, after the calculation of the above formula (50) or formula (51) is completed, cWeightFloat can be fixed-point processed as follows:
[0534] cWeight[i][j][k]=round(cWeightFloat[i][j][k]×2 Shift ) (52)
[0535] Where round(x) = Sign(x) × Floor(abs(x) + 0.5). Here, Floor(x) represents the largest integer less than or equal to x. abs(x) represents the absolute value of x. Shift is the amount of shift set in the fixed-point operation to improve accuracy.
[0536] S1603: For each pixel to be predicted, perform weighted calculation based on the weight vector and the reference chrominance information included in the target information to obtain a chrominance prediction value.
[0537] It should be noted that the chrominance prediction value of the pixel to be predicted is calculated based on the weight vector cWeight[i][j] (or cWeightFloat[i][j]) corresponding to each pixel to be predicted and the reference chrominance information refC. Specifically, each pixel to be predicted C pred The reference chroma information refC of [i][j] is multiplied one by one with the weight vector elements corresponding to each pixel to be predicted to obtain subC[i][j] (or subCFloat[i][j]), and the multiplication results are accumulated to obtain the chroma prediction value C of each pixel to be predicted. pred [i][j], that is, to achieve weighted prediction of chrominance components.
[0538] In a possible implementation, it may include: performing weighted calculation according to floating-point weighting coefficients and reference chrominance information to obtain an initial prediction value of the pixel to be predicted; and performing fixed-point processing on the initial prediction value to obtain a target prediction value of the pixel to be predicted.
[0539] For example, the calculation formula is as follows:
[0540] For k = 0, 1, ..., inSize-1
[0541] subCFloat[i][j][k]=(cWeightFloat[i][j][k]×refC[k]) (53)
[0542] After the calculation is completed, subCFloat[i][j][k] can also be fixed-point processed. In order to retain a certain calculation accuracy during the fixed-point process, a coefficient can be multiplied here, as shown below:
[0543] subC[i][j][k]=round(subCFloat[i][j][k]×2 Shift ) (54)
[0544] or:
[0545] For i=0,…,predSizeW-1, j=0,…,predSizeH-1
[0546]
[0547] After the calculation is completed, pred Float[i][j] is fixed-point, such as:
[0548] C pred [i][j]=round(C pred Float[i][j]) (56)
[0549] In another possible implementation, weighted calculation may be performed based on fixed-point weighting coefficients and reference chrominance information to obtain an initial prediction value of the pixel to be predicted; and fixed-point compensation processing may be performed on the initial prediction value to obtain a target prediction value of the pixel to be predicted.
[0550] For example, the calculation formula is as follows:
[0551] For k = 0, 1, ..., inSize-1
[0552] subC[i][j][k] = (cWeight[i][j][k] × refC[k]) (57)
[0553] Then, the fixed-point subC[i][j][k] is used for calculation, as follows:
[0554] For i = 0,..., predSizeW - 1, j = 0,..., predSizeH - 1
[0555]
[0556] where Offset = 1 << (Shift1 - 1), Shift1 is the shift amount required in the fixed-point operation to improve accuracy when calculating cWeight[i][j][k] or subC[i][j][k] (Shift1 = Shift), or it can also be the shift amount required in other steps for fixed-point operations to improve accuracy.
[0557] S1604: For each pixel to be predicted, the calculated chrominance prediction value is corrected to determine the prediction block of the current block.
[0558] It should be noted that the chrominance prediction value should be limited within a preset range. If it exceeds this preset range, corresponding correction operations need to be performed. Exemplarily, in one possible implementation, clamping operations can be performed on the chrominance prediction value of C pred [i][j], as follows:
[0559] ● When the value of C pred [i][j] is less than 0, set it to 0;
[0560] ● When the value of C pred [i][j] is greater than (1 << BitDepth) - 1, set it to (1 << BitDepth) - 1.
[0561] where BitDepth is the bit depth required for chrominance pixel values to ensure that all chrominance prediction values in the prediction block are between 0 and (1 << BitDepth) - 1.
[0562] That is:
[0563] C pred [i][j] = Clip3(0, (1 << BitDepth) - 1, C pred [i][j]) (59)
[0564] where
[0565]
[0566] Alternatively, by combining equations (59) and (60), we can obtain the following formula:
[0567]
[0568] S1104: Perform post-processing operations on the prediction block to determine a target prediction block for the current block.
[0569] It should be noted that after the chroma prediction value (predWcp) output by the weighted chroma prediction, it is also necessary to undergo post-processing operations under certain conditions as the final chroma prediction value (predSamples), otherwise the final chroma prediction value predSamples is predWcp.
[0570] For example, in order to reduce the instability caused by the independent parallel prediction of WCP pixel by pixel, predWcp can be smoothed and filtered as the final chroma prediction value predSamples. Alternatively, in order to further improve the accuracy of the WCP prediction value, predWcp can be subjected to a position-related correction process. For example: a chroma compensation value is calculated for each pixel to be predicted using a reference pixel with a close spatial position, and the predWcp is corrected using this chroma compensation value, and the corrected prediction value is used as the final chroma prediction value predSamples. Alternatively, in order to further improve the accuracy of the WCP prediction value, the chroma prediction value calculated by other chroma prediction modes can be weighted and fused with the chroma prediction value predWcp calculated by WCP, and the fusion result is used as the final chroma prediction value predSamples. For example: the chroma prediction value obtained by CCLM mode prediction and the chroma prediction value predWcp calculated by WCP can be weighted with equal weight or unequal weight, and the weighted result is used as the final chroma prediction value predSamples. Alternatively, in order to improve the WCP prediction performance, a neural network model may be used to correct the WCP prediction output predWcp, etc., and the embodiments of the present application do not impose any limitation on this.
[0571] It is also understandable that in the embodiment of the present application, if reference chrominance information redCref and reference luminance information redYref of the inSize parameter size determined by the WCP mode, and reconstructed luminance information redYrec of the current block of the (predSizeW×predSizeH) parameter size determined by the WCP mode have been obtained, since some of the above three pieces of information have been upsampled, downsampled, or reduced / enlarged in size, or have not been subjected to these operations, but spatial redundancy still exists between them, then the reference chrominance information redCref, the reference luminance information redYref, and the reconstructed luminance information redYrec can be filtered. Different filtering operations can be used according to different parameter values of inSize and predSizeW×predSizeH determined by the WCP mode to obtain three new sets of information, including reference chrominance information redCref', reference luminance information redYref', and reconstructed luminance information redYrec'. For example, redCref, redYref, and redYrec can be filtered using linear filtering, nonlinear filtering, neural network filtering, and the like using mean, median, and Gaussian filtering. In addition, these filtering operations can also occur before upsampling, downsampling or size reduction / enlargement operations to obtain three new sets of information, including reference chrominance information refC', reference luminance information refY' and reconstructed luminance information recY', and then upsampling, downsampling or size reduction / enlargement operations are performed on them.
[0572] It can also be understood that in the embodiment of the present application, the amount of reference chromaticity information refC obtained from the adjacent area is refSizeC, and the amount of reference luminance information refY is refSizeY; the size of the reconstructed luminance information recY obtained from the current block is recSizeW×recSizeH, recSizeW is the block width of the reconstructed luminance information recY, and recSizeH is the block height of the reconstructed luminance information recY. These reference chromaticity information, reference luminance information and reconstructed luminance information are all matched using a neural network with the inSize and (predSizeW×predSizeH) specified in the step of determining the core parameters. The three can use the same or different neural network models according to various situations, that is, upsampling, downsampling or size reduction / enlargement operations are all replaced by the neural network model. In addition, the filtering operations involved in the aforementioned embodiments can also be implemented before or after using the neural network model to obtain three new sets of information, including reference luminance information, reference chromaticity information and reconstructed luminance information.
[0573] It is also understandable that in the embodiment of the present application, the reference information used for a certain predicted chroma pixel in the weighted chroma prediction of the WCP mode is fixed, which will inevitably affect the prediction accuracy. Therefore, a solution is proposed here to change the reference pixel of each pixel. The detailed process is as follows:
[0574] First, the core parameters include but are not limited to the control parameter (S), the number of weighted chroma prediction inputs inSize[i][j], and the number of weighted chroma prediction outputs (predWcp) (arranged as predSizeW×predSizeH). Among them, inSize[i][j] represents a pixel to be predicted C pred [i][j] The number of reference pixels used, i=0,…,predSizeW-1, j=0,…,predSizeH-1.
[0575] Next, reference chroma information refC and reference luminance information refY are obtained from the adjacent regions. The reference chroma information obtained includes, but is not limited to, selecting a reference chroma reconstruction value for the region above the current block and a reference chroma reconstruction value for the region to the left. The reference luminance information obtained includes, but is not limited to, obtaining corresponding reference luminance information based on the position of the reference chroma information.
[0576] The reconstructed luminance information recY of the current block is obtained, and the obtaining method includes but is not limited to obtaining the corresponding reconstructed luminance information according to the position of the chrominance information in the current block as the reconstructed luminance information of the current block.
[0577] Again, for different prediction positions of the predSizeW×predSizeH size block, the reference chrominance information refC and the reference luminance information refY are obtained from the adjacent areas, and different reference luminance information refY[i][j] and reference chrominance information refC[i][j] are selected. The detailed process is as follows:
[0578] For i=0,…,predSizeW-1, j=0,…,predSizeH-1
[0579] For k = 0, 1, ..., inSize-1
[0580] Obtaining the input target information includes: obtaining reference chrominance information refC of inSize[i][j] quantity, obtaining reference luminance information refY of inSize[i][j] quantity, and obtaining reconstructed luminance information recY[i][j].
[0581] Again, for each pixel to be predicted C within the size specified by the kernel parameters pred[i][j], subtract the corresponding reconstructed brightness information recY[i][j] from the reference brightness information refY[i][j] of inSize[i][j] and take the absolute value to get the brightness difference vector diffY[i][j][k]. Among them, refY[i][j] is the pixel to be predicted The corresponding set of reference brightness vectors. The detailed calculation process is as follows:
[0582] For i=0,…,predSizeW-1, j=0,…,predSizeH-1
[0583] For k = 0, 1, ..., inSize[i][j] - 1
[0584] diffY[i][j][k]=abs(refY[i][j][k]-recY[i][j]) (62)
[0585] Under certain conditions, the brightness difference vector of the pixel to be predicted can be subjected to linear or nonlinear numerical processing. For example, the value of the brightness difference vector of the pixel to be predicted can be scaled according to the control parameter S in the core parameter.
[0586] Again, the chrominance prediction value of the pixel to be predicted is calculated based on the weight vector cWeight[i][j] corresponding to each pixel to be predicted and the reference chrominance information refC. Specifically, each pixel to be predicted C pred The reference chroma information refC[i][j] of [i][j] is multiplied one by one with the weight vector elements corresponding to each pixel to be predicted to obtain subC[i][j]. The multiplication results are accumulated to obtain the chroma prediction value C of each pixel to be predicted. pred [i][j] (i.e. weighted prediction). Where refC[i][j] is the pixel to be predicted C pred A set of reference chromaticity vectors corresponding to [i][j] are calculated as follows:
[0587] For k = 0, 1, ..., inSize-1
[0588] subCFloat[i][j][k]=(cWeightFloat[i][j][k]·refC[i][j][k]) (63)
[0589] After the calculation is completed, subCFloat can be fixed-pointed. In order to retain a certain calculation accuracy during the fixed-pointing process, a coefficient can be multiplied, as shown in the following formula:
[0590] subC[i][j][k]=round(subCFloat[i][j][k]×2 Shift ) (64)
[0591] or,
[0592] subC[i][j][k]=(cWeight[i][j][k]×refC[i][j][k]) (65)
[0593] It can also be understood that in the embodiment of the present application, the adjacent area is the same as the adjacent area used by other existing chrominance prediction modes, which will inevitably affect the prediction accuracy. Therefore, the adjacent area can also be changed. For example, the existing multi-reference row technology in H.266 can be used to expand the reference area of the WCP prediction technology. As shown in Figure 17, it can be expanded to 0, 1, 2, 3, 4, 5, 6, 7 and other multiple rows or columns of adjacent areas as the reference area of the current block; however, it is not limited to this technology, and the reference rows or reference columns can also be increased or decreased, and the embodiment of the present application does not specifically limit it.
[0594] This embodiment provides a decoding method. This embodiment details the specific implementation of the aforementioned embodiment. It can be seen that the technical solutions of the aforementioned embodiments can improve the accuracy of the WCP prediction technique. Specifically, by optimizing the target information acquisition process, different processing methods, including upsampling, downsampling, downscaling, and upscaling, are used based on different core parameters and the current color format information or the amount of acquired information to match the acquired information with the amount of information specified by the WCP. This completes the target information correction process and yields a final chrominance prediction value. Because all pixels in the reference luminance and chrominance pixels acquired from adjacent regions exhibit different spatial characteristics depending on the image's content characteristics when the color format information is different, upsampling or downsampling can enhance the continuity of their spatial correlation or remove certain information with weak spatial correlation to a certain extent. This makes the target information used as the final input of the WCP prediction technique more accurate, thereby improving the accuracy of chrominance prediction, saving bitrate, and enhancing encoding and decoding performance.
[0595] In another embodiment of the present application, see Figure 18, which shows a flow chart of a coding method provided by an embodiment of the present application. As shown in Figure 18, the method may include:
[0596] S1801: Determine reference information of the current block.
[0597] It should be noted that the encoding method of the embodiment of the present application is applied to an encoding device, or an encoding device integrated with the encoding device (also referred to as "encoder"). In addition, the encoding method of the embodiment of the present application can specifically refer to an intra-frame prediction method, more specifically, a pre-processing operation method for weighted chrominance prediction.
[0598] In an embodiment of the present application, a video image may be divided into a plurality of coding blocks, each of which may include a first color component, a second color component, and a third color component, and the current block herein refers to a coding block in the video image that is currently to be intra-frame predicted. In addition, assuming that the current block predicts the first color component, and the first color component is a luminance component, that is, the component to be predicted is a luminance component, then the current block may also be referred to as a luminance prediction block; or, assuming that the current block predicts the second color component, and the second color component is a chrominance component, that is, the component to be predicted is a chrominance component, then the current block may also be referred to as a chrominance prediction block.
[0599] It should also be noted that, in an embodiment of the present application, the reference information of the current block may include the values of the first color component sampling points in the adjacent areas of the current block and the values of the second color component sampling points in the adjacent areas of the current block. These sampling points (Samples) may be determined based on the encoded pixels in the adjacent areas of the current block. In some embodiments, the adjacent areas of the current block may include at least one of the following: an upper adjacent area, an upper right adjacent area, a left adjacent area, and a lower left adjacent area.
[0600] In addition, the reference information of the current block may also include the reconstructed value of the first reference color component sampling point in the current block. Therefore, in some embodiments, determining the reference information of the current block may include:
[0601] Reference information of the current block is determined according to the value of the first color component sampling point in the adjacent area of the current block, the value of the second color component sampling point in the adjacent area of the current block, and the reconstructed value of the first reference color component sampling point in the current block.
[0602] It should be noted that, in the embodiment of the present application, the reference pixel of the current block may refer to a reference pixel point adjacent to the current block, or may be referred to as a first color component sampling point or a second color component sampling point in an adjacent area of the current block, and may be represented by Neighboring Sample or Reference Sample. The term "adjacent" here may refer to spatial proximity, but is not limited thereto. For example, "adjacent" may refer to temporal proximity, spatial and temporal proximity, or even a reference pixel of the current block obtained by performing some processing on spatially adjacent, temporally adjacent, or spatially and temporally adjacent reference pixels, and the embodiment of the present application does not impose any limitation thereto.
[0603] It should also be noted that in the embodiment of the present application, the first color component is the luminance component, and the second color component is the chrominance component. Therefore, the value of the first color component sampling point in the adjacent area of the current block represents the reference luminance information corresponding to the reference pixel of the current block, and the value of the second color component sampling point in the adjacent area of the current block represents the reference chrominance information corresponding to the reference pixel of the current block.
[0604] S1802: Filter part of the reference information of the current block to determine target information of the current block.
[0605] In an embodiment of the present application, since the number of values of the first color component sampling points and the number of values of the second color component sampling points in the adjacent area of the current block are different, it may be necessary to filter the values of the first color component sampling points or to filter the values of the second color component sampling points so that the two numbers are the same.
[0606] In a possible implementation, filtering part of the information in the reference information of the current block to determine the target information of the current block may include: performing a first filtering process on the values of the first color component sampling points in the adjacent area of the current block to obtain filtered sample values of the first color component in the adjacent area of the current block;
[0607] Target information of the current block is determined according to filtered samples of the first color component in the adjacent area of the current block.
[0608] In an embodiment of the present application, the number of filtered sample values of the first color component in the adjacent area of the current block is equal to L, where L represents the number of values of the second color component sampling points in the adjacent area of the current block, and L is a positive integer.
[0609] In a specific embodiment, the first filtering process may be a downsampling filtering process. For example, the amount of reference luminance information of the current block is generally greater than the amount of reference chrominance information, and thus the reference luminance information of the current block may be downsampled and filtered so that the amount of reference luminance information is the same as the amount of reference chrominance information.
[0610] It should also be noted that since the amount of reference luminance information and the amount of reference chrominance information of the current block will also be affected by the color format information; therefore, in some embodiments, the method may also include: based on the color format information, performing a first filtering process on the value of the first color component sampling point in the adjacent area of the current block to obtain the filtered sample value of the first color component in the adjacent area of the current block.
[0611] In a specific embodiment, the first filtering process may further include:
[0612] If the color format information indicates 4:4:4 sampling, downsampling filtering is not performed on the values of the first color component sampling points in the adjacent areas of the current block;
[0613] If the color format information indicates 4:2:2 sampling, downsampling filtering is performed on the values of the first color component sampling points in the adjacent areas of the current block;
[0614] If the color format information indicates 4:2:0 sampling, downsampling filtering is performed on the values of the first color component sampling points in the adjacent areas of the current block.
[0615] It should be noted that, in the embodiment of the present application, when the color format information indicates 4:2:0 sampling, the downsampling rate is the reciprocal of a positive integer multiple of 2.
[0616] In this way, for the downsampling filtering of the reference luminance information obtained from the adjacent area, a multi-tap filter (such as a four-tap filter, a five-tap filter, a six-tap filter, etc.) can be used for downsampling, or a pixel of multiple reference luminance pixels at the corresponding position of the reference chrominance pixel can be obtained, or even other downsampling methods can be used, which are not specifically limited in the embodiments of the present application.
[0617] Furthermore, when the number of reference luminance information after downsampling and filtering is L and is the same as the number of reference chrominance information, it is also necessary to match L with a preset number. Therefore, in some embodiments, the method may further include:
[0618] Compare L with the preset number;
[0619] Based on the comparison result of L and the preset number, performing a second filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block;
[0620] Target information of the current block is determined according to filtered adjacent samples of the first color component of the current block and filtered adjacent samples of the second color component of the current block.
[0621] In a specific embodiment, performing a second filtering process on the filtered samples of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block based on the comparison result of L with the preset number to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block may include:
[0622] If L is greater than a preset number, performing a first sub-filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block;
[0623] If L is less than a preset number, a second sub-filtering process is performed on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0624] It should be noted that in the embodiment of the present application, the preset number can be represented by inSize. Specifically, if L is equal to the preset number, no operation is required; if L is greater than the preset number, a first sub-filtering process is performed on the reference luminance information and the reference chrominance information (i.e., the size is reduced); if L is less than the preset number, a second sub-filtering process is performed on the reference luminance information and the reference chrominance information (i.e., the size is increased).
[0625] In another possible implementation, filtering the portion of the reference information of the current block to determine the target information of the current block may include: performing a third filtering process on the values of the second color component sampling points in the adjacent area of the current block to obtain filtered sample values of the second color component in the adjacent area of the current block;
[0626] The target information of the current block is determined according to the filtered sample values of the second color component in the adjacent area of the current block.
[0627] In an embodiment of the present application, the number of filtered sample values of the second color component in the adjacent area of the current block is equal to K, where K represents the number of values of the first color component sampling points in the adjacent area of the current block, and K is a positive integer.
[0628] In a specific embodiment, the third filtering process is an upsampling filtering process. For example, the amount of reference luminance information of the current block is generally greater than the amount of reference chrominance information. Therefore, the upsampling filtering process may also be performed on the reference chrominance information of the current block to make the amount of reference luminance information equal to the amount of reference chrominance information.
[0629] Since the amount of reference luminance information and the amount of reference chrominance information of the current block will also be affected by the color format information; therefore, in some embodiments, the method may further include: based on the color format information, performing a third filtering process on the values of the second color component sampling points in the adjacent area of the current block to obtain the filtered sample values of the second color component in the adjacent area of the current block.
[0630] In a specific embodiment, the third filtering process may further include:
[0631] If the color format information indicates 4:4:4 sampling, upsampling filtering is not performed on the values of the second color component sampling points in the adjacent areas of the current block;
[0632] If the color format information indicates 4:2:2 sampling, upsampling filtering is performed on the values of the second color component sampling points in the adjacent area of the current block;
[0633] If the color format information indicates 4:2:0 sampling, up-sampling filtering is performed on the values of the second color component sampling points in the adjacent area of the current block.
[0634] It should be noted that, in the embodiment of the present application, when the color format information indicates 4:2:0 sampling, the upsampling rate is a positive integer multiple of 2. In this way, when the amount of reference luminance information obtained from the adjacent area is different from the amount of reference chrominance information, the reference chrominance information obtained from the adjacent area can be upsampled and filtered to match the amount of reference luminance information.
[0635] In this way, for the upsampling filtering of the reference chromaticity information obtained from the adjacent area, upsampling can be performed by linear interpolation or nonlinear interpolation, or a reference chromaticity pixel can be copied, or even other upsampling methods can be used, which are not specifically limited in the embodiments of the present application.
[0636] Furthermore, when the number of reference chrominance information after upsampling and filtering is K and is the same as the number of reference luminance information, it is also necessary to match K with a preset number. Therefore, in some embodiments, the method may further include:
[0637] Compare K with the preset number;
[0638] performing a fourth filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block according to a comparison result of K with a preset number, to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block;
[0639] Target information of the current block is determined according to filtered adjacent samples of the first color component of the current block and filtered adjacent samples of the second color component of the current block.
[0640] In a specific embodiment, performing a fourth filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block based on the comparison result of K with the preset number to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block may include:
[0641] If K is greater than a preset number, performing a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block;
[0642] If K is less than a preset number, a second sub-filtering process is performed on the values of the first color component sampling points in the adjacent area of the current block and the filtered sample values of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0643] It should be noted that in the embodiment of the present application, the preset number can be represented by inSize. Specifically, if K is equal to inSize, no operation is required; if K is greater than inSize, a first sub-filtering process is performed on the reference luminance information and the reference chrominance information (i.e., the size is reduced); if K is less than inSize, a second sub-filtering process is performed on the reference luminance information and the reference chrominance information (i.e., the size is increased).
[0644] It can also be understood that in the embodiment of the present application, if the color format information is not considered, the number of reference luminance information and the number of reference chrominance information obtained from the adjacent area can be directly matched with inSize so that the two numbers are the same.
[0645] In a possible implementation, filtering part of the reference information of the current block to determine the target information of the current block may include: determining that the number of values of the first color component sampling points in the adjacent area of the current block is a first number;
[0646] performing a fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block according to a comparison result of the first number and the preset number, to determine filtered adjacent sample values of the first color component of the current block;
[0647] Target information of the current block is determined according to filtered adjacent samples of the first color component of the current block.
[0648] In a specific embodiment, performing the fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block may include:
[0649] If the first number is greater than the preset number, performing a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the first color component of the current block;
[0650] If the first number is smaller than the preset number, a second sub-filtering process is performed on the values of the first color component sampling points in the adjacent areas of the current block to determine filtered adjacent sample values of the first color component of the current block.
[0651] It should be noted that in an embodiment of the present application, the amount of reference brightness information obtained from adjacent areas can be matched with a preset number. The amount of reference brightness information obtained from adjacent areas (i.e., the first number) can be represented by refYnum; the preset number can be represented by inSize. Specifically, if refYnum is equal to inSize, then no operation is required; if refYnum is greater than inSize, then the reference brightness information needs to be subjected to a first sub-filtering process (i.e., reduced in size); if refYnum is less than inSize, then the reference brightness information needs to be subjected to a second sub-filtering process (i.e., enlarged in size).
[0652] In another possible implementation, filtering part of the reference information of the current block to determine the target information of the current block may further include: determining that the number of values of the second color component sampling points in the adjacent area of the current block is a second number;
[0653] performing a sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block according to a comparison result between the second number and the preset number, to determine filtered adjacent sample values of the second color component of the current block;
[0654] Target information of the current block is determined according to filtered adjacent samples of the second color component of the current block.
[0655] In a specific embodiment, performing the sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block may include:
[0656] If the second number is greater than the preset number, performing a first sub-filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the second color component of the current block;
[0657] If the second number is smaller than the preset number, a second sub-filtering process is performed on the values of the second color component sampling points in the adjacent areas of the current block to determine filtered adjacent sample values of the second color component of the current block.
[0658] It should also be noted that in the embodiment of the present application, the amount of reference chromaticity information obtained from the adjacent area can be matched with a preset number. The amount of reference chromaticity information obtained from the adjacent area (i.e., the second number) can be represented by refCnum; the preset number can be represented by inSize. Specifically, if refCnum is equal to inSize, then no operation is required; if refCnum is greater than inSize, then the reference chromaticity information needs to be subjected to a first sub-filtering process (i.e., reduced in size); if refCnum is less than inSize, then the reference chromaticity information needs to be subjected to a second sub-filtering process (i.e., enlarged in size).
[0659] In the embodiment of the present application, the first sub-filtering process may be a downsampling filtering process. In addition, in some embodiments, the first sub-filtering process may further include at least one of the following:
[0660] The sub-filtering process is performed by selecting fixed interval positions;
[0661] The sub-filtering process is performed by selecting continuous positions.
[0662] In the embodiment of the present application, the second sub-filtering process may be an upsampling filtering process. In addition, in some embodiments, the second sub-filtering process may further include: performing sub-filtering process using an interpolation method at fixed intervals.
[0663] It is also understandable that if the chrominance component of the current block is predicted, the reference information of the current block may also include the reconstructed luminance information of the current block. Therefore, the reference information of the current block may also be determined based on the reconstructed value of the first reference color component sampling point in the current block.
[0664] In an embodiment of the present application, the first reference color component may be a luminance component; then, the reconstructed value of the first reference color component sampling point in the current block is the reconstructed luminance information of the current block. Because the amount of reconstructed luminance information of the current block is different from the preset size, the reconstructed luminance information of the current block also needs to be filtered to convert the amount of reconstructed luminance information obtained from the current block into the preset size for input. Here, the preset size is predSizeW×predSizeH; where predSizeW represents the width of the prediction block, and predSizeH represents the height of the prediction block.
[0665] In some embodiments, filtering part of the information in the reference information of the current block to determine the target information of the current block may include: performing a seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block; and determining the target information of the current block based on the filtered sample value of the first reference color component sampling point in the current block.
[0666] In a specific embodiment, performing the seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block may include:
[0667] Determine a first width and a first height according to a reconstructed value of a first reference color component sampling point in the current block;
[0668] If the first width is greater than the width of the prediction block, or the first height is greater than the height of the prediction block, performing a third sub-filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine a filtered sample value of the first reference color component sampling point in the current block;
[0669] If the first width is smaller than the width of the prediction block, or the first height is smaller than the height of the prediction block, a fourth sub-filtering process is performed on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block.
[0670] It should be noted that, in the embodiment of the present application, assuming that the amount of reconstructed luminance information is recSizeW×recSizeH, then recSizeW represents the first width and recSizeH represents the first height. Specifically, if recSizeW is equal to predSizeW and recSizeH is equal to predSizeH, then no operation is required; if recSizeW is greater than predSizeW, or recSizeH is greater than predSizeH, then it is necessary to perform a first sub-filtering process on the reconstructed luminance information (i.e., reduce the size); if recSizeW is less than predSizeW, or recSizeH is less than predSizeH, then it is necessary to perform a second sub-filtering process on the reconstructed luminance information (i.e., increase the size).
[0671] It should also be noted that, in the embodiment of the present application, the third sub-filtering process may be a downsampling filtering process. In addition, in some embodiments, the third sub-filtering process may also include at least one of the following:
[0672] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction;
[0673] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction;
[0674] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction;
[0675] The reconstructed value of the first reference color component sampling point in the current block is downsampled and filtered in the vertical direction and then downsampled and filtered in the horizontal direction.
[0676] Here, first, the horizontal downsampling factor can be calculated based on recSizeW and predSizeW, and the vertical downsampling factor can be calculated based on recSizeH and predSizeH; then, the reconstructed luminance information is downsampled and filtered according to the horizontal downsampling factor and the vertical downsampling factor. Specifically, if the horizontal downsampling factor is greater than 1 and the vertical downsampling factor is equal to 1, then the reconstructed luminance information only needs to be downsampled in the horizontal direction; if the horizontal downsampling factor is equal to 1 and the vertical downsampling factor is greater than 1, then the reconstructed luminance information only needs to be downsampled in the vertical direction; if the horizontal downsampling factor is greater than 1 and the vertical downsampling factor is greater than 1, then the reconstructed luminance information needs to be downsampled in both the horizontal and vertical directions, wherein downsampling can be performed first in the horizontal direction and then in the vertical direction, or first in the vertical direction and then in the horizontal direction, and even the convolution operation in the neural network structure can be used to replace the downsampling operation here, and the embodiments of the present application do not impose any restrictions.
[0677] It should also be noted that, in the embodiment of the present application, the fourth sub-filtering process is an upsampling filtering process. In addition, in some embodiments, the fourth sub-filtering process may also include at least one of the following:
[0678] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction;
[0679] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction;
[0680] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction;
[0681] The reconstructed value of the first reference color component sampling point in the current block is subjected to up-sampling filtering in the vertical direction and then to up-sampling filtering in the horizontal direction.
[0682] Here, first, the horizontal upsampling factor is calculated according to recSizeW and predSizeW, and the vertical upsampling factor is calculated according to recSizeH and predSizeH; then, the reconstructed luminance information is upsampled and filtered according to the horizontal upsampling factor and the vertical upsampling factor. Specifically, if the horizontal upsampling factor is greater than 1 and the vertical upsampling factor is equal to 1, then the reconstructed luminance information only needs to be upsampled in the horizontal direction; if the horizontal upsampling factor is equal to 1 and the vertical upsampling factor is greater than 1, then the reconstructed luminance information only needs to be upsampled in the vertical direction; if the horizontal upsampling factor is greater than 1 and the vertical upsampling factor is greater than 1, then the reconstructed luminance information needs to be upsampled in both the horizontal and vertical directions, wherein upsampling can be performed first in the horizontal direction and then in the vertical direction, or first in the vertical direction and then in the horizontal direction, and even the upsampling results of "first in the horizontal direction and then in the vertical direction" and the upsampling results of "first in the vertical direction and then in the horizontal direction" can be weighted averaged to determine the final upsampling result, and even the convolution operation in the neural network structure can be used to replace the upsampling operation here, and the embodiments of the present application are not limited in any way.
[0683] That is, in an embodiment of the present application, the reference information of the current block may include: the value of the first color component sampling point in the adjacent area of the current block, the value of the second color component sampling point in the adjacent area of the current block, and the reconstructed value of the first reference color component sampling point in the current block; the target information may include: the filtered adjacent sample value of the first color component of the current block, the filtered adjacent sample value of the second color component of the current block, and the filtered sample value of the first reference color component sampling point in the current block. If the value of the first color component sampling point in the adjacent area of the current block does not require filtering, then the filtered adjacent sample value of the first color component of the current block is equal to the value of the first color component sampling point in the adjacent area of the current block; similarly, if the value of the second color component sampling point in the adjacent area of the current block does not require filtering, then the filtered adjacent sample value of the second color component of the current block is equal to the value of the second color component sampling point in the adjacent area of the current block; if the reconstructed value of the first reference color component sampling point in the current block does not require filtering, then the filtered sample value of the first reference color component sampling point in the current block is equal to the reconstructed value of the first reference color component sampling point in the current block.
[0684] In this way, the obtained target information may include: the filtered adjacent samples of the first color component of the current block, the filtered adjacent samples of the second color component of the current block, and the filtered samples of the first reference color component sampling point in the current block, that is, inSize amount of reference chromaticity information, inSize amount of reference luminance information, and reconstructed luminance information that meets the preset size.
[0685] S1803: Determine a weighting coefficient according to the reference sample value of the first color component in the target information.
[0686] It should be noted that after determining the filtered adjacent samples of the first color component of the current block, the filtered adjacent samples of the second color component of the current block, and the filtered samples of the first reference color component sampling point in the current block, in some embodiments, the method may further include: setting the reference sample of the first color component in the target information to the absolute value of the difference between the filtered adjacent samples of the first color component of the current block and the filtered sample of the first reference color component sampling point in the current block.
[0687] In some embodiments, the method may further include: setting the reference sample value of the second color component in the target information as the filtered adjacent sample value of the second color component of the current block.
[0688] In addition, considering that the filtering processing of the reference information can be filtering processing only for the reference luminance information, filtering processing only for the reconstructed luminance information, filtering processing only for the reference luminance information and the reconstructed luminance information, or even filtering processing for the reference chrominance information, reference luminance information and the reconstructed luminance information, there is no limitation here. Then, for the calculation of the luminance difference, the method may also include: setting the reference sample value of the first color component in the target information to the absolute value of the difference between the value of the first color component sampling point in the adjacent area of the current block and the reconstructed value of the first reference color component sampling point in the current block; or setting the reference sample value of the first color component in the target information to the absolute value of the difference between the filtered adjacent sample value of the first color component of the current block and the reconstructed value of the first reference color component sampling point in the current block; or setting the reference sample value of the first color component in the target information to the absolute value of the difference between the value of the first color component sampling point in the adjacent area of the current block and the filtered sample value of the first reference color component sampling point in the current block, there is no specific limitation here.
[0689] In some embodiments, determining the weighting coefficient according to the reference sample value of the first color component in the target information may include:
[0690] Determining a value corresponding to the reference sample value of the first color component under a preset mapping relationship;
[0691] Set the weighting factor equal to the value.
[0692] It should be noted that, in this embodiment of the present application, the reference sample value of the first color component may be the absolute value of the difference between the filtered adjacent sample value of the first color component of the current block and the filtered sample value of the first reference color component sampling point in the current block. The first reference color component is the first color component, and the first color component is a color component different from the second color component to be predicted in this embodiment of the present application.
[0693] It should be noted that, in the embodiment of the present application, the chrominance component of the pixel to be predicted in the current block is predicted, and at least one pixel to be predicted in the current block is selected, and the brightness difference between its reconstructed brightness and the reference brightness in the adjacent area is calculated (using |ΔY k | represents). Where, if |ΔY k | is small, indicating that the similarity of the brightness values is relatively strong, and the corresponding weighting coefficient (using w k indicates) is given a larger weight; on the contrary, if |ΔY k | is larger, indicating that the similarity of brightness values is weaker, w k Assign a smaller weight. That is, under the preset mapping relationship, w k and|ΔY k | are approximately inversely proportional.
[0694] Further, in some embodiments, determining the value corresponding to the reference sample value of the first color component under a preset mapping relationship may include: determining a first factor; determining a first product value based on the first factor and the reference sample value of the first color component; and determining the value corresponding to the first product value under the preset mapping relationship.
[0695] In a specific embodiment, determining the first factor may include: the first factor is a preset constant value.
[0696] In another specific embodiment, determining the first factor may include: determining a value of the first factor according to a size parameter of the current block.
[0697] Furthermore, in some embodiments, the method may further include: determining the value of the first factor according to a preset mapping lookup table between the size parameter of the current block and the value of the first factor.
[0698] Here, the size parameter of the current block may include at least one of the following parameters: the width of the current block, the height of the current block, and the product of the width and the height of the current block.
[0699] It should be noted that embodiments of the present application can employ a classification approach to fix the value of the first factor. For example, the current block size parameters can be classified into three categories, and the value of the first factor corresponding to each category can be determined. For example, Table 1 above illustrates a schematic representation of the correspondence between the first factor and the current block size parameters.
[0700] In another specific embodiment, determining the first factor may include: determining a value of the first factor according to the number of reference pixels of the current block.
[0701] Furthermore, in some embodiments, the method may further include: determining the value of the first factor according to a preset mapping lookup table between the number of reference pixels of the current block and the value of the first factor.
[0702] It should be noted that the embodiment of the present application can classify the number of reference pixels into three categories, and still use the classification method to fix the value of the first factor. For example, the aforementioned Table 2 shows a corresponding relationship between the first factor and the number of reference pixels of the current block.
[0703] Furthermore, for the first product value, determining the first product value according to the first factor and the reference sample value of the first color component may include:
[0704] Setting the first product value equal to the product of the first factor and the reference sample value of the first color component; or,
[0705] The first product value is set to be equal to a value obtained by right-shifting the reference sample value of the first color component, wherein the number of bits of the right-shift is equal to the first factor; or
[0706] The first product value is set to a value obtained by performing addition and bit shift operations on the reference sample value of the first color component according to the first factor.
[0707] For example, assuming the first factor is 0.25 and the reference sample value of the first color component is denoted by Ref, the first product value can be equal to 0.25×Ref. 0.25×Ref can be expressed as Ref / 4, that is, Ref>>2. Furthermore, during the fixed-point calculation process, floating-point numbers may be converted to addition and shift operations; that is, there is no restriction on the calculation method for the first product value.
[0708] It can also be understood that in the embodiment of the present application, the first reference color component can also be a chroma component, that is, at least one pixel to be predicted with obvious color difference in the current block is selected, and the chroma difference between its reconstructed chroma and the reference chroma in the adjacent area is calculated (using |ΔC k | represents). Where, if |ΔC k | is small, indicating that the similarity of chromaticity values is relatively strong, and the corresponding weighting coefficient (using w k ) can be given a larger weight; on the contrary, if |ΔC k | is larger, indicating that the similarity of chromaticity values is weaker, w kA smaller weight can be assigned, that is, when calculating the weighting coefficient, the reference sample value of the first color component can also be |ΔC k |, and then calculate the weighting coefficient.
[0709] In short, in the embodiment of the present application, for the reference sample value of the first color component, it can be |ΔC k |, the absolute value of the chromaticity difference; or |ΔY k |, that is, the absolute value of the brightness difference; or it can be |αΔY k |, that is, the product of the absolute value of the brightness difference and a preset multiplier, etc. The preset multiplier here is the second factor described in the embodiment of the present application.
[0710] Furthermore, with respect to the second factor, in a specific embodiment, the method may further include: performing a least squares calculation based on the first color component value of the reference pixel and the second color component value of the reference pixel to determine the second factor.
[0711] That is to say, assuming that the number of reference pixels is N, the first color component value of the reference pixel is the reference luminance information of the current block, and the second color component value of the reference pixel is the reference chrominance information of the current block, then the chrominance component values and luminance component values of the N reference pixels can be calculated by least squares method, for example, the least squares regression calculation is as shown in the aforementioned formula (5), and the second factor can be calculated.
[0712] In addition, for the preset mapping relationship, the preset mapping relationship can be a preset function relationship. In some embodiments, the preset mapping relationship can be a Softmax function, as shown in the aforementioned formula (6) or formula (7). Wherein, the Softmax function is a normalized exponential function, but the embodiment of the present application can also be normalized, and its value is not limited to the range of [0,1].
[0713] It should also be noted that, in addition to the Softmax function, in other embodiments, the preset mapping relationship can be a weighted function that has an inverse relationship with the reference sample value of the first color component, as shown in the aforementioned formula (8) or formula (9).
[0714] In this way, the preset mapping relationship can be as shown in formula (4), or as shown in formula (6) or formula (7), or as shown in formula (8) or formula (9), or even other function models that can fit the trend that the closer the reference luminance value of the reference pixel is to the luminance reconstruction value of the pixel to be predicted in the current block, the higher the importance of the reference chrominance value of the reference pixel to the pixel to be predicted in the current block is to construct a weighted coefficient; in addition, the preset mapping relationship can also be a preset look-up table method, such as using an array element look-up table method to reduce a part of the calculation operations, etc., which is not specifically limited in the embodiments of the present application.
[0715] Thus, according to the reference sample value of the first color component (eg |ΔY kij |), the weighting coefficients can be determined, which can include: w1, w2, ..., w N N weighting coefficients are used, where N represents the number of reference samples of the second color component. In theory, the sum of the N weighting coefficients is equal to 1, and each weighting coefficient is a value greater than or equal to 0 and less than or equal to 1.
[0716] S1804: Determine a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information.
[0717] S1805: Determine the predicted difference value of the second color component sampling point of the current block according to the predicted block.
[0718] It should be noted that after the weighting coefficient is determined, the reference sample value of the second color component can be weighted calculated using the weighting coefficient to obtain the predicted value of the second color component sampling point in the prediction block.
[0719] In some embodiments, determining the predicted block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information may include:
[0720] Determining a weighted value of a reference sample value of a second color component and a corresponding weighting coefficient;
[0721] The predicted value of the second color component sampling point in the prediction block is set to be equal to the sum of N weighted values; wherein N represents the number of reference sample values of the second color component, and N is a positive integer.
[0722] That is, if the number of reference samples of the second color component is N, then first determine the weighted value of each reference sample of the second color component and the corresponding weighting coefficient (ie, w k C k), and then use the sum of these N weighted values as the predicted value of the second color component sampling point in the prediction block. Specifically, its calculation formula is as shown in the aforementioned formula (16), and this calculation method is conducive to parallel processing and can speed up the calculation speed.
[0723] It should also be noted that in the embodiments of the present application, the first color component is the luminance component, and the second color component is the chrominance component. Since the resolutions of the luminance component and the chrominance component are different, when filtering the reference information, it includes downsampling filtering the luminance component to align the resolution of the chrominance component, or upsampling filtering the chrominance component to align the resolution of the luminance component.
[0724] Furthermore, for the predicted value of the second color component sampling point in the prediction block, it is usually necessary to be limited within a preset range. Therefore, in some embodiments, the method may further include: performing a correction operation on the predicted value of the second color component sampling point in the prediction block to determine the prediction block of the second color component of the current block.
[0725] It should be noted that in the embodiments of the present application, the preset range may be: between 0 and (1<<BitDepth)-1. If the predicted value exceeds the value range of this preset range, then a corresponding correction operation needs to be performed on the predicted value. Exemplarily, a clamping operation can be performed to ensure that the predicted values of all second color component sampling points in the prediction block are between 0 and (1<<BitDepth)-1.
[0726] Furthermore, after determining the prediction block, the prediction block may also require post-processing operations under certain conditions to be used as the final prediction block. Therefore, in some embodiments, the method may further include:
[0727] Performing relevant processing on the prediction block and using the processed prediction block as the prediction block of the second color component of the current block.
[0728] In a possible implementation manner, performing relevant processing on the prediction block may include: performing filtering enhancement processing on the prediction block and using the processed prediction block as the prediction block of the second color component of the current block.
[0729] In another possible implementation manner, performing relevant processing on the prediction block may include: determining the compensation value of the second color component of the prediction block according to the reference samples in the adjacent region of the current block; correcting the predicted value of the second color component sampling point in the prediction block according to the compensation value to determine the prediction block of the second color component of the current block.
[0730] In another possible implementation, performing related processing on the prediction block may include: performing prediction processing on the second color component sampling point in the prediction block according to at least one prediction mode to determine at least one initial prediction value of the second color component sampling point in the prediction block; and performing weighted fusion processing based on the at least one initial prediction value and the prediction value of the second color component sampling point in the prediction block to determine the prediction block of the second color component of the current block.
[0731] That is to say, for the related processing of the prediction block, in order to reduce the instability caused by the independent parallel prediction of WCP pixel by pixel, the prediction block can be smoothed and filtered as the final chrominance prediction value. Alternatively, in order to further improve the accuracy of the WCP prediction value, the prediction block can be subjected to a position-related correction process. For example: a chrominance compensation value is calculated for each pixel to be predicted using reference pixels with close spatial positions, and the prediction block is corrected using this chrominance compensation value, and the corrected prediction value is used as the final chrominance prediction value. Alternatively, in order to further improve the accuracy of the WCP prediction value, the chrominance prediction value calculated by other chrominance prediction modes can be weightedly fused with the chrominance prediction value calculated by WCP, and the fusion result is used as the final chrominance prediction value. Alternatively, in order to improve the WCP prediction performance, a neural network model can be used to correct the chrominance prediction value calculated by WCP, etc. The embodiments of the present application do not impose any restrictions on this.
[0732] In some embodiments, after the prediction block is determined, as shown in FIG19 , after S1804 , the following steps may be further included:
[0733] S1901: Determine a predicted value of a second color component sampling point of a current block according to a predicted block.
[0734] S1902: Determine a predicted difference value of the second color component sampling point of the current block according to the original value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.
[0735] S1903: Encode the predicted difference value of the second color component sampling point of the current block, and write the obtained coded bits into the bitstream.
[0736] It should be noted that, in an embodiment of the present application, the prediction block may include the predicted values of at least some of the second color component sampling points in the current block. If the prediction block includes the predicted values of all of the second color component sampling points in the current block, then the predicted values of the second color component sampling points in the current block may be set equal to the values of the prediction block. If the prediction block includes the predicted values of some of the second color component sampling points in the current block, then upsampling filtering may be performed on the values of the prediction block, and the predicted values of the second color component sampling points in the current block may be set equal to the output values after the upsampling filtering.
[0737] It should also be noted that after determining the predicted value of the second color component sampling point of the current block, a predicted difference value of the second color component sampling point can be determined based on the original value of the second color component sampling point and the predicted value of the second color component sampling point. Specifically, the predicted difference value of the second color component sampling point can be determined by subtracting the original value of the second color component sampling point from the predicted value of the second color component sampling point. In this way, after the predicted difference value of the second color component sampling point is written into the bitstream, the predicted difference value of the second color component sampling point can be obtained through decoding at the decoding end, so as to restore the reconstructed value of the second color component sampling point of the current block.
[0738] In summary, taking the chroma prediction of the current block as an example, in the embodiment of the present application, the reconstructed luminance information of the current block, the reference luminance information of the adjacent area, and the reference chroma information are all encoded reference information. Therefore, the embodiment of the present application proposes a weighted chroma prediction technology that utilizes the above information, and in the process of determining the target information of the current block based on this information, the information of the adjacent areas of the current block, the information of the current block, and the current color format information can be analyzed and processed. Among them, it mainly includes: on the one hand, according to the current color format information, it is selected whether to downsample the reference luminance information of the adjacent area or whether to upsample the reference chroma information of the adjacent area; on the other hand, the reference luminance information of the adjacent area, the reference chroma information, and the reconstructed luminance information in the current block can be matched with the preset size of the core parameter through different magnification / reduction operations, which can improve the accuracy of the WCP prediction technology.
[0739] The embodiment of the present application also provides a coding method, which determines the reference information of the current block; filters part of the information in the reference information of the current block to determine the target information of the current block; determines the weighting coefficient based on the reference sample value of the first color component in the target information; determines the prediction block of the second color component of the current block based on the weighting coefficient and the reference sample value of the second color component in the target information; and determines the prediction difference of the sampling point of the second color component of the current block based on the prediction block. In this way, for the color component information in the adjacent areas of the current block and the color component information in the current block, the accuracy of these reference information can be improved because the characteristics of the color format information are fully considered; and by filtering these reference information such as upsampling and downsampling, the continuity of their spatial correlation can be enhanced to a certain extent or some information with weak spatial correlation can be removed. The target information obtained after filtering can be used to better perform chroma prediction, thereby improving the accuracy of chroma prediction, saving bit rate, and improving encoding and decoding performance.
[0740] In another embodiment of the present application, the embodiment of the present application further provides a code stream, which is generated by bit encoding based on the information to be encoded; wherein the information to be encoded includes at least: the predicted difference value of the second color component sampling point of the current block.
[0741] In an embodiment of the present application, after the predicted difference value of the second color component sampling point of the current block is transmitted from the encoding end to the decoding end, the decoding end obtains the predicted difference value of the second color component sampling point through decoding, and then combines it with the predicted value of the second color component sampling point of the current block to restore the reconstructed value of the second color component sampling point of the current block. In this way, by fully considering the characteristics of the input color format information, the accuracy of the reference information of the adjacent area is improved; and by fully utilizing spatial correlation, the reference information of the adjacent area and the reconstructed luminance information of the current block are subjected to filtering processes such as upsampling, downsampling, scaling up, and scaling down, thereby not only improving the accuracy of chrominance prediction and saving bit rate, but also improving encoding and decoding performance.
[0742] In another embodiment of the present application, based on the same inventive concept as the above embodiment, refer to FIG20 , which shows a schematic diagram of the composition structure of an encoding device 300 provided in an embodiment of the present application. As shown in FIG20 , the encoding device 300 may include: a first determination unit 3001, a first filtering unit 3002, and a first prediction unit 3003; wherein,
[0743] A first determining unit 3001 is configured to determine reference information of a current block;
[0744] A first filtering unit 3002 is configured to filter part of the reference information of the current block to determine target information of the current block;
[0745] The first determining unit 3001 is further configured to determine a weighting coefficient according to a reference sample value of the first color component in the target information;
[0746] The first prediction unit 3003 is configured to determine a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information;
[0747] The first determining unit 3001 is further configured to determine a prediction difference value of a second color component sampling point of the current block according to the prediction block.
[0748] In some embodiments, the prediction block includes predicted values of at least some of the second color component sampling points in the current block.
[0749] In some embodiments, the first determination unit 3001 is further configured to determine the reference information of the current block based on the value of the first color component sampling point in the adjacent area of the current block, the value of the second color component sampling point in the adjacent area of the current block, and the reconstructed value of the first reference color component sampling point in the current block.
[0750] In some embodiments, the neighboring region of the current block includes at least one of the following: an upper neighboring region, an upper-right neighboring region, a left neighboring region, and a lower-left neighboring region.
[0751] In some embodiments, the first filtering unit 3002 is further configured to perform a first filtering process on the values of the first color component sampling points in the adjacent area of the current block to obtain the filtered sample values of the first color component in the adjacent area of the current block; and determine the target information of the current block based on the filtered sample values of the first color component in the adjacent area of the current block.
[0752] In some embodiments, the number of filtered sample values of the first color component in the adjacent area of the current block is equal to L, where L represents the number of values of the second color component sampling points in the adjacent area of the current block, and L is a positive integer.
[0753] In some embodiments, the first filtering process is a downsampling filtering process.
[0754] In some embodiments, the first filtering unit 3002 is further configured to perform a first filtering process on the values of the first color component sampling points in the adjacent area of the current block based on the color format information to obtain filtered sample values of the first color component in the adjacent area of the current block.
[0755] In some embodiments, the first filtering unit 3002 is further configured to perform downsampling filtering on the values of the first color component sampling points in the adjacent area of the current block if the color format information indicates 4:2:0 sampling; wherein the downsampling rate is the reciprocal of a positive integer multiple of 2.
[0756] In some embodiments, the first filtering unit 3002 is further configured to compare L with a preset number; and based on the comparison result of L with the preset number, perform a second filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; and determine the target information of the current block based on the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0757] In some embodiments, the first filtering unit 3002 is further configured to, if L is greater than a preset number, perform a first sub-filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; if L is less than the preset number, perform a second sub-filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0758] In some embodiments, the first filtering unit 3002 is further configured to perform a third filtering process on the values of the second color component sampling points in the adjacent area of the current block to obtain the filtered sample values of the second color component in the adjacent area of the current block; and determine the target information of the current block based on the filtered sample values of the second color component in the adjacent area of the current block.
[0759] In some embodiments, the number of filtered sample values of the second color component in the adjacent area of the current block is equal to K, where K represents the number of values of the first color component sampling points in the adjacent area of the current block, and K is a positive integer.
[0760] In some embodiments, the third filtering process is an upsampling filtering process.
[0761] In some embodiments, the first filtering unit 3002 is further configured to perform a third filtering process on the values of the second color component sampling points in the adjacent area of the current block based on the color format information to obtain filtered sample values of the second color component in the adjacent area of the current block.
[0762] In some embodiments, the first filtering unit 3002 is further configured to perform upsampling filtering on the values of the second color component sampling points in the adjacent area of the current block if the color format information indicates 4:2:0 sampling; wherein the upsampling rate is a positive integer multiple of 2.
[0763] In some embodiments, the first filtering unit 3002 is further configured to compare K with a preset number; and based on the comparison result of K with the preset number, perform a fourth filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered sample values of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; and determine the target information of the current block based on the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0764] In some embodiments, the first filtering unit 3002 is further configured to, if K is greater than a preset number, perform a first sub-filtering process on the values of the first color component sampling points in the adjacent areas of the current block and the filtered sample values of the second color component in the adjacent areas of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; if K is less than a preset number, perform a second sub-filtering process on the values of the first color component sampling points in the adjacent areas of the current block and the filtered sample values of the second color component in the adjacent areas of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0765] In some embodiments, the first filtering unit 3002 is further configured to determine that the number of values of the first color component sampling points in the adjacent area of the current block is a first number; and based on the comparison result of the first number and the preset number, perform a fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block; and determine the target information of the current block based on the filtered adjacent sample values of the first color component of the current block.
[0766] In some embodiments, the first filtering unit 3002 is further configured to, if the first number is greater than a preset number, perform a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block; if the first number is less than the preset number, perform a second sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block.
[0767] In some embodiments, the first filtering unit 3002 is further configured to determine that the number of values of the second color component sampling points in the adjacent area of the current block is a second number; and based on the comparison result of the second number and the preset number, perform a sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block; and determine the target information of the current block based on the filtered adjacent sample values of the second color component of the current block.
[0768] In some embodiments, the first filtering unit 3002 is further configured to, if the second number is greater than a preset number, perform a first sub-filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block; if the second number is less than the preset number, perform a second sub-filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block.
[0769] In some embodiments, the first filtering unit 3002 is further configured to set the reference sample value of the second color component in the target information as the filtered adjacent sample value of the second color component of the current block.
[0770] In some embodiments, the first sub-filtering process is a downsampling filtering process.
[0771] In some embodiments, the first filtering unit 3002 is further configured to perform a first sub-filtering process of at least one of the following:
[0772] The sub-filtering process is performed by selecting fixed interval positions;
[0773] The sub-filtering process is performed by selecting continuous positions.
[0774] In some embodiments, the second sub-filtering process is an upsampling filtering process.
[0775] In some embodiments, the first filtering unit 3002 is further configured to perform at least one of the following second sub-filtering processes: performing sub-filtering processes using an interpolation method at fixed intervals.
[0776] In some embodiments, the first filtering unit 3002 is further configured to perform a seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block; and determine the target information of the current block based on the filtered sample value of the first reference color component sampling point in the current block.
[0777] In some embodiments, the first filtering unit 3002 is further configured to determine the first width and the first height based on the reconstructed value of the first reference color component sampling point in the current block; and if the first width is greater than the width of the prediction block, or the first height is greater than the height of the prediction block, then performing a third sub-filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block; if the first width is less than the width of the prediction block, or the first height is less than the height of the prediction block, then performing a fourth sub-filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block.
[0778] In some embodiments, the third sub-filtering process is a downsampling filtering process.
[0779] In some embodiments, the first filtering unit 3002 is further configured to perform a third sub-filtering process of at least one of the following:
[0780] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction;
[0781] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction;
[0782] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction;
[0783] The reconstructed value of the first reference color component sampling point in the current block is downsampled and filtered in the vertical direction and then downsampled and filtered in the horizontal direction.
[0784] In some embodiments, the fourth sub-filtering process is an upsampling filtering process.
[0785] In some embodiments, the first filtering unit 3002 is further configured to perform at least one of the following fourth sub-filtering operations:
[0786] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction;
[0787] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction;
[0788] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction;
[0789] The reconstructed value of the first reference color component sampling point in the current block is subjected to up-sampling filtering in the vertical direction and then to up-sampling filtering in the horizontal direction.
[0790] In some embodiments, the first filtering unit 3002 is further configured to set the reference sample value of the first color component in the target information to the absolute value of the difference between the value of the first color component sampling point in the adjacent area of the current block and the reconstructed value of the first reference color component sampling point in the current block.
[0791] In some embodiments, the first filtering unit 3002 is further configured to set the reference sample value of the first color component in the target information to the absolute value of the difference between the filtered adjacent sample value of the first color component of the current block and the filtered sample value of the first reference color component sampling point in the current block.
[0792] In some embodiments, the first determining unit 3001 is further configured to determine a value corresponding to the reference sample value of the first color component under a preset mapping relationship; and set the weighting coefficient to be equal to the value.
[0793] In some embodiments, the first determination unit 3001 is further configured to determine a first factor; determine a first product value based on the first factor and a reference sample value of the first color component; and determine a value corresponding to the first product value under a preset mapping relationship.
[0794] In some embodiments, the first factor is a preset constant value.
[0795] In some embodiments, the first determining unit 3001 is further configured to determine the value of the first factor according to a size parameter of the current block; wherein the size parameter of the current block includes at least one of the following parameters: the width of the current block, the height of the current block.
[0796] In some embodiments, the preset mapping relationship is a Softmax function.
[0797] In some embodiments, the preset mapping relationship is a weighting function that has an inverse proportional relationship with the reference sample value of the first color component.
[0798] In some embodiments, the first prediction unit 3003 is further configured to determine the weighted value of the reference sample value of the second color component and the corresponding weighting coefficient; and set the predicted value of the second color component sampling point in the prediction block to be equal to the sum of N weighted values; wherein N represents the number of reference sample values of the second color component, and N is a positive integer.
[0799] In some embodiments, the first prediction unit 3003 is further configured to perform a correction operation on the predicted value of the second color component sampling point in the prediction block to determine the predicted block of the second color component of the current block.
[0800] In some embodiments, the first prediction unit 3003 is further configured to perform filtering enhancement processing on the prediction block, and use the processed prediction block as the prediction block of the second color component of the current block.
[0801] In some embodiments, the first prediction unit 3003 is further configured to determine a compensation value of the second color component of the prediction block based on a reference sample value in an adjacent area of the current block; and to correct the prediction value of the second color component sampling point in the prediction block based on the compensation value to determine the prediction block of the second color component of the current block.
[0802] In some embodiments, the first prediction unit 3003 is further configured to perform prediction processing on the second color component sampling point in the prediction block according to at least one prediction mode to determine at least one initial prediction value of the second color component sampling point in the prediction block; and perform weighted fusion processing based on the at least one initial prediction value and the prediction value of the second color component sampling point in the prediction block to determine the prediction block of the second color component of the current block.
[0803] In some embodiments, the first determination unit 3001 is further configured to determine a predicted value of the second color component sampling point of the current block based on the prediction block; and determine a predicted difference value of the second color component sampling point of the current block based on the original value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.
[0804] In some embodiments, referring to FIG. 20 , the encoding apparatus 300 may further include an encoding unit 3004 configured to encode the predicted difference value of the second color component sampling point of the current block, and write the obtained encoding bits into the bitstream.
[0805] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.
[0806] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0807] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoding device 300. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.
[0808] Based on the composition of the above-mentioned encoding device 300 and the computer-readable storage medium, refer to Figure 21, which shows a specific hardware structure diagram of the encoding device 310 provided in an embodiment of the present application. As shown in Figure 21, the encoding device 310 may include: a first communication interface 3101, a first memory 3102 and a first processor 3103; each component is coupled together through a first bus system 3104. It can be understood that the first bus system 3104 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 3104 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 3104 in Figure 21. Among them,
[0809] The first communication interface 3101 is used to receive and send signals when sending and receiving information with other external network elements;
[0810] A first memory 3102 is used to store computer programs that can be run on the first processor 3103;
[0811] The first processor 3103 is configured to, when running the computer program, execute:
[0812] Determine the reference information of the current block;
[0813] Filtering part of the reference information of the current block to determine the target information of the current block;
[0814] Determining a weighting coefficient according to a reference sample value of the first color component in the target information;
[0815] Determining a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information;
[0816] Determine a predicted difference value of a second color component sampling point of the current block according to the predicted block.
[0817] It is understood that the first memory 3102 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 3102 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0818] The first processor 3103 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 3103. The above-mentioned first processor 3103 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented as a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 3102 , and the first processor 3103 reads the information in the first memory 3102 and completes the steps of the above method in combination with its hardware.
[0819] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0820] Optionally, as another embodiment, the first processor 3103 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.
[0821] This embodiment provides an encoding device, which may further include the encoding apparatus 300 described in the preceding embodiment. The encoding device improves the accuracy of reference information for adjacent regions by fully considering the characteristics of input color format information. Furthermore, spatial correlation is fully utilized to perform filtering processes such as upsampling, downsampling, scaling, and scaling on the reference information for adjacent regions and the reconstructed luminance information for the current block. This not only improves the accuracy of chrominance prediction, reduces bitrate, and enhances encoding and decoding performance.
[0822] Based on the same inventive concept as the above embodiment, refer to FIG22 , which shows a schematic diagram of the structure of a decoding device 320 provided in an embodiment of the present application. As shown in FIG22 , the decoding device 320 may include: a second determination unit 3201, a second filtering unit 3202, and a second prediction unit 3203; wherein,
[0823] The second determining unit 3201 is configured to determine reference information of the current block;
[0824] The second filtering unit 3202 is configured to filter part of the reference information of the current block to determine the target information of the current block;
[0825] The second determining unit 3201 is further configured to determine a weighting coefficient according to the reference sample value of the first color component in the target information;
[0826] The second prediction unit 3203 is configured to determine a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information;
[0827] The second determining unit 3201 is further configured to determine the reconstructed value of the second color component sampling point of the current block according to the prediction block.
[0828] In some embodiments, the prediction block includes predicted values of at least some of the second color component sampling points in the current block.
[0829] In some embodiments, the second determination unit 3201 is further configured to determine the reference information of the current block based on the value of the first color component sampling point in the adjacent area of the current block, the value of the second color component sampling point in the adjacent area of the current block, and the reconstructed value of the first reference color component sampling point in the current block.
[0830] In some embodiments, the neighboring region of the current block may include at least one of the following: an upper neighboring region, an upper right neighboring region, a left neighboring region, and a lower left neighboring region.
[0831] In some embodiments, the second filtering unit 3202 is further configured to perform a first filtering process on the values of the first color component sampling points in the adjacent area of the current block to obtain the filtered sample values of the first color component in the adjacent area of the current block; and determine the target information of the current block based on the filtered sample values of the first color component in the adjacent area of the current block.
[0832] In some embodiments, the number of filtered sample values of the first color component in the adjacent area of the current block is equal to L, where L represents the number of values of the second color component sampling points in the adjacent area of the current block, and L is a positive integer.
[0833] In some embodiments, the first filtering process is a downsampling filtering process.
[0834] In some embodiments, the second filtering unit 3202 is further configured to perform a first filtering process on the values of the first color component sampling points in the adjacent area of the current block based on the color format information to obtain filtered sample values of the first color component in the adjacent area of the current block.
[0835] In some embodiments, the second filtering unit 3202 is further configured to perform downsampling filtering on the values of the first color component sampling points in the adjacent area of the current block if the color format information indicates 4:2:0 sampling; wherein the downsampling rate is the reciprocal of a positive integer multiple of 2.
[0836] In some embodiments, the second filtering unit 3202 is further configured to compare L with a preset number; and based on the comparison result of L with the preset number, perform a second filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; and determine the target information of the current block based on the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0837] In some embodiments, the second filtering unit 3202 is further configured to, if L is greater than a preset number, perform a first sub-filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; if L is less than the preset number, perform a second sub-filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0838] In some embodiments, the second filtering unit 3202 is further configured to perform a third filtering process on the values of the second color component sampling points in the adjacent area of the current block to obtain the filtered sample values of the second color component in the adjacent area of the current block; and determine the target information of the current block based on the filtered sample values of the second color component in the adjacent area of the current block.
[0839] In some embodiments, the number of filtered sample values of the second color component in the adjacent area of the current block is equal to K, where K represents the number of values of the first color component sampling points in the adjacent area of the current block, and K is a positive integer.
[0840] In some embodiments, the third filtering process is an upsampling filtering process.
[0841] In some embodiments, the second filtering unit 3202 is further configured to perform a third filtering process on the values of the second color component sampling points in the adjacent area of the current block based on the color format information to obtain filtered sample values of the second color component in the adjacent area of the current block.
[0842] In some embodiments, the second filtering unit 3202 is further configured to perform upsampling filtering on the values of the second color component sampling points in the adjacent area of the current block if the color format information indicates 4:2:0 sampling; wherein the upsampling rate is a positive integer multiple of 2.
[0843] In some embodiments, the second filtering unit 3202 is further configured to compare K with a preset number; and based on the comparison result of K with the preset number, perform a fourth filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered sample values of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; and determine the target information of the current block based on the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0844] In some embodiments, the second filtering unit 3202 is further configured to, if K is greater than a preset number, perform a first sub-filtering process on the values of the first color component sampling points in the adjacent areas of the current block and the filtered sample values of the second color component in the adjacent areas of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; if K is less than a preset number, perform a second sub-filtering process on the values of the first color component sampling points in the adjacent areas of the current block and the filtered sample values of the second color component in the adjacent areas of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
[0845] In some embodiments, the second filtering unit 3202 is further configured to determine that the number of values of the first color component sampling points in the adjacent area of the current block is a first number; and based on the comparison result of the first number and the preset number, perform a fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block; and determine the target information of the current block based on the filtered adjacent sample values of the first color component of the current block.
[0846] In some embodiments, the second filtering unit 3202 is further configured to, if the first number is greater than a preset number, perform a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block; if the first number is less than the preset number, perform a second sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block.
[0847] In some embodiments, the second filtering unit 3202 is further configured to determine that the number of values of the second color component sampling points in the adjacent area of the current block is a second number; and based on the comparison result of the second number and the preset number, perform a sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block; and determine the target information of the current block based on the filtered adjacent sample values of the second color component of the current block.
[0848] In some embodiments, the second filtering unit 3202 is further configured to, if the second number is greater than a preset number, perform a first sub-filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block; if the second number is less than the preset number, perform a second sub-filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block.
[0849] In some embodiments, the second filtering unit 3202 is further configured to set the reference sample value of the second color component in the target information as the filtered adjacent sample value of the second color component of the current block.
[0850] In some embodiments, the first sub-filtering process is a downsampling filtering process.
[0851] In some embodiments, the first filtering unit 3002 is further configured to perform a first sub-filtering process of at least one of the following:
[0852] The sub-filtering process is performed by selecting fixed interval positions;
[0853] The sub-filtering process is performed by selecting continuous positions.
[0854] In some embodiments, the second sub-filtering process is an upsampling filtering process.
[0855] In some embodiments, the first filtering unit 3002 is further configured to perform at least one of the following second sub-filtering processes: performing sub-filtering processes using an interpolation method at fixed intervals.
[0856] In some embodiments, the second filtering unit 3202 is further configured to perform a seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block; and determine the target information of the current block based on the filtered sample value of the first reference color component sampling point in the current block.
[0857] In some embodiments, the second filtering unit 3202 is further configured to determine the first width and the first height based on the reconstructed value of the first reference color component sampling point in the current block; and if the first width is greater than the width of the prediction block, or the first height is greater than the height of the prediction block, then performing a third sub-filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block; if the first width is less than the width of the prediction block, or the first height is less than the height of the prediction block, then performing a fourth sub-filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block.
[0858] In some embodiments, the third sub-filtering process is a downsampling filtering process.
[0859] In some embodiments, the first filtering unit 3002 is further configured to perform a third sub-filtering process of at least one of the following:
[0860] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction;
[0861] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction;
[0862] Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction;
[0863] The reconstructed value of the first reference color component sampling point in the current block is downsampled and filtered in the vertical direction and then downsampled and filtered in the horizontal direction.
[0864] In some embodiments, the fourth sub-filtering process is an upsampling filtering process.
[0865] In some embodiments, the second filtering unit 3202 is further configured to perform a fourth sub-filtering process of at least one of the following:
[0866] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction;
[0867] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction;
[0868] Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction;
[0869] The reconstructed value of the first reference color component sampling point in the current block is subjected to up-sampling filtering in the vertical direction and then to up-sampling filtering in the horizontal direction.
[0870] In some embodiments, the second filtering unit 3202 is further configured to set the reference sample value of the first color component in the target information to the absolute value of the difference between the value of the first color component sampling point in the adjacent area of the current block and the reconstructed value of the first reference color component sampling point in the current block.
[0871] In some embodiments, the second filtering unit 3202 is further configured to set the reference sample value of the first color component in the target information to the absolute value of the difference between the filtered adjacent sample value of the first color component of the current block and the filtered sample value of the first reference color component sampling point in the current block.
[0872] In some embodiments, the second determining unit 3201 is further configured to determine a value corresponding to the reference sample value of the first color component under a preset mapping relationship; and set the weighting coefficient to be equal to the value.
[0873] In some embodiments, the second determination unit 3201 is further configured to determine the first factor; determine the first product value based on the first factor and the reference sample value of the first color component; and determine the value corresponding to the first product value under a preset mapping relationship.
[0874] In some embodiments, the first factor is a preset constant value.
[0875] In some embodiments, the second determining unit 3201 is further configured to determine the value of the first factor according to a size parameter of the current block; wherein the size parameter of the current block includes at least one of the following parameters: the width of the current block, the height of the current block.
[0876] In some embodiments, the preset mapping relationship is a Softmax function.
[0877] In some embodiments, the preset mapping relationship is a weighting function that has an inverse proportional relationship with the reference sample value of the first color component.
[0878] In some embodiments, the second prediction unit 3203 is further configured to determine the weighted value of the reference sample value of the second color component and the corresponding weighting coefficient; and set the predicted value of the second color component sampling point in the prediction block to be equal to the sum of N weighted values; wherein N represents the number of reference sample values of the second color component, and N is a positive integer.
[0879] In some embodiments, the second prediction unit 3203 is further configured to perform a correction operation on the predicted value of the second color component sampling point in the prediction...
Claims
1. A decoding method, comprising: Determine the reference information of the current block; performing filtering processing on at least part of the reference information of the current block to determine target information of the current block; Determining a weighting coefficient according to a reference sample value of the first color component in the target information; determining a predicted block of the second color component of the current block according to the weighting coefficient and a reference sample value of the second color component in the target information; Determine, according to the prediction block, a reconstructed value of a second color component sampling point of the current block.
2. The method according to claim 1, wherein The prediction block includes predicted values of at least some of the second color component sampling points in the current block.
3. The method according to claim 1, wherein The determining of the reference information of the current block includes: Determine reference information of the current block according to a value of a first color component sampling point in an adjacent area of the current block, a value of a second color component sampling point in an adjacent area of the current block, and a reconstructed value of a first reference color component sampling point in the current block.
4. The method according to claim 3, wherein: The adjacent area of the current block includes at least one of the following: an upper adjacent area, an upper right adjacent area, a left adjacent area, and a lower left adjacent area.
5. The method according to claim 3, wherein The filtering of at least part of the reference information of the current block to determine the target information of the current block includes: Performing a first filtering process on the values of the first color component sampling points in the adjacent area of the current block to obtain filtered sample values of the first color component in the adjacent area of the current block; The target information of the current block is determined according to the filtered sample values of the first color component in the adjacent area of the current block.
6. The method according to claim 5, wherein: The number of filtered sample values of the first color component in the adjacent area of the current block is equal to L, where L represents the number of values of the second color component sampling points in the adjacent area of the current block, and L is a positive integer.
7. The method according to claim 5, wherein: The first filtering process is a downsampling filtering process.
8. The method according to claim 5, wherein The method further comprises: Based on the color format information, a first filtering process is performed on the values of the first color component sampling points in the adjacent area of the current block to obtain filtered sample values of the first color component in the adjacent area of the current block.
9. The method according to claim 8, wherein The first filtering process further includes: If the color format information indicates 4:2:0 sampling, downsampling filtering is performed on the values of the first color component sampling points in the adjacent area of the current block; wherein the downsampling rate is the reciprocal of a positive integer multiple of 2.
10. The method according to claim 6, wherein: The method further comprises: Compare L with the preset number; performing a second filtering process on the filtered samples of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block according to a comparison result of L with a preset number, to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block; Target information of the current block is determined according to filtered adjacent samples of a first color component of the current block and filtered adjacent samples of a second color component of the current block.
11. The method according to claim 10, wherein: The performing a second filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block, comprises: If L is greater than the preset number, performing a first sub-filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; If L is less than the preset number, a second sub-filtering process is performed on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
12. The method according to claim 3, wherein: The filtering of at least part of the reference information of the current block to determine the target information of the current block includes: Performing a third filtering process on the values of the second color component sampling points in the adjacent area of the current block to obtain filtered sample values of the second color component in the adjacent area of the current block; The target information of the current block is determined according to the filtered sample values of the second color component in the adjacent area of the current block.
13. The method according to claim 12, wherein: The number of filtered sample values of the second color component in the adjacent area of the current block is equal to K, where K represents the number of values of the first color component sampling points in the adjacent area of the current block, and K is a positive integer.
14. The method according to claim 12, wherein: The third filtering process is an upsampling filtering process.
15. The method according to claim 12, wherein: The method further comprises: Based on the color format information, a third filtering process is performed on the values of the second color component sampling points in the adjacent area of the current block to obtain filtered sample values of the second color component in the adjacent area of the current block.
16. The method according to claim 15, wherein The third filtering process further includes: If the color format information indicates 4:2:0 sampling, upsampling filtering is performed on the values of the second color component sampling points in the adjacent area of the current block; wherein the upsampling rate is a positive integer multiple of 2.
17. The method according to claim 13, wherein: The method further comprises: Compare K with the preset number; performing a fourth filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block according to a comparison result of K with a preset number, to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block; Target information of the current block is determined according to filtered adjacent samples of a first color component of the current block and filtered adjacent samples of a second color component of the current block.
18. The method according to claim 17, wherein The performing fourth filtering on the values of the first color component sampling points in the adjacent area of the current block and the filtered sample values of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block includes: If K is greater than the preset number, performing a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block; If K is less than the preset number, a second sub-filtering process is performed on the values of the first color component sampling points in the adjacent area of the current block and the filtered sample values of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
19. The method according to claim 3, wherein: The filtering of at least part of the reference information of the current block to determine the target information of the current block includes: Determine that the number of values of the first color component sampling points in the adjacent area of the current block is a first number; performing a fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block according to a comparison result of the first number and a preset number, to determine filtered adjacent sample values of the first color component of the current block; Target information of the current block is determined according to filtered adjacent samples of a first color component of the current block.
20. The method according to claim 19, wherein The performing a fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block includes: If the first number is greater than the preset number, performing a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the first color component of the current block; If the first number is smaller than the preset number, a second sub-filtering process is performed on the values of the first color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the first color component of the current block.
21. The method according to claim 3, wherein The filtering of at least part of the reference information of the current block to determine the target information of the current block further includes: Determine that the number of values of the second color component sampling points in the adjacent area of the current block is a second number; performing a sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block according to a comparison result of the second number and a preset number, to determine filtered adjacent sample values of the second color component of the current block; Target information of the current block is determined according to filtered adjacent samples of the second color component of the current block.
22. The method according to claim 21, wherein The performing a sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block includes: If the second number is greater than the preset number, performing a first sub-filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the second color component of the current block; If the second number is smaller than the preset number, a second sub-filtering process is performed on the values of the second color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the second color component of the current block.
23. The method of claim 10, 17 or 21, wherein: The method further comprises: The reference sample value of the second color component in the target information is set as the filtered adjacent sample value of the second color component of the current block.
24. The method of claim 11, 18, 20 or 22, wherein: The first sub-filtering process is a downsampling filtering process.
25. The method of claim 11, 18, 20 or 22, wherein The first sub-filtering process includes at least one of the following: The sub-filtering process is performed by selecting fixed interval positions; The sub-filtering process is performed by selecting continuous positions.
26. The method of claim 11, 18, 20 or 22, wherein: The second sub-filtering process is an upsampling filtering process.
27. The method of claim 11, 18, 20 or 22, wherein The second sub-filtering process includes: The sub-filtering process is performed using an interpolation method with fixed interval positions.
28. The method according to claim 3 or 10, wherein The filtering of at least part of the reference information of the current block to determine the target information of the current block includes: performing a seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine a filtered sample value of the first reference color component sampling point in the current block; Determine target information of the current block according to the filtered sample value of the first reference color component sampling point in the current block.
29. The method according to claim 28, wherein The performing a seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block includes: Determining a first width and a first height according to a reconstructed value of a first reference color component sampling point in the current block; If the first width is greater than the width of the prediction block, or the first height is greater than the height of the prediction block, performing a third sub-filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine a filtered sample value of the first reference color component sampling point in the current block; If the first width is smaller than the width of the prediction block, or the first height is smaller than the height of the prediction block, a fourth sub-filtering process is performed on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block.
30. The method according to claim 29, wherein The third sub-filtering process is a downsampling filtering process.
31. The method according to claim 30, wherein The third sub-filtering process includes at least one of the following: Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction; Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction; Performing downsampling filtering processing on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction; The reconstructed value of the first reference color component sampling point in the current block is downsampled and filtered in the vertical direction and then downsampled and filtered in the horizontal direction.
32. The method of claim 29, wherein: The fourth sub-filtering process is an upsampling filtering process.
33. The method according to claim 32, wherein The fourth sub-filtering process includes at least one of the following: Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction; Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction; Performing upsampling filtering processing on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction; The reconstructed value of the first reference color component sampling point in the current block is subjected to up-sampling filtering in the vertical direction and then to up-sampling filtering in the horizontal direction.
34. The method of claim 28, wherein The method further comprises: The reference sample value of the first color component in the target information is set to the absolute value of the difference between the value of the first color component sampling point in the adjacent area of the current block and the reconstructed value of the first reference color component sampling point in the current block.
35. The method of claim 28, wherein The method further comprises: The reference sample value of the first color component in the target information is set to the absolute value of the difference between the filtered adjacent sample value of the first color component of the current block and the filtered sample value of the first reference color component sampling point in the current block.
36. The method of claim 1, wherein The determining of the weighting coefficient according to the reference sample value of the first color component in the target information includes: Determining a value corresponding to the reference sample value of the first color component under a preset mapping relationship; The weighting coefficient is set equal to the value.
37. The method according to claim 36, wherein The determining of a value corresponding to the reference sample value of the first color component under a preset mapping relationship includes: Determine the first factor; determining a first product value according to the first factor and a reference sample value of the first color component; Determine a value corresponding to the first product value under the preset mapping relationship.
38. The method of claim 37, wherein: The determining of the first factor comprises: The first factor is a preset constant value.
39. The method of claim 37, wherein: The determining of the first factor comprises: Determining a value of the first factor according to a size parameter of the current block; The size parameter of the current block includes at least one of the following parameters: the width of the current block and the height of the current block.
40. The method of claim 36, wherein The determining of a value corresponding to the reference sample value of the first color component under a preset mapping relationship includes: The preset mapping relationship is a Softmax function.
41. The method of claim 36, wherein: The determining of a value corresponding to the reference sample value of the first color component under a preset mapping relationship includes: The preset mapping relationship is a weighting function that has an inverse proportional relationship with the reference sample value of the first color component.
42. The method of claim 1, wherein The determining, according to the weighting coefficient and the reference sample value of the second color component in the target information, a predicted block of the second color component of the current block, comprises: Determining a weighted value of a reference sample value of the second color component and the corresponding weighting coefficient; The predicted value of the second color component sampling point in the prediction block is set to be equal to the sum of N weighted values; wherein N represents the number of reference sample values of the second color component, and N is a positive integer.
43. The method according to claim 42, wherein The method further comprises: A correction operation is performed on the predicted value of the second color component sampling point in the prediction block to determine the predicted block of the second color component of the current block.
44. The method of claim 1, wherein The method further comprises: Perform filtering and enhancement processing on the prediction block, and use the processed prediction block as the prediction block of the second color component of the current block.
45. The method of claim 1, wherein The method further comprises: determining a compensation value for a second color component of the prediction block according to reference sample values in an adjacent area of the current block; The predicted value of the second color component sampling point in the predicted block is corrected according to the compensation value to determine the predicted block of the second color component of the current block.
46. The method of claim 1, wherein The method further comprises: performing prediction processing on the second color component sampling point in the prediction block according to at least one prediction mode to determine at least one initial prediction value of the second color component sampling point in the prediction block; A weighted fusion process is performed on the at least one initial prediction value and the prediction value of the second color component sampling point in the prediction block to determine a prediction block for the second color component of the current block.
47. The method of claim 1, wherein The determining, according to the prediction block, a reconstructed value of a second color component sampling point of the current block, comprises: Determining a predicted difference value of a second color component sampling point of the current block; Determining, according to the prediction block, a predicted value of a second color component sampling point of the current block; Determine a reconstructed value of the second color component sampling point of the current block according to the predicted difference value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.
48. A coding method comprising: Determine the reference information of the current block; performing filtering processing on at least part of the reference information of the current block to determine target information of the current block; Determining a weighting coefficient according to a reference sample value of the first color component in the target information; determining a predicted block of the second color component of the current block according to the weighting coefficient and a reference sample value of the second color component in the target information; Determine, according to the prediction block, a prediction difference value of a second color component sampling point of the current block.
49. The method according to claim 48, wherein The prediction block includes predicted values of at least some of the second color component sampling points in the current block.
50. The method of claim 48, wherein The determining of the reference information of the current block includes: Determine reference information of the current block according to a value of a first color component sampling point in an adjacent area of the current block, a value of a second color component sampling point in an adjacent area of the current block, and a reconstructed value of a first reference color component sampling point in the current block.
51. The method of claim 50, wherein: The adjacent area of the current block includes at least one of the following: an upper adjacent area, an upper right adjacent area, a left adjacent area, and a lower left adjacent area.
52. The method of claim 50, wherein: The filtering of at least part of the reference information of the current block to determine the target information of the current block includes: Performing a first filtering process on the values of the first color component sampling points in the adjacent area of the current block to obtain filtered sample values of the first color component in the adjacent area of the current block; The target information of the current block is determined according to the filtered sample values of the first color component in the adjacent area of the current block.
53. The method of claim 52, wherein: The number of filtered sample values of the first color component in the adjacent area of the current block is equal to L, where L represents the number of values of the second color component sampling points in the adjacent area of the current block, and L is a positive integer.
54. The method of claim 52, wherein: The first filtering process is a downsampling filtering process.
55. The method of claim 52, wherein: The method further comprises: Based on the color format information, a first filtering process is performed on the values of the first color component sampling points in the adjacent area of the current block to obtain filtered sample values of the first color component in the adjacent area of the current block.
56. The method of claim 55, wherein: The first filtering process further includes: If the color format information indicates 4:2:0 sampling, downsampling filtering is performed on the values of the first color component sampling points in the adjacent area of the current block; wherein the downsampling rate is the reciprocal of a positive integer multiple of 2.
57. The method of claim 53, wherein: The method further comprises: Compare L with the preset number; performing a second filtering process on the filtered samples of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block according to a comparison result of L with a preset number, to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block; Target information of the current block is determined according to filtered adjacent samples of a first color component of the current block and filtered adjacent samples of a second color component of the current block.
58. The method of claim 57, wherein The performing a second filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block, comprises: If L is greater than the preset number, performing a first sub-filtering process on the filtered sample values of the first color component in the adjacent area of the current block and the values of the sampling points of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block; If L is less than the preset number, a second sub-filtering process is performed on the filtered sample values of the first color component in the adjacent area of the current block and the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
59. The method of claim 50, wherein: The filtering of at least part of the reference information of the current block to determine the target information of the current block includes: Performing a third filtering process on the values of the second color component sampling points in the adjacent area of the current block to obtain filtered sample values of the second color component in the adjacent area of the current block; The target information of the current block is determined according to the filtered sample values of the second color component in the adjacent area of the current block.
60. The method of claim 59, wherein The number of filtered sample values of the second color component in the adjacent area of the current block is equal to K, where K represents the number of values of the first color component sampling points in the adjacent area of the current block, and K is a positive integer.
61. The method of claim 59, wherein: The third filtering process is an upsampling filtering process.
62. The method of claim 59, wherein: The method further comprises: Based on the color format information, a third filtering process is performed on the values of the second color component sampling points in the adjacent area of the current block to obtain filtered sample values of the second color component in the adjacent area of the current block.
63. The method of claim 62, wherein: The third filtering process further includes: If the color format information indicates 4:2:0 sampling, upsampling filtering is performed on the values of the second color component sampling points in the adjacent area of the current block; wherein the upsampling rate is a positive integer multiple of 2.
64. The method of claim 60, wherein: The method further comprises: Compare K with the preset number; performing a fourth filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block according to a comparison result of K with a preset number, to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block; Target information of the current block is determined according to filtered adjacent samples of a first color component of the current block and filtered adjacent samples of a second color component of the current block.
65. The method of claim 64, wherein The performing fourth filtering on the values of the first color component sampling points in the adjacent area of the current block and the filtered sample values of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block includes: If K is greater than the preset number, performing a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block and the filtered samples of the second color component in the adjacent area of the current block to determine the filtered adjacent samples of the first color component of the current block and the filtered adjacent samples of the second color component of the current block; If K is less than the preset number, a second sub-filtering process is performed on the values of the first color component sampling points in the adjacent area of the current block and the filtered sample values of the second color component in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block and the filtered adjacent sample values of the second color component of the current block.
66. The method of claim 49, wherein The filtering of at least part of the reference information of the current block to determine the target information of the current block includes: Determine that the number of values of the first color component sampling points in the adjacent area of the current block is a first number; performing a fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block according to a comparison result of the first number and a preset number, to determine filtered adjacent sample values of the first color component of the current block; Target information of the current block is determined according to filtered adjacent samples of a first color component of the current block.
67. The method of claim 66, wherein The performing a fifth filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the first color component of the current block includes: If the first number is greater than the preset number, performing a first sub-filtering process on the values of the first color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the first color component of the current block; If the first number is smaller than the preset number, a second sub-filtering process is performed on the values of the first color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the first color component of the current block.
68. The method of claim 50, wherein The filtering of at least part of the reference information of the current block to determine the target information of the current block further includes: Determine that the number of values of the second color component sampling points in the adjacent area of the current block is a second number; performing a sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block according to a comparison result of the second number and a preset number, to determine filtered adjacent sample values of the second color component of the current block; Target information of the current block is determined according to filtered adjacent samples of the second color component of the current block.
69. The method of claim 68, wherein The performing a sixth filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine the filtered adjacent sample values of the second color component of the current block includes: If the second number is greater than the preset number, performing a first sub-filtering process on the values of the second color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the second color component of the current block; If the second number is smaller than the preset number, a second sub-filtering process is performed on the values of the second color component sampling points in the adjacent area of the current block to determine filtered adjacent sample values of the second color component of the current block.
70. The method of claim 57, 64 or 68, wherein The method further comprises: The reference sample value of the second color component in the target information is set as the filtered adjacent sample value of the second color component of the current block.
71. The method of claim 58, 65, 67 or 69, wherein The first sub-filtering process is a downsampling filtering process.
72. The method of claim 58, 65, 67 or 69, wherein The first sub-filtering process includes at least one of the following: The sub-filtering process is performed by selecting fixed interval positions; The sub-filtering process is performed by selecting continuous positions.
73. The method of claim 58, 65, 67 or 69, wherein The second sub-filtering process is an upsampling filtering process.
74. The method of claim 58, 65, 67 or 69, wherein The second sub-filtering process includes: The sub-filtering process is performed using an interpolation method with fixed interval positions.
75. The method of claim 50 or 57, wherein: The filtering of at least part of the reference information of the current block to determine the target information of the current block includes: performing a seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine a filtered sample value of the first reference color component sampling point in the current block; Determine target information of the current block according to the filtered sample value of the first reference color component sampling point in the current block.
76. The method of claim 75, wherein The performing a seventh filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block includes: Determining a first width and a first height according to a reconstructed value of a first reference color component sampling point in the current block; If the first width is greater than the width of the prediction block, or the first height is greater than the height of the prediction block, performing a third sub-filtering process on the reconstructed value of the first reference color component sampling point in the current block to determine a filtered sample value of the first reference color component sampling point in the current block; If the first width is smaller than the width of the prediction block, or the first height is smaller than the height of the prediction block, a fourth sub-filtering process is performed on the reconstructed value of the first reference color component sampling point in the current block to determine the filtered sample value of the first reference color component sampling point in the current block.
77. The method of claim 76, wherein The third sub-filtering process is a downsampling filtering process.
78. The method of claim 77, wherein The third sub-filtering process includes at least one of the following: Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction; Performing downsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction; Performing downsampling filtering processing on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction; The reconstructed value of the first reference color component sampling point in the current block is downsampled and filtered in the vertical direction and then downsampled and filtered in the horizontal direction.
79. The method of claim 76, wherein The fourth sub-filtering process is an upsampling filtering process.
80. The method of claim 79, wherein The fourth sub-filtering process includes at least one of the following: Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction; Performing upsampling filtering on the reconstructed value of the first reference color component sampling point in the current block in the vertical direction; Performing upsampling filtering processing on the reconstructed value of the first reference color component sampling point in the current block in the horizontal direction and then in the vertical direction; The reconstructed value of the first reference color component sampling point in the current block is subjected to up-sampling filtering in the vertical direction and then to up-sampling filtering in the horizontal direction.
81. The method of claim 75, wherein The method further comprises: The reference sample value of the first color component in the target information is set to the absolute value of the difference between the value of the first color component sampling point in the adjacent area of the current block and the reconstructed value of the first reference color component sampling point in the current block.
82. The method of claim 75, wherein The method further comprises: The reference sample value of the first color component in the target information is set to the absolute value of the difference between the filtered adjacent sample value of the first color component of the current block and the filtered sample value of the first reference color component sampling point in the current block.
83. The method of claim 48, wherein The determining of the weighting coefficient according to the reference sample value of the first color component in the target information includes: Determining a value corresponding to the reference sample value of the first color component under a preset mapping relationship; The weighting coefficient is set equal to the value.
84. The method of claim 83, wherein The determining of a value corresponding to the reference sample value of the first color component under a preset mapping relationship includes: Determine the first factor; determining a first product value according to the first factor and a reference sample value of the first color component; Determine a value corresponding to the first product value under the preset mapping relationship.
85. The method of claim 84, wherein The determining of the first factor comprises: The first factor is a preset constant value.
86. The method of claim 84, wherein The determining of the first factor comprises: Determining a value of the first factor according to a size parameter of the current block; The size parameter of the current block includes at least one of the following parameters: the width of the current block and the height of the current block.
87. The method of claim 83, wherein The determining of a value corresponding to the reference sample value of the first color component under a preset mapping relationship includes: The preset mapping relationship is a Softmax function.
88. The method of claim 83, wherein The determining of a value corresponding to the reference sample value of the first color component under a preset mapping relationship includes: The preset mapping relationship is a weighting function that has an inverse proportional relationship with the reference sample value of the first color component.
89. The method of claim 48, wherein The determining, according to the weighting coefficient and the reference sample value of the second color component in the target information, a predicted block of the second color component of the current block, comprises: Determining a weighted value of a reference sample value of the second color component and the corresponding weighting coefficient; The predicted value of the second color component sampling point in the prediction block is set to be equal to the sum of N weighted values; wherein N represents the number of reference sample values of the second color component, and N is a positive integer.
90. The method of claim 89, wherein The method further comprises: A correction operation is performed on the predicted value of the second color component sampling point in the prediction block to determine the predicted block of the second color component of the current block.
91. The method of claim 48, wherein The method further comprises: Perform filtering and enhancement processing on the prediction block, and use the processed prediction block as the prediction block of the second color component of the current block.
92. The method of claim 48, wherein The method further comprises: determining a compensation value for a second color component of the prediction block according to reference sample values in an adjacent area of the current block; The predicted value of the second color component sampling point in the predicted block is corrected according to the compensation value to determine the predicted block of the second color component of the current block.
93. The method of claim 48, wherein The method further comprises: performing prediction processing on the second color component sampling point in the prediction block according to at least one prediction mode to determine at least one initial prediction value of the second color component sampling point in the prediction block; A weighted fusion process is performed on the at least one initial prediction value and the prediction value of the second color component sampling point in the prediction block to determine a prediction block for the second color component of the current block.
94. The method of claim 48, wherein The determining, according to the prediction block, a predicted difference value of a second color component sampling point of the current block, comprises: Determining, according to the prediction block, a predicted value of a second color component sampling point of the current block; Determine a predicted difference value of the second color component sampling point of the current block according to the original value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.
95. The method of claim 94, wherein The method further comprises: The predicted difference value of the second color component sampling point of the current block is encoded, and the obtained encoding bits are written into a bit stream.
96. A coding device comprising a first determining unit, a first filtering unit, and a first predicting unit; wherein, The first determining unit is configured to determine reference information of the current block; The first filtering unit is configured to perform filtering processing on at least part of the reference information of the current block to determine target information of the current block; The first determining unit is further configured to determine a weighting coefficient based on a reference sample value of the first color component in the target information; The first prediction unit is configured to determine a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information; The first determining unit is further configured to determine a predicted difference value of a second color component sampling point of the current block according to the predicted block.
97. A coding device comprising a first memory and a first processor; wherein, The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 48 to 95 when running the computer program.
98. A decoding device comprising a second determining unit, a second filtering unit, and a second prediction unit; wherein: The second determining unit is configured to determine reference information of the current block; The second filtering unit is configured to perform filtering processing on at least part of the reference information of the current block to determine target information of the current block; The second determining unit is further configured to determine a weighting coefficient according to a reference sample value of the first color component in the target information; The second prediction unit is configured to determine a prediction block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component in the target information; The second determining unit is further configured to determine a reconstructed value of a second color component sampling point of the current block according to the predicted block.
99. A decoding device comprising a second memory and a second processor; wherein, The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 47 when running the computer program.
100. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 47 is implemented, or the method according to any one of claims 48 to 95 is implemented.
