Image or video processing method, coder, decoder and coding / decoding system

Abstract

Disclosed in the present invention are an image or video processing method, a coder, a decoder and a coding / decoding system. The method comprises: acquiring a first candidate parameter list, wherein the first candidate parameter list is a candidate parameter list of a cross-component prediction merge mode CCPMERGE, the first candidate parameter list comprises a plurality of candidate parameters, and at least one of the plurality of candidate parameters includes a parameter for predictive boosting filtering (PBF); and on the basis of a selected candidate parameter, predicting chroma information of a current block in an image so as to generate a chroma prediction block. The present invention can simultaneously retain the respective advantages of CCPMERGE and PBF, thereby improving the coding and decoding performance.

Description

Image or video processing method, encoder, decoder and encoding / decoding system Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image or video processing method, an encoder, a decoder, and a coding and decoding system. Background Art

[0002] In image and video compression, an image or frame of video typically consists of three color components: luma (Y) and two chroma components (Cb and Cr). Each component is represented as a data matrix. The data matrix for each component is divided into blocks associated with specific coding parameters. For any block on a particular component, which can be a square or rectangle with a side length that is a power of 2, there is a unique corresponding block in the other two components, located at the same spatial location. The encoder operates according to a specific encoding order: luma is processed first, followed by chroma components; processing begins in the upper left corner and proceeds from left to right and top to bottom.

[0003] In video coding standards such as Versatile Video Coding (VVC), intra-frame prediction is a method of using previously coded blocks within the same image to predict the current block. When performing intra-frame prediction on the current block, the encoder tries a variety of preset intra-frame prediction modes, generating a prediction block using the reconstructed values ​​from previously coded blocks. These prediction blocks are then compared to select the best prediction mode. The difference between the current block and the best prediction block (residual) is also encoded. By signaling the best prediction mode and the residual, the decoder can reconstruct the same reconstructed image or video as the encoder.

[0004] When encoding the chrominance components, the luma components in the same image have already been encoded, and the reconstructed luma block can be used as a reference sample for encoding the chrominance components. Therefore, when encoding the current block of the chrominance components, the current block of the chrominance components can be predicted by analyzing the relationship between the luma components and the chrominance components in nearby encoded blocks and applying this relationship to the current block of the luma components. This method is usually called cross-component prediction (CCP). However, the existing CCP method still has many shortcomings and needs to be improved.

[0005] Summary of the Invention

[0006] The present invention provides an image or video processing method, wherein the method includes: obtaining a first candidate parameter list, wherein the first candidate parameter list is a candidate parameter list of a cross-component prediction fusion mode CCPMERGE, the first candidate parameter list includes multiple candidate parameters, and at least one of the multiple candidate parameters has a parameter of a prediction enhancement filter (PBF); and predicting chrominance information of a current block in an image based on the selected candidate parameter to generate a chrominance prediction block.

[0007] Optionally, the method further includes:

[0008] The candidate parameters in the first candidate parameter list are reordered according to certain rules to obtain a second candidate parameter list,

[0009] The step of generating a chroma prediction block comprises:

[0010] The chroma prediction block is generated according to the candidate parameters selected from the second candidate parameter list.

[0011] Optionally, the method further includes:

[0012] Truncating the first candidate parameter list or the second candidate parameter list to obtain a third candidate parameter list;

[0013] The step of generating a chroma prediction block comprises:

[0014] The chroma prediction block is generated according to the candidate parameters selected from the third candidate parameter list.

[0015] Optionally, the method further includes:

[0016] reordering the candidate parameters in the first candidate parameter list according to a certain rule, and truncating the reordered candidate parameter list; or

[0017] The first candidate parameter list is truncated, and the truncated candidate parameter list is reordered according to a certain rule.

[0018] Optionally, the first candidate parameter list includes identification information of the prediction enhancement filter, and the identification information is used to identify whether one or more candidate parameters in the first candidate parameter list carry parameters of the prediction enhancement filter or support the prediction enhancement filter.

[0019] Optionally, the method further includes:

[0020] Determining whether a candidate parameter in the first candidate parameter list carries parameters for the prediction enhancement filtering or supports the prediction enhancement filtering, wherein the determining step is based on at least one of the following:

[0021] an encoding mode of the candidate parameters;

[0022] The coding cost of the candidate parameter after combining the prediction enhancement filtering in the cross-component prediction fusion mode; and

[0023] Statistical characteristics of the predicted block.

[0024] Optionally, if the encoding mode of the candidate parameters is a multi-model cross-component linear model MM-CCLM or a multi-model convolutional cross-component model MM-CCCM, it means that it is necessary to determine whether the candidate parameters support the prediction enhancement filtering; otherwise, it is not necessary to determine whether the candidate parameters support the prediction enhancement filtering.

[0025] Optionally, when the coding cost of the candidate parameter is less than one or more thresholds, the candidate parameter does not use the prediction enhancement filtering; and / or when the coding cost of the candidate parameter is in different threshold intervals, different prediction enhancement filtering is introduced.

[0026] Optionally, the coding cost is calculated based on the sum of absolute differences between a result of attempting to reconstruct a chrominance image reference area obtained using the candidate parameters and then adding the prediction enhancement filtering and an actually reconstructed chrominance image reference area.

[0027] Optionally, when the statistical characteristics of the prediction block are less than one or more thresholds, the candidate parameters do not use the prediction enhancement filter; and / or when the statistical characteristics of the prediction block are in different threshold intervals, different prediction enhancement filters are introduced.

[0028] Optionally, in the process of reordering the candidate parameters in the first candidate parameter list, the candidate parameters are uniformly ordered according to the coding costs corresponding to the candidate parameters without using the prediction enhancement filtering and with using the prediction enhancement filtering.

[0029] Optionally, in the process of reordering the candidate parameters in the first candidate parameter list, the candidate parameters are divided into different candidate parameter sets, and different sorting rules are selected for different candidate parameter sets, and finally the candidate parameters of these candidate parameter sets are merged and sorted.

[0030] Optionally, in the process of truncating the first candidate parameter list or the second candidate parameter list, it is determined whether to put the candidate parameters in the first candidate parameter list or the second candidate parameter list into the third candidate parameter list based on a coding cost threshold.

[0031] Optionally, in the process of truncating the first candidate parameter list or the second candidate parameter list, it is determined whether to put the candidate parameters in the first candidate parameter list or the second candidate parameter list into the third candidate parameter list based on the length of the parameter list.

[0032] Optionally, the selected candidate parameters carry parameters of the prediction enhancement filter PBF, a chroma prediction result is generated based on the selected candidate parameters, and the prediction enhancement filter is applied to the chroma prediction result to generate the chroma prediction block.

[0033] Optionally, the selected candidate parameters carry parameters of the prediction enhancement filter PBF, the prediction enhancement filter is applied to the luminance block, and the chrominance prediction block is generated by calculation based on the luminance block to which the prediction enhancement filter is applied and the selected candidate parameters.

[0034] The present invention further provides an image or video processing method, wherein the image or video processing method comprises:

[0035] Get the statistics of the current prediction block;

[0036] updating a filter threshold based on the statistical information and a previously determined filter threshold;

[0037] selecting a prediction enhancement filter based on the updated filter threshold; and

[0038] The image is reconstructed using the selected prediction enhancement filter.

[0039] Optionally, the statistical information includes a degree of change of the current prediction block in a horizontal direction, a vertical direction, a diagonal direction and / or a counter-diagonal direction.

[0040] Optionally, the filter threshold is dynamically adjusted based on an exponential sliding average method.

[0041] Optionally, in the process of selecting the prediction enhancement filter, the statistical information of the current prediction block is compared with the updated filter threshold, and the prediction enhancement filter is selected according to the comparison result.

[0042] Optionally, in the process of selecting the prediction enhancement filter, the prediction enhancement filter is selected from at least one of the following directional filters: horizontal direction, vertical direction, diagonal direction, anti-diagonal direction and non-directional filter.

[0043] The present invention also provides an encoder, comprising a processor, wherein the processor is configured to execute instructions to implement the method described above.

[0044] The present invention also provides a decoder, comprising a processor, wherein the processor is configured to execute instructions to implement the method described above.

[0045] The present invention also provides a coding and decoding system, which includes a coding device and a decoding device, the coding device includes a first processor, the first processor is used to execute instructions to implement the above method, and the decoding device includes a second processor, the second processor is used to execute instructions to implement the above method.

[0046] The beneficial effect of the present invention is that in the image or video processing method proposed by the present invention, the candidate parameters in the candidate parameter list of the cross-component prediction fusion mode CCPMERGE include parameters of the prediction enhancement filter PBF. Therefore, during the execution of CCPMERGE, the prediction enhancement filter PBF can be used for reconstruction, thereby improving the deficiency of the prior art in which only cross-component prediction without LBCCP is performed when predicting the current block during the execution of CCPMERGE. Therefore, the present invention can simultaneously maintain the respective advantages of CCPMERGE and the prediction enhancement filter PBF, produce a certain gain in the encoding and decoding process, and can further improve the encoding and decoding performance. In addition, the prediction enhancement filter can use a filter implemented according to the present invention, which can produce a better gain for the prediction.

[0047] On the other hand, the present invention uses a prediction enhancement filter to reconstruct an image. Specifically, the image or video processing method of the present invention includes obtaining statistical information of a current prediction block; updating a filter threshold based on the statistical information and a previously determined filter threshold; selecting a prediction enhancement filter based on the updated filter threshold; and reconstructing the image using the selected prediction enhancement filter. Compared to the prior art that uses a single-intensity, isotropic low-pass filter (e.g., LBCCP), the embodiments of the present invention utilize statistical information of the prediction block (e.g., features such as texture intensity and direction) to select a more suitable filter, thereby producing a better gain for the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0049] FIG1A is a schematic diagram of a video encoding system according to an embodiment of the present invention.

[0050] FIG1B is a schematic diagram of a video decoding system according to an embodiment of the present invention.

[0051] FIG2 is a schematic diagram of three color components in a color image or video frame according to an embodiment of the present invention.

[0052] FIG3 is a schematic diagram of luminance components and chrominance components in a color image or video frame according to an embodiment of the present invention.

[0053] FIG4 is a schematic diagram showing the local boosting of cross-component prediction (LBCCP).

[0054] FIG5 shows the decoding process related to cross-component prediction in the prior art.

[0055] FIG6 is a flowchart of an image or video processing method according to an embodiment of the present invention.

[0056] FIG7 is a flowchart of an image or video processing method according to another embodiment of the present invention.

[0057] FIG8 is a flowchart of an image or video processing method according to yet another embodiment of the present invention.

[0058] FIG9 is a flowchart of a cross-component prediction fusion process and a cross-component prediction process according to an embodiment of the present invention.

[0059] FIG. 10 is a schematic diagram showing that the CCPMERGE parameter includes a flag indicating whether PBF is supported in an embodiment of the present invention.

[0060] FIG11 is a schematic diagram of a unified list parameter sorting rule according to an embodiment of the present invention.

[0061] FIG12 is a schematic diagram of independent list parameter sorting rules according to an embodiment of the present invention.

[0062] FIG13 is a schematic diagram of an operation of truncating a CCPMERGE parameter list according to an embodiment of the present invention.

[0063] FIG14 is a flowchart of another image or video processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to more clearly illustrate the embodiments of the present disclosure or related technologies, a brief overview of several embodiments will be introduced below. Obviously, these illustrations are only some embodiments of the present disclosure, and people with ordinary skills can derive other illustrations based on these illustrations.

[0065] In order to better understand the technical solutions proposed in the embodiments of the present invention, it is best to start with a brief introduction to several basic concepts: video encoding and decoding systems.

[0066] Embodiments of the present invention relate to a video coding system. As shown in FIG1A , the video coding system 110 includes multiple units: a block segmentation unit 1101, a transform and quantization unit 1102, an intra-frame estimation unit 1103, an intra-frame prediction unit 1104, a motion compensation unit 1105, a motion estimation unit 1106, an inverse transform and inverse quantization unit 1107, a filter control analysis unit 1108, a loop filter unit 1109, a coding unit 1110, an image buffer unit 1111, and a subtractor 1112.

[0067] Starting with the original video signal, the image frame is divided into coding tree blocks (CTUs, hereinafter referred to as "blocks") by the block segmentation unit 1101. For each current block, the subtractor 1112 subtracts the prediction block output by the intra prediction unit 1103 or the motion compensation unit 1104 from the current block to generate residual pixel information, resulting in a residual block. The transform and quantization unit 1102 transforms and quantizes the residual pixel information obtained through intra prediction or inter prediction (motion compensation). The transform converts the residual pixel information from the pixel domain to the transform domain, and then quantizes the resulting transform coefficients to further reduce the bit rate. The intra estimation unit 1103 performs intra estimation, and the intra prediction unit 1104 performs intra prediction on the video reconstructed block. Motion estimation, performed by the motion estimation unit 1106, generates a motion vector that estimates the motion of the video reconstructed block. The motion compensation unit 1105 then performs motion compensation based on the determined motion vector. After determining the intra prediction mode, the intra prediction unit 1104 provides the selected intra prediction data to the encoding unit 1110. The motion estimation unit 1106 sends the calculated motion vector data to the encoding unit 1110. The inverse transform and inverse quantization unit 1107 reconstructs the video reconstruction block and the reconstructed residual block in the pixel domain. The loop filter unit 1109 is controlled by the filter control and analysis unit 1108 to eliminate artifacts in the reconstructed block. The encoding unit 1110 adds the reconstructed residual block to the prediction block in the coded image buffer unit 1111 to generate a reconstructed block. The encoding unit 1110 is used to encode various coding parameters and quantized transform coefficients (quantized transform coefficients) into a bitstream and output the bitstream of the video signal. The coded image buffer unit 1111 is used to store the reconstructed block as a reference block. As video image encoding progresses, new reconstructed blocks are continuously generated and stored in the coded image buffer unit 1111.

[0068] An embodiment of the present invention provides a video decoding system. As shown in FIG1B , the video decoding system 120 comprises multiple units: a decoding unit 1201, an inverse transform and inverse quantization unit 1202, an intra-frame prediction unit 1203, a motion compensation unit 1204, a loop filter unit 1205, a decoded image buffer unit 1206, and a post-filter unit 1207.

[0069] The video encoding system 110 encodes a video frame to obtain an output bitstream. The video encoding system 110 transmits the bitstream to the video decoding system 120. The video decoding system 120 receives a bitstream representing a video frame in an encoded format (i.e., a compressed format). In the video decoding system 120, the decoding unit 1201 processes the bitstream to obtain decoded transform coefficients. The inverse transform and dequantization unit 1202 processes the transform coefficients to generate a residual block in the pixel domain. The intra-frame prediction unit 1203 generates an intra-frame prediction block for the current video decoding block based on a determined intra-frame prediction mode and previously decoded block data for the current video decoding block. The motion compensation unit 1204 determines inter-frame prediction information for the current video decoding block and generates an inter-frame prediction block by parsing motion vectors and other relevant syntax elements. Finally, the decoded video block is formed by adding the residual block generated by the inverse transform and dequantization unit 1202 to the corresponding prediction block generated by the intra-frame prediction unit 1203 or the motion compensation unit 1204. To improve video quality, the decoded video blocks are filtered by filtering unit 1205 to remove artifacts. The decoded video blocks are then stored in decoded image buffer 1206, serving as reference blocks for subsequent intra-frame prediction or motion compensation, and reconstructing the video signal. The reconstructed video signal can then be optionally further processed by post-filtering unit 1207 to achieve a more suitable or enhanced visual experience.

[0070] Cross-component prediction mode

[0071] The data structure of a color image is shown below. As shown in Figure 2, a color image or color video frame typically consists of three color components: luma component Y 301 and two chroma components, Cb 302 and Cr 303. Each component is represented as a data matrix. The data matrix for each component is decomposed into blocks associated with specific coding parameters. For example, block 304 of luma component Y, block 305 of chroma component Cb, and block 306 of chroma component Cr all correspond to spatial locations 307 in the color image. In video coding, the luma component is typically processed first, followed by the chroma components.

[0072] The luminance component of a color image or video frame has a higher resolution in both the vertical and horizontal directions than any of the chrominance components, with twice as many pixels in each direction as the chrominance components. To reflect the spatial correspondence between the luminance and chrominance components during the intra-frame prediction process, the luminance component is downsampled during the intra-frame prediction process to match the resolution of the chrominance components, so that all components have an equal number of pixels in the vertical and horizontal directions.

[0073] When encoding chrominance components, the luma component in the same image has already been encoded, and the reconstructed luma block can serve as a reference sample for encoding the chrominance components. Therefore, when encoding the current chrominance block, the current chrominance block can be predicted by analyzing the relationship between the luma and chrominance components in nearby encoded blocks and applying this relationship to the current luma block. This method is commonly known as cross-component prediction (CCP). The principle of cross-component intra-frame prediction is to use the reconstructed image region of one color component to predict the image region of another color component in the same image. As shown in Figure 3, a current block 405 of any chrominance component (either Cb or Cr) is about to be encoded. At this time, a current block 401 of the luma component Y corresponding to the same spatial position as the current block 405 has already been encoded and reconstructed. Reference blocks 402, 403, and 404 corresponding to the current block 401 also correspond to the same spatial position as reference blocks 406, 407, and 408 corresponding to the current block 405 and have already been encoded and reconstructed. Reference blocks 406, 407, and 408 may constitute a reference region for the current block 405. Alternatively, a subset of reference blocks 406, 407, and 408 may constitute a reference region for the current block 405. The principle of cross-component prediction is to calculate the relationship between reference blocks 402, 403, and 404 of the luma component and reference blocks 406, 407, and 408 of the chroma components, and apply the relationship to the current block 401 of the luma component to calculate the predicted value of the chroma component of the current block 405.

[0074] When intra-coding chrominance images, a cross-component linear model (CCLM) can be used as one of the prediction modes. CCLM uses a reconstructed luma image (or luma block) to predict the chrominance image (or chroma block), achieving significant compression gains, especially when luma and chroma are highly correlated.

[0075] When using CCLM to predict Cb or Cr chrominance blocks, the following formula can be used: Pred C (x,y)=α·Rec′ L (x,y)+β

[0076] Where Pred C (x, y) is a pixel value being predicted in the Cb or Cr chroma block, Rec′ L (x, y) is the pixel value of the reconstructed luminance block after adjusting it to the same size as the chrominance block, and (x, y) is the coordinate of the pixel. In the 4:2:0 image format, because the length and width of the luminance image are both twice that of the chrominance image, Rec′ L Usually from the reconstructed brightness image Rec L Obtained after downsampling.

[0077] The parameters α and β of CCLM are obtained from the numerical relationship between the luminance image reference area and the chrominance image reference area, and the calculation method is: β=M(t C )-αM(t L )

[0078] where t L is a pixel value in the reference area of ​​the luminance image, t C A pixel value in the reference region of the chroma image currently being encoded, (x, y) is the coordinate of the pixel, and M(X) represents the average value of sample X in the corresponding reference region.

[0079] In another embodiment, when CCLM is used to predict the Cb or Cr chrominance block, the following formula may be used:

[0080] Where α0 and β0 represent the use of only the pixel values ​​in the reference area of ​​the brightness image that are not less than M(t L ) are obtained for the subset consisting of the pixels in the luminance image, and α1 and β1 represent the α and β obtained for the subset consisting of the remaining pixels in the reference region of the luminance image. This method is called a multi-model cross-component linear model (Multi-Model CCLM, hereinafter referred to as MM-CCLM).

[0081] When intra-coding chrominance images, the Convolutional Cross-Component Model (CCCM) can also be used as a prediction mode. CCCM uses the reconstructed luma image (or luma block) to predict the chroma image (or chroma block), achieving significant compression gains, especially when luma and chroma are highly correlated.

[0082] When using CCCM to predict Cb or Cr chrominance blocks, the following formula can be used: Pred C (x,y)=c0·Rec′ L (x,y)+c1·Rec′ L (x-1,y)+c2·Rec′ L (x+1,y)+c3 ·Rec′ L (x,y-1)+c4·Rec′ L (x,y+1)+c5·[Rec′ L (x,y)] 2 +c6·2 D-1

[0083] Where D is the color depth of the image.

[0084] The CCCM parameter set c = {c0, c1, ..., c6} is obtained as follows:

[0085] Where R represents the set of all coordinates included in the corresponding chrominance image reference area, Rec C (x, y) is the pixel value of the reconstructed chrominance image. The numerical optimization solution can be obtained by Gaussian elimination or other approximate algorithms.

[0086] Similar to the above MM-CCLM, the pixel value in the reference area of ​​the brightness image is not less than M(t L ) to obtain the parameter group c G ={c G0 ,c G1 ,…,c G6}, obtain the parameter group c for the subset of pixels remaining in the reference area of ​​the luminance image L ={c L0 ,c L1 ,…,c L6}, when predicting Cb or Cr chrominance blocks, the following formula can be used:

[0087] When Rec′ L (x,y)≥M(t L ), Pred C (x,y)=c G0 ·Rec′ L (x,y)+c G1 Rec′ L (x-1,y)+c G2 ·Rec′ L (x+1,y) +c G3 Rec′ L (x,y-1)+c G4 Rec′ L (x,y+1)+c G5 ·[Rec′ L (x,y)] 2 +c G6 2 D-1

[0088] When Rec′ L (x,y) <M(t L ), Pred C (x,y)=c L0 ·Rec′ L (x,y)+c L1 Rec′ L (x-1,y)+c L2 ·Rec′L (x+1,y) +c L3 ·Rec′ L (x,y-1)+c L4 Rec′ L (x,y+1)+c L5 ·[Rec′ L (x,y)] 2 +c L6 2 D-1

[0089] This method is called Multi-Model Convolutional Cross-Component Model (Multi-Model CCCM, hereinafter referred to as MM-CCCM).

[0090] Cross-component prediction of local enhancement

[0091] After using MM-CCLM or MM-CCCM to predict the chrominance image, the generated predicted chrominance block can be locally enhanced through cross-component prediction (Local-boosting Cross-Component Prediction, hereinafter referred to as LBCCP) to form a new predicted chrominance block. This method applies a two-dimensional filter to the original predicted chrominance block to form a new predicted block. As shown in Figure 4, the filter calculation method is: Pred′ C (x,y) = [Pred C (x-1,y-1)+Pred C (x,y-1)+Pred C (x+1,y-1)+Pred C (x-1,y)+8×Pred C (x,y)+Pred C (x+1,y)+Pred C (x-1,y+1)+Pred C (x,y+1)+Pred C (x+1,y+1)+8]>>4

[0092] where Pred′ C is the predicted value after filtering, and >> is the right shift operation.

[0093] Cross-component prediction fusion mode

[0094] In addition to directly deducing the numerical relationship between nearby luminance and chrominance from the reference area near the current block, there is also a fusion mode (Merge Mode) technology. The main principle of the fusion mode is to find blocks that have used the same prediction mode from the previously encoded data, obtain their mode parameters, and derive a fusion list. The decoder needs to reproduce the encoder's fusion list construction process, and can obtain a complete set of mode parameters from an index number. The fusion mode is applied to cross-component prediction to form a cross-component prediction fusion mode (CCPMERGE), that is, to find blocks that use cross-component prediction modes from historical encoded data, and combine the corresponding parameters of each block into a parameter list. The decoder can obtain the prediction mode and parameters of the current block from an index number.

[0095] FIG5 shows a decoding process related to cross-component prediction in the prior art. In the prior art, as shown in FIG5 , if the cross-component prediction fusion mode (CCPMERGE) is used, the following steps are performed when performing the CCPMERGE decoding process:

[0096] -Derivation of a list of candidate parameters for cross-component prediction from the reconstructed image and a preset configuration

[0097] - Reorder the above cross-component prediction candidate parameter list

[0098] - Get the index of the candidate parameters for cross-component prediction

[0099] - Generate chroma prediction block using the selected candidate parameters

[0100] If cross-component prediction mode (CCP) is used, the following steps are performed when performing the CCP decoding process:

[0101] -Decoding cross-component prediction modes

[0102] - If the prediction mode is MM-CCLM or MM-CCCM, further determine whether to use LBCCP during decoding

[0103] -Use the corresponding prediction method to generate the chroma prediction block. If LBCCP is used, the chroma prediction block is filtered to form the final chroma prediction block.

[0104] In the related decoding process of the prior art shown in Figure 5, CCPMERGE does not consider LBCCP during execution. That is, if the use of CCPMERGE is detected during the decoding process, the entire CCP decoding process, including LBCCP, is immediately skipped. Even if the block pointed to by the CCPMERGE index, which previously used cross-component prediction, actually uses the LBCCP-enhanced MM-CCLM / MM-CCCM model, only the MM-CCLM / MM-CCCM without LBCCP is still executed when predicting the current block. In principle, there is no conflict between LBCCP and CCPMERGE. The former strengthens the connection between adjacent pixels within the predicted block and eliminates local noise, while the latter searches for blocks similar to the current block from the encoded data. The two do not conflict or overlap in the predictive coding gains, and their combined use should result in additive gains.

[0105] Furthermore, in the related decoding process of the prior art shown in Figure 5, LBCCP uses a single-intensity, unidirectional low-pass filter. Single-intensity filtering may not be able to adapt to the filtering requirements of images with different texture intensities; unidirectional filtering may destroy the original texture direction of the image while smoothing the predicted block, which has the negative effect of hindering coding efficiency. Therefore, for specific images, it is often necessary to consider more optimal filter coefficients. Leveraging the local statistical characteristics of cross-component predictions, especially those of texture intensity and direction, to generate adaptive filter selection can achieve better prediction gains.

[0106] Embodiments of the present invention are primarily used in the intra-frame prediction unit 1104 of the video encoding system 110 and the intra-frame prediction unit 1203 of the video decoding system 120. If the intra-frame prediction method provided by the embodiments of the present application can achieve better prediction results, the quality of video decoding and reconstruction can also be improved. The video decoding system 120 receives a bitstream representing a video frame. Therefore, the intra-frame prediction unit 1203 of the video decoding system 120 can obtain reference luminance blocks, reference chrominance blocks, and the current luminance block to predict the current chrominance block. The intra-frame prediction unit 1203 can perform the same intra-frame prediction method as the intra-frame prediction unit 1103.

[0107] An embodiment of the present invention provides an image or video processing method, which is applied to an image or video encoding or decoding device. The functions implemented by this method can be implemented by a processor in the image or video encoding or decoding device calling program code. Of course, the program code can be stored in a computer storage medium (e.g., a non-volatile computer storage medium). As can be seen, the image or video encoding or decoding device includes a processor and a storage medium.

[0108] FIG6 is a flowchart of an image or video processing method according to an embodiment of the present invention. The image or video processing method may be a cross-component intra-frame prediction method, which uses a reconstructed reference block within the same image or video frame to predict the current block. As shown in FIG6 , the method includes the following steps S110 to S120:

[0109] Step S110: Obtain a first candidate parameter list, wherein the first candidate parameter list is a candidate parameter list for the cross-component prediction fusion mode CCPMERGE, and the first candidate parameter list includes multiple candidate parameters, at least one of the multiple candidate parameters includes a parameter of a prediction enhancement filter (PBF).

[0110] In this step, a first candidate parameter list is obtained, which is a candidate parameter list for the cross-component prediction fusion mode CCPMERGE mentioned above. For example, if a reconstructed image block adopts the multi-model cross-component linear model MM-CCLM encoding method, the candidate parameters in the first candidate parameter list may include the parameters α0, β0, α1 and β1 mentioned above; if a reconstructed image block adopts the multi-model convolution cross-component model MM-CCCM encoding method, the candidate parameters in the first candidate parameter list may include the parameters c mentioned above. G0 ,c G1 ,…,c G6 and c L0 ,c L1 ,…,c L6 The first candidate parameter list may include multiple sets of parameters (e.g., 12 sets of parameters) corresponding to multiple reconstructed image blocks using the cross-component prediction CCP method as the candidate parameters. In the first candidate parameter list, each set of candidate parameters corresponds to an index, and each index indicates a set of candidate parameters.

[0111] Furthermore, at least one of the multiple candidate parameters in the first candidate parameter list includes a prediction boosting filter (PBF) parameter. That is, some candidate parameters in the first candidate parameter list include PBF parameters, while some candidate parameters do not use or support PBF. Alternatively, all candidate parameters in the first candidate parameter list include PBF parameters, or can use or support PBF.

[0112] The aforementioned predictive boosting filter can employ the cross-component predictive local boosting (LBCCP) filter mentioned above, or a new filter developed according to the present invention that takes into account statistical characteristics of image blocks (such as texture and directional characteristics) (described in detail later). The present invention does not impose any particular restrictions on the type, size, or characteristics of the predictive boosting filter. A set of candidate parameters containing predictive boosting filter (PBF) parameters can indicate that predictive boosting filtering was used during image reconstruction using this set of candidate parameters to obtain the final reconstructed image.

[0113] Step S120: predicting the chrominance information of a current block in an image according to the selected candidate parameters to generate a chrominance prediction block.

[0114] As mentioned above, the principle of cross-component prediction is to calculate the relationship between the reference block of the luminance component and the reference block of the chrominance component, and apply it to the current block of the luminance component to calculate the predicted value of the chrominance component of the current block. The cross-component prediction fusion mode is to find a block using the cross-component prediction mode from the historical coding data, and apply the parameters of the prediction mode used by the block to the luminance component of the current block to predict the chrominance component of the current block. In other words, the chrominance information of the current block is predicted to generate a chrominance prediction block.

[0115] In this step, the selected candidate parameters are used to generate the chroma prediction block. The selected candidate parameters may be the candidate parameters with the prediction enhancement filter (PBF) in step S110, that is, the prediction enhancement filter was used in the process of generating the chroma prediction block. The selected candidate parameters may also be the candidate parameters that do not use or do not support the prediction enhancement filter (PBF), that is, the prediction enhancement filter was not used in the process of generating the chroma prediction block.

[0116] The selected candidate parameters may be candidate parameters in the first candidate parameter list that can generate the best chroma prediction block. The best chroma prediction block may be a chroma prediction block that has the smallest difference from the current chroma block. In other words, the residual between the best chroma prediction block and the current chroma block can achieve the best coding efficiency.

[0117] In the image or video processing method proposed in the present invention, the candidate parameters in the candidate parameter list of the cross-component prediction fusion mode CCPMERGE include parameters of the prediction enhancement filter PBF. Therefore, during the execution of CCPMERGE, the prediction enhancement filter PBF can be used for reconstruction, thereby improving the deficiency of the prior art in which only cross-component prediction without LBCCP is performed when predicting the current block during the execution of CCPMERGE. Therefore, the present invention can simultaneously maintain the respective advantages of CCPMERGE and the prediction enhancement filter PBF, generate certain gains in the encoding and decoding process, and further improve encoding and decoding performance. In addition, the prediction enhancement filter can use a filter implemented according to the present invention, which can produce better gains in prediction.

[0118] In some embodiments, the image or video processing method of the present invention further includes the following steps:

[0119] Step S130: reorder the candidate parameters in the first candidate parameter list according to certain rules to obtain a second candidate parameter list.

[0120] In this step, the candidate parameters in the first candidate parameter list are reordered to obtain a second candidate parameter list. After reordering, the candidate parameters in the first candidate parameter list may be the same as or different from the candidate parameters in the second candidate parameter list. The second candidate parameter list may include only part of the candidate parameters in the first candidate parameter list, or may include candidate parameters that do not appear in the first candidate parameter list. In addition, the candidate parameters may be sorted according to the coding cost corresponding to the candidate parameters, and the coding cost may be calculated based on the absolute difference between the chrominance image reference area attempted to be reconstructed using the candidate parameters and the chrominance image reference area actually reconstructed (detailed description later). If the candidate parameter uses a prediction enhancement filter, then when calculating the coding parameter of the candidate parameter, the chrominance image reference area attempted to be reconstructed obtained by the candidate parameter and the absolute difference between the result after the prediction enhancement filter and the chrominance image reference area actually reconstructed may be considered.

[0121] Because the candidate parameters in the first candidate parameter list have varying probabilities of producing the optimal reconstructed image, their index numbers are encoded using variable-length coding, such as truncated unary codes. Reordering the candidate parameters allows those with a higher probability of producing the optimal reconstructed image to receive index numbers corresponding to shorter code lengths, thereby reducing code length. For each current block reconstructed using the cross-component prediction merging mode CCPMERGE, candidate parameter reordering is performed to reduce the code length required to encode the index numbers, achieving higher coding efficiency.

[0122] The step S120 of generating a chroma prediction block includes:

[0123] Step S121: Generate the chroma prediction block according to the candidate parameters selected from the second candidate parameter list

[0124] In this step, the reordered candidate parameters, ie, the candidate parameters selected in the second candidate parameter list, can be used to generate a chroma prediction block. The process of generating a chroma prediction block using the candidate parameters is as described above and will not be repeated here.

[0125] In some embodiments, the image or video processing method of the present invention further includes the following steps:

[0126] Step S140: truncate the first candidate parameter list or the second candidate parameter list to obtain a third candidate parameter list.

[0127] In this step, the first candidate parameter list or the second candidate parameter list can be truncated to obtain a third candidate parameter list. That is, the number of candidate parameters in the third candidate parameter list is less than the number of candidate parameters in the first candidate parameter list or the second candidate parameter list. Specifically, whether to place the candidate parameters in the first candidate parameter list or the second candidate parameter list into the third candidate parameter list can be determined based on the coding cost threshold or the parameter list length. Candidate parameters in the first candidate parameter list or the second candidate parameter list that are lower than the coding cost threshold can be placed into the third candidate parameter list, or, only a certain number of candidate parameters are maintained in the candidate parameter list, and this number may depend on the set parameter list length.

[0128] Truncating the candidate parameter list can reduce the number of candidate parameters that need to be calculated, reduce the system's computing load during the encoding and decoding process, improve encoding and decoding efficiency, and also reduce the storage load of storing candidate parameters and reduce the requirements for system storage capacity.

[0129] The step S120 of generating a chroma prediction block includes:

[0130] Step S122: Generate the chroma prediction block according to the candidate parameters selected from the third candidate parameter list.

[0131] In this step, the reordered candidate parameters, ie, the candidate parameters selected in the third candidate parameter list, can be used to generate a chroma prediction block. The process of generating a chroma prediction block using the candidate parameters is as described above and will not be repeated here.

[0132] In some embodiments, as shown in FIG7 , the candidate parameters in the first candidate parameter list may be reordered according to certain rules (step S130′), and then the reordered candidate parameter list may be truncated (step S140′). In other embodiments, as shown in FIG8 , the first candidate parameter list may be truncated (step S140″), and then the truncated candidate parameter list may be reordered according to certain rules (step S130″).

[0133] In some embodiments, the cross-component prediction fusion mode list includes candidate parameters with PBF, and all candidate parameters in the cross-component prediction fusion mode list may be reordered according to certain rules. In other embodiments, the cross-component prediction fusion mode list includes candidate parameters with PBF and candidate parameters without PBF, and all candidate parameters in the cross-component prediction fusion mode list may be reordered according to certain rules. In still other embodiments, the cross-component prediction fusion mode list only includes candidate parameters without PBF, and all candidate parameters in the cross-component prediction fusion mode list may be reordered according to certain rules, and in subsequent steps, prediction enhancement filtering may be added to certain reordered candidate parameters.

[0134] Figure 9 is a flow chart illustrating the cross-component prediction fusion process and the cross-component prediction process in an embodiment of the present invention. As shown in Figure 9, during the cross-component prediction fusion (CCPMERGE) process, a prediction enhancement filter (PBF) processing operation is added. The cross-component prediction candidate parameter list is reordered based on the prediction coding costs before and after the addition of the PBF. If the selected candidate parameter includes a prediction enhancement filter, the prediction enhancement filter is used during the generation of the chrominance prediction block. Alternatively, the selected prediction enhancement filter can be derived from the statistical information of the prediction block. During the cross-component prediction CCP process, the prediction mode used can be determined by parameter content, such as MM-CCLM or MM-CCCM. If a prediction enhancement filter is used, the prediction enhancement filter is used during the generation of the chrominance prediction block. Alternatively, the selected prediction enhancement filter can be derived from the statistical information of the prediction block.

[0135] The following is a detailed description of the marking process of the prediction enhancement filter PBF:

[0136] In CCPMERGE mode, two implementation methods are provided to increase support for predictive filtering (PBF): direct marking and inferred extension. Direct marking directly indicates whether a candidate parameter in the CCPMERGE candidate parameter list supports PBF, while inferred extension infers whether a candidate parameter in the CCPMERGE candidate parameter list supports PBF. In specific implementations, a combination of direct marking and inferred extension is also possible. Direct marking and inferred extension are described in detail below.

[0137] Direct marking

[0138] In some embodiments, for the case of direct marking, the first candidate parameter list includes identification information of the prediction enhancement filter, and the identification information is used to identify whether one or more candidate parameters in the first candidate parameter list contain parameters of the prediction enhancement filter or support the prediction enhancement filter. The description is as follows:

[0139] In this solution, whether one or more parameter flags in the CCPMERGE candidate parameter list support predictive boost filtering (PBF) may be implemented as shown in the following examples. Other implementations with the same meaning are also included in the solution of the present invention and will not be described in detail.

[0140] 1) The CCPMERGE parameter contains a flag indicating whether PBF is supported: If the flag is "Use PBF," PBF operations are introduced during the CCPMERGE calculation process; if the flag is "Do not use PBF," PBF operations are not introduced during the CCPMERGE calculation process. For example, the flag can be 0 or 1, with 1 indicating "Use PBF" and 0 indicating "Do not use PBF," as shown in Figure 10. Alternatively, the flag can indicate that PBF is supported when the CCPMERGE parameter contains the PBF parameter; otherwise, PBF is not supported. In addition, the encoder needs to write the candidate parameter index and the signaling of whether PBF is used into the bitstream.

[0141] 2) Marking whether multiple parameters in a CCPMERGE parameter list support PBF: Multiple CCPMERGE parameters share a single PBF flag. For example, a CCPMERGE parameter list corresponds to one PBF flag. If the flag is valid, it indicates that the candidate parameters in the CCPMERGE parameter list support PBF. Otherwise, it does not. Furthermore, the encoder needs to write the candidate parameter index and the signaling of whether PBF is used into the bitstream.

[0142] Derivation and Extension

[0143] In some embodiments, for the case of deriving extensions, it can be based on at least one of the following: the coding mode of the candidate parameter; the coding cost of the candidate parameter after combining the cross-component prediction fusion mode and the prediction enhancement filtering; and the statistical characteristics of the prediction block, to determine whether the candidate parameter in the first candidate parameter list carries the parameters of the prediction enhancement filtering or supports the prediction enhancement filtering. The description is as follows:

[0144] In this solution, according to certain rules, it is obtained whether one or more parameters in the CCPMERGE parameter list support and use the PBF parameters, and it is illustrated with the following examples. Other implementation manners with the same meaning are also included in the solution of the present invention and will not be elaborated repeatedly:

[0145] 1) Rule 1, derived from the CCPMERGE parameter content: If the coding mode of the candidate parameter in the CCPMERGE list is a specific mode, it is necessary to determine whether the candidate parameter in this list supports the PBF operation, otherwise it does not support. For example, if the candidate parameter in this CCPMERGE list is the multi-model cross-component linear model MM-CCLM or the multi-model convolutional cross-component model MM-CCCM, it is necessary to determine whether the candidate parameter supports the PBF operation. If the candidate parameter in this list is not MM-CCLM or MM-CCCM, it does not include the PBF operation. To determine whether the candidate parameter supports the PBF operation, it can be derived according to the coding cost and other methods, which will not be further elaborated here.

[0146] 2) Rule 2, derived from the coding cost of the CCPMERGE parameter: When the coding cost of the CCPMERGE parameter reaches a certain threshold, the PBF operation should be included, otherwise the PBF operation is not included. The thresholds described here may include one or more, and the PBF operations described here may include various different types of PBF, without limitation. The coding cost of the CCPMERGE parameter can be calculated based on the absolute difference sum between the chrominance image reference region of the attempted reconstruction obtained by using this CCPMERGE parameter and the actual reconstructed chrominance image reference region (to be detailed later). Specifically, if the coding cost of a candidate parameter is less than one or more thresholds, the candidate parameter does not use the said prediction enhancement filtering. When the coding cost of the candidate parameter is in different threshold intervals, different prediction enhancement filterings are introduced. For example, as shown in Table 1, c cand represents the coding cost corresponding to the CCPMERGE candidate parameter cand, and Th x represents the xth cost threshold, where x is a positive integer. When c cand < Th1, it means that PBF is not used; when Th x ≤ c cand < Thx, it means that the 1st to the xth PBF filterings are all included in consideration. Table 1 3) Rule 3, derived from the statistical characteristics of the prediction block: When the statistical characteristics of the prediction block reach a certain threshold, a PBF operation should be performed; otherwise, no PBF operation is performed. The threshold described here may include one or more, and the PBF operation described here may include multiple different types of PBF. The derivation method is similar to that of Rule 2 above and is not repeated here. The statistical characteristics of the prediction block can be found in the following description.

[0147] Specifically, if the statistical characteristics of the prediction block are less than one or more thresholds, the candidate parameters do not use prediction enhancement filtering. When the statistical characteristics of the prediction block are within different threshold intervals, different prediction enhancement filtering is introduced.

[0148] The following describes in detail the process of reordering candidate parameters in the CCPMERGE parameter list:

[0149] After the cross-component prediction candidate parameter list is constructed, it can be reordered to reduce the code length required for encoding the index number. The sorting method is to compare the absolute difference sum of the result of attempting to reconstruct the chrominance image reference area using each set of candidate parameters with the chrominance image reference area actually reconstructed one by one, and arrange them in ascending order according to the size of the absolute difference sum. After the PBF operation is introduced in the CCPMERGE model, the parameter list contains PBF operations or non-PBF operations. Therefore, it is necessary to define the calculation method of the above-mentioned absolute difference sum and the reordering rules for the CCPMERGE parameter list after the newly added PBF operation according to certain rules. The sorting rules exemplarily include two modes, unified list parameter sorting rules and independent list parameter sorting rules, which are explained as follows.

[0150] Unified list parameter sorting rules

[0151] In some embodiments, for the case of unified list parameter sorting, in the process of reordering the candidate parameters in the CCPMERGE parameter list, the candidate parameters are uniformly sorted according to the coding costs corresponding to the candidate parameters without and with prediction enhancement filtering. The explanation is as follows:

[0152] In this solution, the encoding costs corresponding to the CCPMERGE candidate parameters without and with PBF are uniformly sorted, and a new parameter list containing both parameters with and without PBF is obtained. The implementation method includes the following steps.

[0153] Step 1: When the candidate parameter involved is marked as using PBF, add a filter to the result of the attempted reconstruction of the chrominance image reference area and then calculate the absolute difference between the result and the actual reconstructed chrominance image reference area as the coding cost of the candidate parameter. The calculation method of the filter is: Rec″C (x,y)=[Rec′ C (x-1,y-1)+Rec′ C (x,y-1)+Rec′ C (x+1,y-1) +Rec′ C (x-1,y)+8×Rec′ C (x,y)+Rec′ C (x+1,y)+Rec′ C (x-1,y+1)+Rec′ C (x,y+1)+Rec′ C (x+1,y+1)+8]>>4

[0154] Where Rec C and Rec′ C The pixel values ​​of the chrominance image reference area after and before filtering are respectively attempted to be reconstructed, and (x, y) is the coordinate of the pixel, where the first value is the horizontal coordinate and the second value is the vertical coordinate. The exceptions are as follows:

[0155] - If the pixel above the pixel of the reference area being reconstructed does not belong to the reference area, then all y-1 vertical coordinate values ​​in the above formula are changed to y;

[0156] - If the pixel below the pixel of the reference area being reconstructed does not belong to the reference area, then all y+1 vertical coordinate values ​​in the above formula are changed to y;

[0157] - If the pixel to the left of the pixel being reconstructed does not belong to the reference area, then all x-1 values ​​in the above equations are changed to x;

[0158] If the pixel to the right of the pixel attempting to reconstruct the reference region does not belong to the reference region, all x+1 values ​​in the above equations are changed to x.

[0159] The absolute difference between the chrominance image reference area attempted to be reconstructed and the chrominance image reference area actually reconstructed is calculated as follows: SAD = S(|Rec C -Rec C |)

[0160] Among them, SAD is the sum of absolute differences, S(X) represents the sum of sample X in the corresponding reference area, Rec C is the pixel value of the reference area of ​​the chrominance image actually reconstructed.

[0161] As an alternative to the above step 1, the above filtering is applied to the luminance image reference area, and then the coding mode of the candidate parameters is used to try to reconstruct the chrominance image reference area, and then the absolute difference sum is calculated from the actually reconstructed chrominance image reference area.

[0162] Step 2: When the CCPMERGE candidate parameters do not use PBF, the traditional scheme is used to calculate the coding cost corresponding to the candidate parameters, such as the sum of the absolute differences between the attempted reconstructed chrominance image reference area and the actually reconstructed chrominance image reference area.

[0163] Step 3: After obtaining the coding cost corresponding to each candidate parameter (including the coding cost of candidate parameters using PBF and not using PBF), each candidate parameter is arranged as an individual in ascending order of coding cost and assigned an index number, as shown in Figure 11.

[0164] Independent list parameter sorting rules

[0165] In some embodiments, for the case of unified list parameter sorting, in the process of re-sorting the candidate parameters in the CCPMERGE parameter list, the candidate parameters are divided into different candidate parameter sets, and different sorting rules are selected for different candidate parameter sets. Finally, the candidate parameters of these candidate parameter sets are merged and sorted. The following is an explanation:

[0166] In this scheme, the CCPMERGE parameter list is divided into different candidate parameter sets according to the parameter content, and different sorting rules are selected for different candidate parameter sets. Finally, multiple different candidate parameter sets are merged and sorted to obtain a new CCPMERGE parameter list.

[0167] Regarding dividing the CCPMERGE parameter list into different candidate parameter sets based on parameter content, there are multiple implementation options, such as division based on different prediction modes or division based on whether PBF is supported. The following examples illustrate the division based on prediction modes and division based on whether PBF is supported:

[0168] 1) Divide different CCPMERGE parameter sets by prediction mode: Candidate parameters in the CCPMERGE parameter list are divided according to the prediction mode categories supported by them. Taking the MM-CCLM or MM-CCCM mode as an example, candidate parameters that use the MM-CCLM or MM-CCCM mode are divided into the first parameter set, and candidate parameters that do not use the MM-CCLM or MM-CCCM mode are divided into the second parameter set.

[0169] 2) Divide different CCPMERGE parameter sets based on PBF support: Candidate parameters in the CCPMERGE parameter list are divided based on whether they support PBF operations. Candidate parameters that support PBF operations are divided into the first parameter set, and candidate parameters that do not support PBF operations are divided into the second parameter set.

[0170] For selecting different sorting rules for different candidate parameter sets, the sorting rules include calculation methods, sorting parameter dimensions, etc. There are different ways in the specific implementation, including coding cost, or coding cost including other modes. If sorting is based on coding cost, the coding or decoding cost in the parameter set is calculated, and sorting is performed according to certain criteria of the cost. The criteria may include but are not limited to ascending, descending, etc. If sorting is based on coding cost including other modes, other operating modes need to be introduced when calculating the coding or decoding cost in the parameter set; for example, in the first parameter set mentioned above, because MM-CCLM or MM-CCCM mode is supported, when calculating the cost of the parameter, it is necessary to calculate the cost of supporting or not supporting PBF corresponding to the parameter, and then sort according to certain rules.

[0171] The following takes the example of obtaining the above-mentioned first parameter set and second parameter set by dividing the parameter sets into different CCPMERGE parameters according to the MM-CCLM or MM-CCCM mode, and describes in detail the sorting rules of the second parameter set. In this way, the first candidate parameter set and the second candidate parameter set are generated according to the second parameter set, and the second candidate parameter is marked as using PDF, and the first candidate parameter is not using PDF. For the parameters in the first candidate parameter set, the encoding cost is the sum of the absolute differences between the result of the first candidate parameter's attempt to reconstruct the chrominance image reference area and the actual reconstructed luminance image reference area. For the parameters in the second candidate parameter set, the encoding cost is the sum of the absolute differences between the result of the second candidate parameter's attempt to reconstruct the chrominance image reference area and the actual reconstructed luminance image reference area after adding a filter.

[0172] Compare the encoding cost of the parameters in the first candidate parameter set with the encoding cost of the parameters in the second candidate parameter set. If the latter is smaller, mark the parameter as "use PDF" in the candidate parameter list; otherwise, keep it marked as "do not use PDF" and use the one with the smaller encoding cost as the encoding cost of the parameter.

[0173] After calculating whether to use PBF and selecting the corresponding coding cost for all candidate parameters using the MM-CCLM or MM-CCCM mode, the candidate list can be sorted into a unified parameter list. The entire process is shown in Figure 12.

[0174] The following describes in detail the process of truncating the CCPMERGE parameter list:

[0175] For the cross-component prediction candidate parameter list after adding PBF, the CCPMERGE parameter list obtained by reordering according to certain rules needs to be truncated to maintain a limited CCPMERGE parameter list. The truncation method can be determined by at least one of the following methods:

[0176] In some embodiments, during the process of truncating the CCPMERGE parameter list, it may be determined whether to place the candidate parameters in the CCPMERGE parameter list into the truncated parameter list based on a coding cost threshold. The following description is provided:

[0177] Threshold-based determination: The length of the target parameter list is determined based on a threshold. For example, only parameters above or below a certain threshold are included in the target parameter list, while parameters below a certain threshold are not. For example, this can be determined based on the encoding or decoding cost threshold. Different parameter lists can use the same or different thresholds.

[0178] In some embodiments, when truncating a CCPMERGE parameter list, it may be determined based on the parameter list length whether to place candidate parameters in the CCPMERGE parameter list into the truncated parameter list. The following description is provided:

[0179] Determined by parameter list length: For example, the parameter list length of the CCPMERGE function after adding a PBF remains the same as the parameter list length of the CCPMERGE function without the PBF. This is either system-configured or derived from other parameters to determine the target parameter list length. The parameter list length here can be for one or more parameter lists. For example, if the parameter list length of the CCPMERGE function after adding a PBF remains the same as the parameter list length of the CCPMERGE function without the PBF, as shown in Figure 13, candidate parameters at the end of the list are discarded until the total number of candidate parameters does not exceed the configured parameter list length.

[0180] The following describes in detail the process of generating a chroma prediction block using the selected candidate parameters and prediction enhancement filtering:

[0181] Finally, after determining the final CCPMERGE parameter list, all candidate parameters in the list are traversed (i.e., each candidate parameter is used to predict the chrominance information of the current block) to generate a chrominance prediction block for the candidate parameters. If the candidate parameters use PBF, the chrominance prediction block is generated using one of the following methods.

[0182] Method 1: Apply prediction enhancement filtering to the prediction result of the chrominance information and then generate the final chrominance prediction block. Specifically, a chrominance prediction result is generated based on the candidate parameters, and prediction enhancement filtering is applied to the chrominance prediction result to generate the chrominance prediction block.

[0183] Method 2: After applying the prediction enhancement filter to the current luma block, the chroma prediction block is calculated according to the coding mode in the candidate parameters. Specifically, the prediction enhancement filter is applied to the luma block, and the chroma prediction block is generated based on the luma block with the prediction enhancement filter applied and the candidate parameters.

[0184] After calculating the coding cost of each chroma prediction block and the chroma current block one by one, the smallest one is the best cross-component prediction candidate parameter. The encoder then writes the index of the best cross-component prediction candidate parameter into the bitstream. After decoding the index, the decoder obtains the selected candidate parameter and whether it uses PBF, and generates the corresponding chroma prediction block.

[0185] FIG14 is a flowchart of another image or video processing method according to an embodiment of the present invention. This image or video processing method can be applied to, but is not limited to, cross-component prediction. This method involves deriving a corresponding prediction enhancement filter PDF from statistical information of a prediction block, which can produce a better gain for prediction and improve the quality of video decoding and reconstruction. As shown in FIG14 , the method includes the following steps S210 to S240:

[0186] Step S210: Obtain statistical information of the current prediction block.

[0187] In this step, the statistical information of the current prediction block in an image is obtained. Specifically, the statistical information may include the type, size, relative position, texture direction, texture complexity, edge strength, etc. of the prediction block. Based on the statistical information of the prediction block, the applicability of various filters can be calculated. For example, for prediction blocks with different texture complexities, filters of corresponding strengths can be selected to better capture texture information. For prediction blocks with obvious edges, direction-adaptive filters can be selected to smooth the texture direction while maintaining edge information. When calculating the horizontal change of the prediction block, the following formula can be used:

[0188] Where H and W are the pixel height and width of the current predicted chroma block, and (i, j) is the coordinate of the pixel relative to the predicted block.

[0189] When calculating the vertical change of the prediction block, the following formula can be used:

[0190] When calculating the diagonal change of the prediction block, the following formula can be used:

[0191] When calculating the anti-diagonal change of the prediction block, the following formula can be used:

[0192] The statistical information may include the degree of change of the current prediction block in the horizontal, vertical, diagonal and / or anti-diagonal directions. By obtaining the degree of change in these directions, statistical information such as texture features and edge strength features of the prediction block can be obtained.

[0193] Step S220: Update the filter threshold based on the statistical information and the previously determined filter threshold.

[0194] In this step, the statistical information of the current prediction block can be used to update the filter selection strategy, and in subsequent steps, a suitable prediction enhancement filter PBF can be selected based on the updated filter selection strategy. The filter selection strategy takes into account the thresholds corresponding to each filter, as well as the relationship between the statistical information and thresholds of the prediction block. In one embodiment, if the directional change of the prediction block is within a certain threshold interval, a filter suitable for the direction is selected; if the directional change of the prediction block is not within the given threshold, a default low-pass filter is selected. In a specific implementation process, an exponential sliding average method can be used to dynamically adjust the filter threshold based on the statistical information of the prediction block and the previously determined filter threshold. The initial value of the filter threshold is given at the beginning of the intra-frame prediction, and is dynamically updated according to the following formula during the calculation of each prediction block: Threshold(X) t =α·X t +(1-α)·Threshold(X) t-1

[0195] Where Threshold(X) t is the updated filter threshold for the current prediction block, Threshold(X) t-1 is the filter threshold before the current prediction block is updated, X t It refers to a certain type of statistical information of the current prediction block (e.g., the degree of change in the horizontal or vertical direction), and α is the forgetting factor (0<α<1).

[0196] Step S230: Selecting a prediction enhancement filter based on the updated filter threshold.

[0197] In this step, during the selection process of the prediction enhancement filter, the statistical information of the current prediction block in step S210 may be compared with the updated filter threshold in step S220 , and the prediction enhancement filter may be selected according to the comparison result.

[0198] For a block with large horizontal changes and small vertical changes, the predicted block is considered to have vertical texture information, so it is suitable to use a low-pass filter for vertical smoothing to maintain the texture intensity in the horizontal direction. For example, a block that is considered to have large horizontal changes and small vertical changes can meet the following conditions: horMTV t >Threshold(horMTV) t verMTV t <β·Threshold(verMTV) t

[0199] Among them, 0<β<1.

[0200] For a block with large vertical changes and small horizontal changes, the predicted block is considered to have texture information in the horizontal direction. Therefore, it is suitable to use a low-pass filter for smoothing in the horizontal direction to maintain the texture intensity in the vertical direction.

[0201] For a block with large changes in the diagonal direction and small changes in the anti-diagonal direction, the predicted block is considered to have texture information in the anti-diagonal direction. Therefore, a low-pass filter for diagonal smoothing is suitable to maintain the texture intensity in the anti-diagonal direction.

[0202] For a block with large anti-diagonal change and small diagonal change, the predicted block is considered to have diagonal texture information, and is therefore suitable for a low-pass filter that performs smoothing in the anti-diagonal direction to maintain the texture intensity in the diagonal direction.

[0203] Select a suitable filter for the prediction block that meets the corresponding threshold judgment conditions. The judgment conditions can be executed in sequence. After determining a certain filter, the judgment is stopped. If all threshold conditions are not met, the default non-directional filter is used.

[0204] That is, in the process of selecting the prediction enhancement filter, the prediction enhancement filter may be selected from at least one of the following directional filters: horizontal direction, vertical direction, diagonal direction, anti-diagonal direction, and non-directional filter.

[0205] Based on the selected filter type and parameters, the current prediction block is subjected to a filter enhancement operation. In this process, the filter enhancement operation uses a specific calculation method and can filter in different directions and intensities.

[0206] For example, the calculation method of horizontal filtering can be: Pred′ C (x,y)=[Pred C (x-1,y)+2×Pred C (x,y)+Pred C(x+1,y)+2]>>2

[0207] For example, the vertical filtering can be calculated as:

[0208] Pred′ C (x,y)=[Pred C (x,y-1)+2×Pred C (x,y)+Pred C (x,y+1)+2]>>2

[0209] For example, the calculation method for diagonal filtering can be: Pred′ C (x,y)=[Pred C (x-1,y-1)+6×Pred C (x,y)+Pred C (x+1,y+1)+4]>>3

[0210] For example, the calculation method for anti-diagonal filtering can be: Pred′ C (x,y)=[Pred C (x+1,y-1)+6×Pred C (x,y)+Pred C (x-1,y+1)+4]>>3

[0211] For example, the calculation method of non-directional filtering can be: Pred′ C (x,y)=[Pred C (x-1,y-1)+Pred C (x,y-1)+Pred C (x+1,y-1) +Pred C (x-1,y)+8×Pred C (x,y)+Pred C (x+1,y)+Pred C (x-1,y+1)+Pred C (x,y+1)+Pred C (x+1,y+1)+8]>>4

[0212] Step S240: reconstructing the image using the selected prediction enhancement filter.

[0213] In this step, the prediction enhancement filter selected in step S230 is used to reconstruct the image. For example, the selected prediction enhancement filter can be used in the generation process of the chrominance prediction block. Specifically, it can be implemented by referring to the method 1 and method 2 in the generation process of the chrominance prediction block mentioned above, which will not be repeated here.

[0214] The image or video processing method proposed in the present invention utilizes a prediction enhancement filter to reconstruct an image. Specifically, the method includes obtaining statistical information of a current prediction block; updating a filter threshold based on the statistical information and a previously determined filter threshold; selecting a prediction enhancement filter based on the updated filter threshold; and reconstructing the image using the selected prediction enhancement filter. Compared to prior art techniques that utilize a single-intensity, isotropic low-pass filter (e.g., LBCCP), embodiments of the present invention utilize statistical information of the prediction block (e.g., texture intensity and direction features) to select a more suitable filter, thereby achieving a better prediction gain.

[0215] An embodiment of the present invention further provides an encoder, comprising a memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the method of any of the above embodiments is implemented.

[0216] An embodiment of the present invention further provides a decoder including a memory and a processor. The memory stores an image processing program, and when the image processing program is executed by the processor, the method of any of the above embodiments is implemented.

[0217] An embodiment of the present invention also provides a coding and decoding system, which includes a coding device and a decoding device, wherein the coding device includes a first processor, and the first processor is used to execute instructions to implement the method of any of the above embodiments; the decoding device includes a second processor, and the second processor is used to execute instructions to implement the method of any of the above embodiments.

[0218] The present invention further provides a non-volatile storage medium having an image processing program stored thereon. When the image processing program is executed by a processor, the method of any of the above embodiments is implemented.

[0219] In the embodiments of the encoder, decoder, encoding and decoding system and storage medium provided by the present invention, all technical features of any of the above-mentioned method embodiments may be included. The expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods and will not be repeated here.

[0220] An embodiment of the present invention further provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer executes the methods in the various possible implementation modes described above.

[0221] An embodiment of the present invention also provides a chip, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that a device equipped with the chip executes the methods in various possible implementations as described above.

[0222] It should be understood that the above scenarios are merely examples and do not limit the application scenarios of the technical solutions provided by the embodiments of the present invention. The technical solutions of the present invention can also be applied to other scenarios. For example, those skilled in the art will appreciate that as system architectures evolve and new business scenarios emerge, the technical solutions provided by the embodiments of the present invention will also be applicable to similar technical problems.

[0223] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0224] The steps in the method of the embodiment of the present invention can be adjusted in sequence, combined, or deleted according to actual needs.

[0225] The units in the device of the embodiment of the present invention can be merged, divided, and deleted according to actual needs.

[0226] In the present invention, the same or similar terminology, technical solutions and / or application scenario descriptions are generally only described in detail the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solution and other contents of the present invention, for the same or similar terminology, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0227] In the present invention, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0228] The various technical features of the technical solution of the present invention can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

[0229] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present invention.

[0230] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a storage disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0231] The above description is only an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for processing an image or video, wherein: The method comprises: Acquire a first candidate parameter list, wherein the first candidate parameter list is a candidate parameter list of a cross-component prediction fusion mode CCPMERGE, the first candidate parameter list includes multiple candidate parameters, and at least one of the multiple candidate parameters includes a parameter of a prediction enhancement filter PBF; and According to the selected candidate parameters, the chrominance information of a current block in an image is predicted to generate a chrominance prediction block.

2. The method according to claim 1, wherein: The method further comprises: The candidate parameters in the first candidate parameter list are reordered according to a certain rule to obtain a second candidate parameter list, The step of generating a chroma prediction block comprises: The chroma prediction block is generated according to the candidate parameters selected from the second candidate parameter list.

3. The method according to claim 1 or 2, wherein: The method further comprises: Truncating the first candidate parameter list or the second candidate parameter list to obtain a third candidate parameter list; The step of generating a chroma prediction block comprises: The chroma prediction block is generated according to the candidate parameters selected from the third candidate parameter list.

4. The method according to claim 1, wherein: The method further comprises: reordering the candidate parameters in the first candidate parameter list according to a certain rule, and truncating the reordered candidate parameter list; or The first candidate parameter list is truncated, and the truncated candidate parameter list is reordered according to a certain rule.

5. The method according to claim 1, wherein: The first candidate parameter list includes identification information of the prediction enhancement filter, and the identification information is used to identify whether one or more candidate parameters in the first candidate parameter list carry parameters of the prediction enhancement filter or support the prediction enhancement filter.

6. The method according to claim 1, wherein: The method further comprises: Determine whether a candidate parameter in the first candidate parameter list carries a parameter of the prediction enhancement filtering or supports the prediction enhancement filtering, wherein the step of determining is based on at least one of the following: an encoding mode of the candidate parameters; The coding cost of the candidate parameter after combining the prediction enhancement filtering in the cross-component prediction fusion mode; and Statistical characteristics of the predicted block.

7. The method according to claim 6, wherein: If the encoding mode of the candidate parameters is a multi-model cross-component linear model MM-CCLM or a multi-model convolutional cross-component model MM-CCCM, it means that it is necessary to determine whether the candidate parameters support the prediction enhancement filtering. Otherwise, it is not necessary to determine whether the candidate parameters support the prediction enhancement filtering.

8. The method according to claim 6, wherein: When the encoding cost of the candidate parameter is less than one or more thresholds, the candidate parameter does not use the prediction enhancement filtering; and / or when the encoding cost of the candidate parameter is in different threshold intervals, different prediction enhancement filtering is introduced.

9. The method according to claim 8, wherein: The coding cost is calculated based on the absolute difference between the result of attempting to reconstruct the chrominance image reference area obtained by using the candidate parameters and adding the prediction enhancement filtering and the actual reconstructed chrominance image reference area.

10. The method according to claim 6, wherein: When the statistical characteristics of the prediction block are less than one or more thresholds, the candidate parameters do not use the prediction enhancement filter; and / or when the statistical characteristics of the prediction block are within different threshold intervals, different prediction enhancement filters are introduced.

11. The method according to claim 2, wherein: In the process of reordering the candidate parameters in the first candidate parameter list, the candidate parameters are uniformly ordered according to the encoding costs corresponding to the candidate parameters without using the prediction enhancement filtering and with using the prediction enhancement filtering.

12. The method according to claim 2, wherein: In the process of reordering the candidate parameters in the first candidate parameter list, the candidate parameters are divided into different candidate parameter sets, and different sorting rules are selected for different candidate parameter sets. Finally, the candidate parameters of these candidate parameter sets are merged and sorted.

13. The method according to claim 3, wherein: In the process of truncating the first candidate parameter list or the second candidate parameter list, it is determined whether to put the candidate parameters in the first candidate parameter list or the second candidate parameter list into the third candidate parameter list based on a coding cost threshold.

14. The method according to claim 3, wherein: In the process of truncating the first candidate parameter list or the second candidate parameter list, it is determined whether to put the candidate parameters in the first candidate parameter list or the second candidate parameter list into the third candidate parameter list based on the parameter list length.

15. The method according to claim 1, wherein: The selected candidate parameters carry the parameters of the prediction enhancement filter PBF, a chroma prediction result is generated based on the selected candidate parameters, and the prediction enhancement filter is applied to the chroma prediction result to generate the chroma prediction block.

16. The method according to claim 1, wherein: The selected candidate parameters carry the parameters of the prediction enhancement filter PBF, the prediction enhancement filter is applied to the luminance block, and the chrominance prediction block is generated by calculation based on the luminance block to which the prediction enhancement filter is applied and the selected candidate parameters.

17. An encoder comprising a processor, the processor being configured to execute instructions to implement the method of any one of claims 1 to 16.

18. A decoder comprising a processor, wherein the processor is configured to execute instructions to implement the method according to any one of claims 1 to 16.

19. A coding and decoding system, comprising a coding device and a decoding device, wherein the coding device comprises a first processor, the first processor is used to execute instructions to implement the method described in any one of claims 1 to 16, and the decoding device comprises a second processor, the second processor is used to execute instructions to implement the method described in any one of claims 1 to 16.

20. A method for processing an image or video, wherein: The method comprises: Get the statistics of the current prediction block; updating a filter threshold based on the statistical information and a previously determined filter threshold; selecting a prediction enhancement filter based on the updated filter threshold; and The image is reconstructed using the selected prediction enhancement filter.

21. The method according to claim 20, wherein: The statistical information includes a degree of change of the current prediction block in a horizontal direction, a vertical direction, a diagonal direction and / or a counter-diagonal direction.

22. The method according to claim 20, wherein: The filter threshold is dynamically adjusted based on an exponential moving average method.

23. The method according to claim 20, wherein: In the process of selecting the prediction enhancement filter, the statistical information of the current prediction block is compared with the updated filter threshold, and the prediction enhancement filter is selected according to the comparison result.

24. The method according to claim 23, wherein: In the process of selecting the prediction enhancement filter, the prediction enhancement filter is selected from at least one of the following directional filters: a horizontal direction, a vertical direction, a diagonal direction, a counter-diagonal direction, and a non-directional filter.

25. An encoder comprising a processor, the processor being configured to execute instructions to implement the method of any one of claims 20 to 24.

26. A decoder comprising a processor, the processor being configured to execute instructions to implement the method of any one of claims 20 to 24.

27. A coding and decoding system, comprising a coding device and a decoding device, wherein the coding device comprises a first processor, the first processor is used to execute instructions to implement the method described in any one of claims 20 to 24, and the decoding device comprises a second processor, the second processor is used to execute instructions to implement the method described in any one of claims 20 to 24.

Images