Encoding method and decoding method, encoder, decoder, and storage medium
By dividing the chroma sample set in the cross-component convolution model and determining the corresponding filtering coefficients, the multi-model scheme solves the problem of insufficient filtering performance in ALF-CCCM technology and improves the overall performance of encoding and decoding.
Patent Information
- Application Number
- PCT/CN2025/095268
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2026-11-19
AI Technical Summary
The existing ALF-CCCM technology suffers from poor filtering performance in cross-component convolution models due to incomplete consideration of filtering scenarios.
When using a cross-component convolution model for filtering in the current block, the chroma sample set is divided into at least two sets based on the sample partitioning threshold, and the corresponding filtering coefficients are determined for each set. A multi-model approach is then used for filtering.
The filtering performance of the CCCM model has been improved, thus enhancing the overall encoding and decoding effect.
Smart Images

Figure CN2025095268_19112026_PF_FP_ABST
Abstract
Description
Encoding / decoding methods, encoders, decoders, and storage media Technical Field
[0001] This application relates to the field of video encoding and decoding technology, specifically to an encoding and decoding method, an encoder, a decoder, and a storage medium. Background Technology
[0002] For in-loop filters, the Adaptive Loop Filter-Cross Component Convolution Model (ALF-CCCM) primarily utilizes the relationship between the luminance and chrominance components to map the filtering result of the luminance component adaptive filter onto the chrominance component. However, in certain filtering scenarios, the ALF-CCCM technique suffers from incomplete consideration, leading to suboptimal filtering performance. Summary of the Invention
[0003] This application provides an encoding / decoding method, an encoder, a decoder, and a storage medium.
[0004] The technical solution of this application embodiment can be implemented as follows:
[0005] In a first aspect, embodiments of this application provide a decoding method applied to a decoder, the method comprising:
[0006] When filtering is performed using a cross-component convolution model in the current block, at least two chroma sample sets for the current block are determined based on the sample partitioning threshold of the current block.
[0007] Based on at least two sets of chroma samples and at least two sets of corresponding luminance samples of the current block, determine at least two sets of filter coefficients for the current block;
[0008] Based on the first set of filter coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set, the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set is determined. The first chromaticity sample set is any one of at least two chromaticity sample sets.
[0009] Secondly, embodiments of this application provide an encoding method applied to an encoder, the method comprising:
[0010] When filtering is performed using a cross-component convolution model in the current block, at least two chroma sample sets for the current block are determined based on the sample partitioning threshold of the current block.
[0011] Based on at least two sets of chroma samples and at least two sets of corresponding luminance samples of the current block, determine at least two sets of filter coefficients for the current block;
[0012] Based on the first set of filter coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set, the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set is determined. The first chromaticity sample set is any one of at least two chromaticity sample sets.
[0013] Thirdly, embodiments of this application provide an encoder, which includes a first determining unit and a first filtering unit; wherein:
[0014] The first determining unit is configured to determine at least two chroma sample sets of the current block based on the sample partitioning threshold of the current block when filtering is performed using a cross-component convolution model in the current block.
[0015] The first determining unit is further configured to determine at least two sets of filter coefficients for the current block based on at least two sets of chroma samples and at least two sets of corresponding luminance samples.
[0016] The first filtering unit is configured to determine the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set based on the first set of filtering coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set. The first chromaticity sample set is any one of at least two chromaticity sample sets.
[0017] Fourthly, embodiments of this application provide an encoder, which includes a first memory and a first processor; wherein,
[0018] A first memory for storing computer programs that can run on a first processor;
[0019] The first processor is used to execute the method described in the second aspect when running a computer program.
[0020] Fifthly, embodiments of this application provide a decoder, which includes a second determining unit and a second filtering unit; wherein:
[0021] The second determining unit is configured to determine at least two chroma sample sets of the current block based on the sample partitioning threshold of the current block when the current block is filtered using a cross-component convolution model.
[0022] The second determining unit is further configured to determine at least two sets of filter coefficients for the current block based on at least two sets of chroma samples and at least two sets of corresponding luminance samples.
[0023] The second filtering unit is configured to determine the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set based on the first set of filtering coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set. The first chromaticity sample set is any one of at least two chromaticity sample sets.
[0024] Sixthly, embodiments of this application provide a decoder, which includes a second memory and a second processor; wherein,
[0025] The second memory is used to store computer programs that can run on the second processor;
[0026] The second processor is used to execute the methods described in the first aspect when running a computer program.
[0027] In a seventh aspect, embodiments of this application provide a computer-readable storage medium that stores a bitstream generated by such encoding method.
[0028] Eighthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed, implements the method as described in the first aspect or the method as described in the second aspect.
[0029] This application provides an encoding / decoding method, encoder, decoder, and storage medium. The method includes: when filtering is performed on a current block using a cross-component convolution model, determining at least two chroma sample sets for the current block based on a sample partitioning threshold of the current block; determining at least two sets of filtering coefficients for the current block based on the at least two chroma sample sets and the corresponding at least two luminance sample sets; and determining a first filtered chroma sample value for a chroma sample in the first chroma sample set based on the first set of filtering coefficients corresponding to the first chroma sample set and the luminance sample value of a luminance sample in the first luminance sample set, wherein the first chroma sample set is any one of the at least two chroma sample sets.
[0030] Thus, whether at the encoding or decoding end, when the current block uses the cross-component convolutional model (CCCM) for filtering, the current block directly uses a multi-model scheme to filter the chroma samples of the current block. Specifically, based on the sample partitioning threshold of the current block, the chroma samples in the current block are accurately divided into at least two chroma sample sets; and a corresponding set of filtering coefficients are used to filter different chroma sample sets of the current block, thereby improving the filtering performance of the CCCM model and thus improving the encoding and decoding performance. Attached Figure Description
[0031] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0032] Figure 1 is a schematic diagram of a block-based hybrid video compression coding framework;
[0033] Figure 2 is a schematic diagram of the decoding framework of ALF-CCCM;
[0034] Figure 3 is a schematic diagram of a video encoding and decoding network architecture provided in an embodiment of this application;
[0035] Figure 4 is a schematic block diagram of an encoder provided in an embodiment of this application;
[0036] Figure 5 is a schematic block diagram of a decoder provided in an embodiment of this application;
[0037] Figure 6 is a flowchart illustrating a decoding method provided in an embodiment of this application;
[0038] Figure 7 is a flowchart illustrating an encoding method provided in an embodiment of this application;
[0039] Figure 8 is a schematic diagram of the composition structure of an encoder provided in an embodiment of this application;
[0040] Figure 9 is a schematic diagram of the specific hardware structure of an encoder provided in an embodiment of this application;
[0041] Figure 10 is a schematic diagram of the composition structure of a decoder provided in an embodiment of this application;
[0042] Figure 11 is a schematic diagram of the specific hardware structure of a decoder provided in an embodiment of this application;
[0043] Figure 12 is a schematic diagram of the composition structure of an encoding and decoding system provided in an embodiment of this application. Detailed Implementation
[0044] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0046] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0047] It should also be noted that the terms "first, second, and third" used in the embodiments of this application are only used to distinguish similar objects and do not represent a specific order of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.
[0048] In video images, a coding block (CB) is typically represented by a first image component, a second image component, and a third image component. These three image components are a luma component, a blue chroma component, and a red chroma component, respectively. Specifically, the luma component is usually represented by the symbol Y, the blue chroma component is usually represented by the symbol Cb or U, and the red chroma component is usually represented by the symbol Cr or V. Thus, video images can be represented in YCbCr format or YUV format.
[0049] Before providing a further detailed description of the embodiments of this application, the nouns and terms used in the embodiments of this application will be explained. The nouns and terms used in the embodiments of this application shall be interpreted as follows:
[0050] H.266 / Versatile Video Coding (VVC);
[0051] Joint Video Experts Team (JVET);
[0052] Traditional video coding exploration platform reference software testing model - Enhanced Compression Model (ECM);
[0053] A neural network-based video coding model (NNVC) platform;
[0054] Luminance (Luma, or represented as Y);
[0055] Chrominance (also known as Chroma or Cb, Cr);
[0056] Intra;
[0057] Inter-frame;
[0058] Deblocking Filter (DBF);
[0059] Loop filter (LF);
[0060] Bilateral filtering (BIF);
[0061] Adaptive Loop Filter (ALF);
[0062] Sample Adaptive Offset (SAO);
[0063] Cross Component Sample Adaptive Offset (CCSAO);
[0064] Cross Component Adaptive Loop Filter (CCALF);
[0065] Cross Component Convolution Model (CCCM);
[0066] Coding Unit (CU);
[0067] Coding Tree Unit (CTU);
[0068] Adaptive Parameter Set (APS).
[0069] It's understandable that digital video compression technology primarily compresses massive amounts of digital video data to facilitate transmission and storage. With the surge in internet video and increasing demands for video clarity, while existing digital video compression standards can save considerable video data, there is still a need to pursue better digital video compression technologies to reduce the bandwidth and traffic pressure on digital video transmission.
[0070] In digital video encoding, the encoder reads unequal samples from raw video sequences of different color formats, including luminance and chrominance components; that is, the encoder reads a black-and-white or color image. These samples are then divided into blocks, and the block data is encoded by the encoder. Modern encoders typically use a hybrid block-based encoding model, generally including intra-frame and inter-frame prediction, transform and quantization, inverse transform and inverse quantization, loop filtering, and entropy coding. The processing flow is shown in Figure 1. Intra-frame prediction only references information from the same frame to predict sample information within the current block, eliminating spatial redundancy. Inter-frame prediction can reference image information from different frames, using motion estimation to search for the motion vector information that best matches the current block, eliminating temporal redundancy. The transform converts the predicted image blocks to the frequency domain, redistributing energy; combined with quantization, it removes information that the human eye is insensitive to, eliminating visual redundancy. Loop filtering uses statistical information to filter the image, improving both subjective and objective image quality. Entropy coding can eliminate character redundancy based on the current context model and the probability information of the binary bitstream.
[0071] It should be noted that the current block (CB) can be the current CTU, the current CU, etc. Due to the need for parallel processing, images can be divided into slices, etc. Slices within the same image can be processed in parallel, meaning there is no data dependency between them. A "frame" is a commonly used term, generally understood as one image. In the embodiments of this application, the frame can also be replaced with an image or a slice, etc. Furthermore, in the embodiments of this application, a sample can also be called a pixel, and a sample includes location information and a value.
[0072] In ECM, loop filters mainly include bilateral filters, luma mapping and chroma scaling, deblocking (DB) filters, sample offset compensation filters, cross-component sample offset compensation filters, adaptive loop filters, and cross-component adaptive loop filters. Among these, the adaptive loop filter (ALF) is designed to minimize the mean square error between the reconstructed image and the original image. The cross-component adaptive loop filter (CCALF) is designed to utilize luma information to minimize the mean square error between the chroma reconstructed image and the original image.
[0073] In the ECM exploration experiment, the cross-component convolution model based on adaptive filters (hereinafter referred to as ALF-CCCM) mainly utilizes the relationship between the luminance component and the chrominance component to map the filtering result of the luminance component adaptive filter onto the chrominance component, thereby improving the filtering performance of the chrominance component. The framework diagram of its decoding end is shown in Figure 2, which provides a schematic diagram of the ALF-CCCM decoding framework.
[0074] Specifically, this technology consists of three parts: parameter analysis, CCCM filter coefficient derivation, and CCCM filtering. This will be explained in detail from the perspective of the decoding end.
[0075] (1) Parameter analysis:
[0076] The parameters of ALF-CCCM are mainly CTU-level parameters, including: whether the current CTU uses ALF-CCCM (alf_cccm_enabled), the window size when the current CTU uses ALF-CCCM (alf_cccm_window_size_idx), and the filter template used (alf_cccm_model_type_idx), which are located in the coding_tree_unit() function of the syntax element table. In one implementation, the syntax elements of ALF-CCCM are represented as shown in Tables 1 and 2, for example.
[0077] Table 1
[0078] Table 2
[0079] Wherein, alfCccm(CtbAddrX, CtbAddrY): the main function for encoding ALF-CCCM syntax elements, where CtbAddrX represents the x-coordinate of the current CTU in the image, in units of CTU. CtbAddrY represents the y-coordinate of the CTU to be encoded in the image, in units of CTU.
[0080] is_intra_slice: A binary variable. A value of 1 indicates that the slice containing the current CTU is an intra-frame slice; a value of 0 indicates that the slice containing the current CTU is not an intra-frame slice.
[0081] has_ref_pic: A binary variable. A value of 1 indicates that the slice containing the current CTU contains a reference image that uses ALF-CCCM; a value of 0 indicates that the slice containing the current CTU does not contain a reference image that uses ALF-CCCM.
[0082] `inherit_from_ref_pic`: A binary variable. A value of 1 indicates that the CCCM parameter of each CTU in the current frame is the same as the CCCM parameter of the CTU at the same position in the nearest reference frame; a value of 0 indicates that the CCCM parameter of each CTU in the current frame is different from the CCCM parameter of the CTU at the same position in the nearest reference frame. This syntax element is actually a frame-level parameter, and writing it at the CTU level is unreasonable.
[0083] alf_cccm_enabled: A binary variable. A value of 1 indicates that the current CTU uses ALF-CCCM for filtering; a value of 0 indicates that the current CTU does not use ALF-CCCM for filtering.
[0084] alf_cccm_merge_mode_enabled: A binary variable. A value of 1 indicates that the current CTU uses the ALF-CCCM parameters of the left-adjacent or top-adjacent CTU for filtering; a value of 0 indicates that the current CTU does not use the ALF-CCCM parameters of the left-adjacent or top-adjacent CTU for filtering.
[0085] alf_cccm_use_left_or_above_ctu_info: A binary variable. A value of 1 indicates that the current CTU uses the ALF-CCCM parameters of the left adjacent CTU for filtering; a value of 0 indicates that the current CTU uses the ALF-CCCM parameters of the upper adjacent CTU for filtering.
[0086] `alf_cccm_window_size_idx`: This syntax element has 8 possible values. Each value corresponds to a different window size, used to divide the current CTU into multiple adjacent and non-overlapping blocks. Window sizes can be: 4×4, 8×2, 2×8, 8×8, 16×16, 32×32, 64×64, and 128×128.
[0087] alf_cccm_model_type_idx: This syntax element has 8 values, each corresponding to a different filter template, as shown in Table 3. It schematically illustrates several ALF_CCCM filter templates. It is used to filter adjacent samples at different positions as input.
[0088] Table 3
[0089] First, determine if the slice containing the current CTU is an intra-frame slice. If it is, the current CTU does not perform ALF-CCCM, and the ALF-CCCM parameter parsing process ends. If the slice containing the current CTU is not an intra-frame slice, determine if the current CTU is the first CTU in the top-left corner of the image. If the current CTU is the first CTU in the top-left corner of the image and there is a reference frame in the current image that uses ALF-CCCM, then parse the syntax element `inherit_from_ref_pic`. Otherwise, do not parse the syntax element `inherit_from_ref_pic`. If this syntax element is 1, the current CTU uses the ALF-CCCM parameters of the CTU at the same position in the reference frame, and the parsing process ends. Next, parse the syntax element `alf_cccm_enabled`. Determine the value of the syntax element `alf_cccm_enabled`. If this syntax element is 1, determine whether the current CTU's left-adjacent CTU and top-adjacent CTU use ALF-CCCM filtering. If at least one of the upper or left adjacent CTUs uses ALF_CCCM filtering, the syntax element `alf_cccm_merge_mode_enabled` is further parsed. Otherwise, the current CTU does not use the ALF_CCCM parameters of its adjacent CTUs for filtering. The value of the syntax element `alf_cccm_merge_mode_enabled` is checked. If the value is 1, the current CTU uses the ALF_CCCM parameters of either the left or upper adjacent CTU for filtering. Then, the value of `alf_cccm_merge_cand_number` is checked. If the value is greater than 1, the syntax element `alf_cccm_use_left_or_above_ctu_info` is further parsed. The syntax element `alf_cccm_use_left_or_above_ctu_info` indicates whether the current CTU uses the ALF_CCCM parameters of its left or upper adjacent CTU for filtering. Otherwise, if `alf_cccm_merge_mode_enabled` = 0, the current CTU does not use the ALF_CCCM parameter of adjacent CTUs for filtering. In this case, the syntax elements `alf_cccm_window_size_idx` and `alf_cccm_model_type_idx` need to be parsed. Here, the value of `alf_cccm_merge_cand_number` is checked: a value of 2 indicates that both the left and top adjacent CTUs use ALF_CCCM filtering; a value of 1 indicates that either the top or left adjacent CTU uses ALF_CCCM filtering; and a value of 0 indicates that neither the top nor left adjacent CTUs use ALF_CCCM filtering.
[0090] (2) Derivation of CCCM filter coefficients:
[0091] The first step is to obtain the reconstructed sample values of the current CTU output by the front-end module, including luminance sample values and two chromaticity sample values. For example, the front-end module in Figure 2 can be SAO and CCSAO.
[0092] The second step is to determine whether the current CTU uses ALF_CCCM filtering based on the syntax element `alf_cccm_enabled`. The current CTU uses ALF_CCCM filtering only if all values of `alf_cccm_enabled` are 1. Otherwise, the current CTU does not perform ALF_CCCM filtering.
[0093] The third step is to divide the current CTU into several adjacent and non-overlapping blocks according to the syntax element alf_cccm_window_size_idx when the current CTU needs to be filtered by ALF_CCCM. The block size has been introduced above and will not be repeated here.
[0094] The fourth step is to select the filter template for ALF_CCCM based on the syntax element alf_cccm_model_type_idx.
[0095] The fifth step is to downsample the brightness sample output by the front-end module for each CTU to obtain the downsampled brightness sample.
[0096] The sixth step involves solving for the filtering coefficients of the CCCM filter used in each block, using the downsampled luminance sample as input and the chrominance sample output from the previous module as output. Unlike ALF, the target for solving the filtering coefficients is no longer the difference between the original sample and the sample to be filtered, but rather the available chrominance sample. This allows both the encoder and decoder to solve the equations, eliminating the need to transmit the filtering coefficients in the bitstream.
[0097] It should also be noted that the filtering coefficients for the first and second chromaticity components are solved separately.
[0098] (3) ALF-CCCM filtering process:
[0099] The first step is to determine whether the current CTU uses ALF_CCCM filtering based on the syntax element `alf_cccm_enabled`. The current CTU uses ALF_CCCM filtering only if both `alf_cccm_enabled` values are 1. Otherwise, the current CTU does not perform ALF_CCCM filtering.
[0100] The second step is to divide the current CTU into several adjacent and non-overlapping blocks according to the syntax element alf_cccm_window_size_idx. The block size has been introduced above and will not be repeated here.
[0101] The third step is to select the ALF_CCCM filter template based on the syntax element alf_cccm_model_type_idx.
[0102] The fourth step is to downsample the brightness sample output by the front-end module for each CTU to obtain the downsampled brightness sample.
[0103] Fifth, for each block, the downsampled luminance sample is used as input, and filtered using the filter coefficients of the CCCM filter derived from the corresponding chrominance component to obtain the corresponding chrominance sample. The filtering formula can be: CCCMoutput(x0,y0)=∑ x,y ∈ S Coeffi*DownSampleLuma(x,y) (1)
[0104] Where S is the sample position adjacent to the sample to be filtered (x0, y0), Coeffi is the filter coefficient obtained in the derivation of CCCM filter coefficients, and DownSampleLuma is the sample after downsampling the brightness sample output by the front-end module.
[0105] The sixth step is to average the chromaticity sample obtained after CCCM filtering at the same position with the chromaticity sample output from the previous stage to obtain the chromaticity sample value after ALF-CCCM filtering.
[0106] Step 7: Compare the ALF-CCCM filtered chroma sample values with the chroma sample values output from the previous stage. If the difference between all samples in the block is less than or equal to a preset threshold, the block is determined to use a single filter for filtering; otherwise, no single filter is used (this can be referred to as "outliers" or "abnormal samples"). At this point, it is determined whether the block should use multiple filters. Calculate the average luminance value after downsampling of the current block. Use samples whose downsampling luminance sample values are less than this average value to solve for filter coefficient group 1. Use samples whose downsampling luminance sample values are not less than this average value to solve for filter coefficient group 2. Use the solved filter coefficient groups to perform steps 5 and 6 on the corresponding samples to obtain the corresponding ALF-CCCM filtered chroma sample values. Compare the difference between the ALF-CCCM filtered chroma sample values and the chroma sample values output from the previous stage again. If the difference between the ALF-CCCM filtered chroma sample values and the chroma sample values output from the previous stage is greater than a preset threshold, the block is not subjected to ALF-CCCM filtering. Otherwise, use a multi-model filter to filter the block.
[0107] Step 8: If the block uses ALF-CCCM filtering, the difference between the ALF-CCCM filtered chroma sample value and the chroma sample value output from the previous stage is calculated to obtain the ALF-CCCM filtered compensation value. This compensation value is then summed with the chroma sample output from the ALF / CCALF filter and output to the reconstructed image.
[0108] In the multi-model approach of ALF-CCCM technology, a filtering model is first solved for all samples in the block. If there are no bad pixels in the chroma samples after single-model filtering (e.g., the difference is greater than a preset threshold), the single model is directly used to filter the entire block, and the multi-model approach is no longer attempted. In this case, all samples in the block are used to derive the filtering coefficients of the single model, which reduces the accuracy of the filter and results in suboptimal filtering performance.
[0109] Based on this, embodiments of this application provide an encoding / decoding method. When filtering is performed using a cross-component convolution model for the current block, at least two chroma sample sets of the current block are determined based on the sample partitioning threshold of the current block; at least two sets of filtering coefficients of the current block are determined based on the at least two chroma sample sets and the corresponding at least two luminance sample sets; and a first filtered chroma sample value of the chroma sample in the first chroma sample set is determined based on the first set of filtering coefficients corresponding to the first chroma sample set and the luminance sample value of the luminance sample in the first luminance sample set, wherein the first chroma sample set is any one of the at least two chroma sample sets.
[0110] Thus, whether at the encoding or decoding end, when the current block uses the cross-component convolutional model (CCCM) for filtering, the current block directly uses a multi-model scheme to filter the chroma samples of the current block. Specifically, based on the sample partitioning threshold of the current block, the chroma samples in the current block are accurately divided into at least two chroma sample sets; and a corresponding set of filtering coefficients are used to filter different chroma sample sets of the current block, thereby improving the filtering performance of the CCCM model and thus improving the encoding and decoding performance.
[0111] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0112] Figure 3 is a schematic diagram of a video encoding and decoding network architecture provided in an embodiment of this application. As shown in Figure 3, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. The electronic devices can be various types of devices with video encoding and decoding capabilities, such as mobile phones, tablets, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensing devices, servers, etc., and this embodiment of the application does not impose any limitations.
[0113] This application provides a network architecture for a video encoding / decoding system that includes decoding and encoding methods. The decoder or encoder in this application can be the aforementioned electronic device. That is, the electronic device in this application has video encoding / decoding capabilities and generally includes a video / image encoder (referred to as an encoder) and a video / image decoder (referred to as a decoder).
[0114] Figure 4 is a schematic block diagram of an encoder provided in an embodiment of this application. As shown in Figure 4, the encoder 100 may include a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control and analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image buffer unit 110, etc. Among them, the filtering unit 108 can implement deblocking filtering and sample adaptive offset (SAO) filtering, and the encoding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (CABAC).For the input raw video signal, a video coding block can be obtained by partitioning it through a Coding Tree Unit (CTU). Then, the residual sample information obtained after intra-frame or inter-frame prediction is transformed by the transform and quantization unit 101, including transforming the residual information from the sample domain to the transform domain and quantizing the resulting transform coefficients to further reduce the bit rate. The intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block. Specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video coding block. The motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-frame prediction coding of the received video coding block relative to one or more blocks in one or more reference frames to provide time prediction information. The motion estimation performed by the motion estimation unit 105 is a process of generating motion vectors, which can estimate the motion of the video coding block. Then, the motion compensation unit 104 is used to perform the motion estimation based on the motion vectors determined by the motion estimation unit 105. The motion compensation is performed. After determining the intra-prediction mode, the intra-prediction unit 103 is also used to provide the selected intra-prediction data to the coding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the coding unit 109. In addition, the inverse transform and inverse quantization unit 106 is used to reconstruct the video coding block, reconstruct the residual block in the sample domain, and remove the block artifacts by the filter control analysis unit 107 and the filtering unit 108. Then, the reconstructed residual block is added to a predictive block in the frame of the decoding image buffer unit 110 to generate the reconstructed video coding block. The coding unit 109 is used to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-prediction mode and output the bitstream of the video signal. The decoding image buffer unit 110 is used to store the reconstructed video coding block for prediction reference. As video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoding image buffer unit 110.
[0115] Figure 5 is a schematic block diagram of a decoder provided in an embodiment of this application. As shown in Figure 5, the decoder 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image buffer unit 206, etc. The decoding unit 201 can perform header information decoding and CABAC decoding, and the filtering unit 205 can perform deblocking filtering and SAO filtering. After the input video signal undergoes the encoding processing shown in Figure 4, the bitstream of the video signal is output. This bitstream is input into the decoder 200, first passing through the decoding unit 201 to obtain the decoded transform coefficients. The transform coefficients are then processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the sample domain. The intra-frame prediction unit 203 can be used to generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from previously decoded blocks in the current frame or image. The motion compensation unit 204 determines the prediction information for the video decoding block by analyzing motion vectors and other associated syntax elements, and uses... The prediction information is used to generate a predictive block of the video block being decoded; the decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-prediction unit 203 or the motion compensation unit 204; the decoded video signal is passed through the filtering unit 205 to remove block artifacts, which can improve video quality; then the decoded video block is stored in the decoding image buffer unit 206, which stores reference images for subsequent intra-prediction or motion compensation, and is also used for the output of the video signal, thus obtaining the recovered original video signal.
[0116] It should be noted that the method in this application embodiment mainly affects the loop filtering part in the hybrid video compression coding framework. For example, the method in this application embodiment can be applied to the filtering unit 108 shown in FIG. 4 and the filtering unit 205 shown in FIG. 5. That is, the method in this application embodiment can be applied to the encoder, the decoder, or even simultaneously, without any limitation.
[0117] It should also be noted that when the embodiments of this application are applied to encoder 100, "current block" can refer to the block currently to be encoded in the video image (also called "encoded block"); when the embodiments of this application are applied to decoder 200, "current block" can refer to the block currently to be decoded in the video image (also called "decoded block"). In the embodiments of this application, "current block" can also be a chroma block to be filtered.
[0118] It should also be noted that in the embodiments of this application, "set" can also be defined as "group", "array", etc., and each "set" defines a type of chroma sample or a type of luminance sample.
[0119] To facilitate understanding of the technical solutions of the embodiments of this application, the technical solutions of this application are described in detail below through specific embodiments. The above-mentioned related technologies are optional solutions and can be arbitrarily combined with the technical solutions of the embodiments of this application, all of which fall within the protection scope of the embodiments of this application. The embodiments of this application include at least some of the following contents.
[0120] In one embodiment of this application, see Figure 6, which shows a flowchart of a decoding method provided in this embodiment.
[0121] As shown in Figure 6, the method may include:
[0122] S601: When filtering is performed using a cross-component convolution model in the current block, determine at least two chroma sample sets for the current block based on the sample partitioning threshold of the current block.
[0123] It should be noted that the decoding method provided in this application embodiment is applied to the decoder. Specifically, the decoding method can refer to a filtering method, and more specifically, it can refer to a filtering method based on multiple cross-component convolution models (which can be referred to as a multi-model filtering method). Using this method can improve the filtering performance of the CCCM model, thereby improving the decoding performance.
[0124] For example, the multi-model filtering method provided in this application embodiment is applicable to CCCM filters, such as LF-CCCM filters, ALF-CCCM filters, Inter-CCCM filters, Intra-CCCM filters, etc.
[0125] It should be noted that some syntactic element-based information, such as flags or indexes, can be written into the bitstream. By parsing the values of these syntactic elements in the bitstream, it is possible to determine whether the current block uses the CCCM model for filtering, the block size of the current block, and the filtering template of the CCCM model.
[0126] In some embodiments, the method further includes: parsing the value of a first syntax element in the bitstream; and determining whether the current block uses a CCCM model for filtering based on the value of the first syntax element. The first syntax element indicates whether the current block uses a CCCM model for filtering; it can also be understood as indicating whether the current block uses a multi-CCCM model for filtering. The first syntax element includes at least one of the following: block-level syntax element, coding unit-level syntax element, CTU-level syntax element, slice-level syntax element, etc.
[0127] For example, for the ALF-CCCM filter, the `alf_cccm_enabled` (i.e., the first syntax element) in the parsed bitstream is used to determine whether the current CTU uses ALF_CCCM filtering. The current CTU uses ALF_CCCM filtering only when `alf_cccm_enabled` is 1; this can be understood as any block of the current CTU using ALF_CCCM filtering. Otherwise, when `alf_cccm_enabled` is 0, the current CTU does not perform ALF_CCCM filtering; this can be understood as all blocks of the current CTU not using ALF_CCCM filtering.
[0128] In some embodiments, the method may further include: parsing the value of a second syntax element in the bitstream; and determining the size of the current block based on the value of the second syntax element. The second syntax element may be a block-level syntax element, a CTU-level syntax element, a slice-level syntax element, etc. The second syntax element may be a syntax element of the current block, a syntax element inherited from a reference block, or a syntax element inherited from a chroma block at the same position in a reference frame.
[0129] In some embodiments, the method may further include: determining the size of the current block based on the size of the prediction block or the size of the transformation block in the prediction phase.
[0130] In some embodiments, the method may further include: parsing the value of a third syntax element in the bitstream; and determining a filtering template based on the value of the third syntax element. The third syntax element may be a block-level syntax element, a CTU-level syntax element, a slice-level syntax element, etc. The third syntax element may be a syntax element of the current block, a syntax element inherited from a reference block, or a syntax element inherited from a chroma block at the same position in a reference frame.
[0131] For example, for the ALF-CCCM filter, the `alf_cccm_window_size_idx` (i.e., the second syntax element) in the parsed bitstream is used to divide the current CTU into several adjacent and non-overlapping blocks. For instance, this syntax element has eight values, corresponding to eight different window sizes used to divide the current CTU into multiple adjacent and non-overlapping blocks. Window sizes include: 4x4, 8x2, 2x8, 8x8, 16x16, 32x32, 64x64, 128x128, etc.
[0132] The `alf_cccm_model_type_idx` (i.e., the third syntax element) in the parsed bitstream is used to select the filter template for ALF_CCCM. For example, this syntax element has 8 possible values, each corresponding to one of the 8 different CCCM filter templates illustrated in Table 3 above.
[0133] The current block's sample partitioning threshold is used to divide the chroma samples in the current block into at least two chroma sample sets. The current block's sample partitioning threshold may include one or more thresholds.
[0134] In some embodiments, the method may further include: performing a first processing on the luminance parameters of the current block to determine M first thresholds for the current block, where M is a positive integer. Thus, adaptively determining the M first thresholds based on the distribution characteristics of the luminance parameters of the current block is beneficial for improving the accuracy of chroma sample classification, thereby enhancing the filtering performance of the multi-model approach.
[0135] For example, the brightness parameters include, but are not limited to, brightness sample values or brightness residual values. The brightness sample value of the current block can be a brightness reconstruction value, or it can be a brightness sample value before loop filtering (LF), a brightness sample value before deblocking filtering (DB), a brightness sample value before sample offset compensation (SAO), a brightness sample value before cross-component sample offset compensation (CCSAO), a brightness sample value before adaptive loop filtering (ALF), a brightness sample value before filtering by the cross-component convolution model of the adaptive filter (ALF-CCCM), or a brightness sample value after ALF-CCCM filtering.
[0136] The brightness sample value can also be the brightness sample value input or output of any of the front-end modules such as the LF filter module, DB filter module, SAO filter module, CCSAO filter module, ALF filter module, ALF-CCCM filter module, etc.
[0137] In some embodiments, the brightness sample value can be the downsampled brightness sample value, and the brightness residual value can be the downsampled brightness residual value.
[0138] It should be noted that in the embodiments of this application, the current block can be the current chroma block to be filtered, and the luminance parameter of the current block can be the luminance parameter of the luminance sample in the same luminance block of the current chroma block. Correspondingly, the downsampled luminance sample value can be the downsampled luminance sample value of the same luminance block, and the downsampled luminance residual value can be the downsampled luminance residual value of the same luminance block.
[0139] For example, the first process may calculate an average value, using the average value of the brightness parameters of the current block as a first threshold. The first process may also perform median statistics, using the median of the brightness parameters of the current block as the first threshold. Alternatively, the first process may perform mode statistics, using the mode of the brightness parameters of the current block as the first threshold.
[0140] For example, the first process can be interval division. Performing the first process on the brightness parameters of the current block to determine M first thresholds for the current block may include: determining a first value range of the brightness parameters of the current block; and determining M first thresholds for the current block based on the first value range of the brightness parameters of the current block and the number of first sub-intervals.
[0141] In some embodiments, the number of the first sub-intervals can be adaptively set. For example, the method further includes determining the number of the first sub-intervals based on the brightness parameters of the current block. Adaptively determining the number of the first sub-intervals according to the distribution characteristics of the brightness parameters of the current block is beneficial to improving the classification accuracy of chroma samples, thereby improving the performance of multi-model filtering.
[0142] In some embodiments, the number of the first sub-intervals can be fixed. Dividing the first value interval into a fixed number of first sub-intervals corresponds to dividing the chroma samples of the current block into a fixed number of chroma sample sets. For example, the number of the first sub-intervals can be 2, 3, 4, etc.
[0143] In some embodiments, the number of first sub-intervals is M+1, that is, the first value interval is divided into M+1 first sub-intervals, and the M first thresholds of the current block are determined. For example, the first value interval is divided equally or approximately equally into M+1 first sub-intervals, and the M intermediate dividing values are used as the M first thresholds.
[0144] In some embodiments, performing a first processing on the brightness parameters of the current block to determine M first thresholds for the current block may include: determining a first value range of the brightness parameters of the current block; and determining M first thresholds for the current block based on the distribution characteristics of the brightness parameters of the current block, the first value range, and the number of first sub-ranges.
[0145] In some embodiments, determining at least two chroma sample sets for the current block based on the sample partitioning threshold of the current block may include: determining M+1 chroma sample sets for the current block based on M first thresholds and the luminance parameter of the current block. That is, M+1 first sub-intervals are defined by the M first thresholds, and chroma samples in the current block whose luminance parameters belong to the same first sub-interval are assigned to the same chroma sample set. The luminance parameter can be the luminance parameter of the luminance sample at the corresponding position of the chroma sample.
[0146] For example, when M is 1, based on M first thresholds and the luminance parameters of the current block, determining the M+1 chromaticity sample sets of the current block can include: assigning chromaticity samples in the current block whose luminance parameter values are less than the first thresholds to the second chromaticity sample set; and assigning chromaticity samples in the current block whose luminance parameter values are greater than or equal to the first thresholds to the third chromaticity sample set. In other words, by defining two first sub-intervals using a first threshold, the chromaticity samples of the current block are divided into two chromaticity sample sets.
[0147] For example, when M is greater than 1, based on the M+1 first sub-intervals defined by the M first thresholds, the chromaticity samples of the current block whose luminance parameters belong to the same first sub-interval are divided into the same sample set, thereby obtaining the M+1 sample sets of the current block.
[0148] In some embodiments, the method may further include: performing a second processing on the chromaticity parameters of the current block to determine N second thresholds for the current block, where N is a positive integer.
[0149] For example, chromaticity parameters include, but are not limited to, chromaticity sample values or chromaticity residual values. The chromaticity sample values of the current block can be chromaticity reconstructed values, or they can be chromaticity sample values before loop filtering (LF), before deblocking filtering (DB), before sample offset compensation (SAO), before cross-component sample offset compensation (CCSAO), before adaptive loop filtering (ALF), or before cross-component convolution model ALF-CCCM filtering, etc.
[0150] The chroma sample value of the current block can also be the input or output chroma sample value of any module such as the LF filter module, DB filter module, SAO filter module, CCSAO filter module, ALF filter module, etc.
[0151] For example, the second process can calculate an average value, using the average value of the chroma parameters of the current block as a second threshold. The second process can also be median statistics, using the median of the chroma parameters of the current block as a second threshold. Finally, the second process can be mode statistics, using the mode of the chroma parameters of the current block as a second threshold.
[0152] For example, the second processing can be interval division. Performing the second processing on the chroma parameters of the current block to determine N second thresholds for the current block may include: determining the second value range of the chroma parameters of the current block; and determining the N second thresholds for the current block based on the second value range and the number of second sub-intervals of the chroma parameters of the current block.
[0153] In some embodiments, the number of second sub-intervals can be adaptively set. For example, the method further includes determining the number of second sub-intervals based on the chroma parameters of the current block. Adaptively determining the number of second sub-intervals according to the distribution characteristics of the chroma parameters of the current block is beneficial to improving the classification accuracy of chroma samples, thereby improving the performance of multi-model filtering.
[0154] In some embodiments, the number of second sub-intervals can be fixed. Dividing the second value interval into a fixed number of second sub-intervals corresponds to dividing the chroma samples of the current block into a fixed number of chroma sample sets. For example, the number of second sub-intervals can be 2, 3, 4, etc.
[0155] In some embodiments, the number of second sub-intervals is N+1, that is, the first value interval is divided into N+1 second sub-intervals to determine the N second thresholds of the current block. For example, the second value interval is divided equally or approximately equally into N+1 second sub-intervals, and the N intermediate dividing values are used as the N second thresholds.
[0156] In some embodiments, performing a second processing on the chromaticity parameters of the current block to determine N second thresholds for the current block may include: determining a second value range of the chromaticity parameters of the current block; and determining N second thresholds for the current block based on the distribution characteristics of the chromaticity parameters of the current block, the second value range, and the number of second sub-ranges.
[0157] In some embodiments, determining at least two chroma sample sets for the current block based on the sample partitioning threshold of the current block includes: determining N+1 chroma sample sets for the current block based on N second thresholds and the chroma parameters of the current block. That is, the N second thresholds define N+1 second sub-intervals, and chroma samples whose chroma parameters of the current block belong to the same second sub-interval are partitioned into the same chroma sample set.
[0158] For example, when N is 1, determining at least two chromaticity sample sets for the current block based on at least two second sub-intervals defined by N second thresholds and the chromaticity parameters of the current block may include: assigning chromaticity samples in the current block whose chromaticity parameter values are less than the second threshold to a fourth chromaticity sample set; and assigning chromaticity samples in the current block whose chromaticity parameter values are greater than or equal to the second threshold to a fifth chromaticity sample set. That is, one second threshold defines two second sub-intervals, and the chromaticity samples of the current block are divided into two chromaticity sample sets based on these two second sub-intervals.
[0159] When N is 1, based on at least two second sub-intervals defined by N second thresholds and the chromaticity parameters of the current block, at least two chromaticity sample sets of the current block are determined. This may also include: assigning chromaticity samples in the current block whose chromaticity parameter values are less than or equal to the second threshold to the fourth chromaticity sample set; and assigning chromaticity samples in the current block whose chromaticity parameter values are greater than the second threshold to the fifth chromaticity sample set.
[0160] For example, when N is greater than 1, determining at least two chromaticity sample sets for the current block based on at least two second sub-intervals defined by N second thresholds and the chromaticity parameters of the current block may include: determining N+1 second sub-intervals defined by N second thresholds, and dividing samples whose chromaticity parameters of the current block belong to the same second sub-interval into the same chromaticity sample set, thereby obtaining N+1 chromaticity sample sets for the current block.
[0161] In some embodiments, the sample partitioning threshold includes M first thresholds and N second thresholds, where M and N are both positive integers. Based on the sample partitioning threshold of the current block, determining at least two chroma sample sets for the current block can include: determining a (M+1)×(N+1) chroma sample set for the current block based on the M first thresholds, N second thresholds, the luminance parameter of the current block, and the chroma parameter of the current block. That is, considering both the chroma parameter of the chroma sample and the luminance parameter of the luminance sample at the corresponding position, the chroma samples within the block are classified to adapt to the filtering requirements of a specific scenario.
[0162] For example, when both M and N are 1, based on M first thresholds, N second thresholds, the luminance parameter of the current block, and the chrominance parameter of the current block, a (M+1)×(N+1) chrominance sample set for the current block is determined, including: assigning chrominance samples in the current block whose luminance parameter values are less than the first threshold and whose chrominance parameter values are less than the second threshold to the sixth chrominance sample set; assigning chrominance samples in the current block whose luminance parameter values are greater than or equal to the first threshold and whose chrominance parameter values are less than the second threshold to the seventh chrominance sample set; assigning chrominance samples in the current block whose luminance parameter values are less than the first threshold and whose chrominance parameter values are greater than or equal to the second threshold to the eighth chrominance sample set; and assigning chrominance samples in the current block whose luminance parameter values are greater than or equal to the first threshold and whose chrominance parameter values are greater than or equal to the second threshold to the ninth chrominance sample set.
[0163] For example, when both M and N are 1, based on M first thresholds, N second thresholds, the luminance parameter of the current block, and the chrominance parameter of the current block, a (M+1)×(N+1) chrominance sample set for the current block is determined, including: assigning chrominance samples in the current block whose luminance parameter values are less than or equal to the first threshold and whose chrominance parameter values are less than or equal to the second threshold to the sixth chrominance sample set; assigning chrominance samples in the current block whose luminance parameter values are greater than the first threshold and whose chrominance parameter values are less than or equal to the second threshold to the seventh chrominance sample set; assigning chrominance samples in the current block whose luminance parameter values are less than or equal to the first threshold and whose chrominance parameter values are greater than the second threshold to the eighth chrominance sample set; and assigning chrominance samples in the current block whose luminance parameter values are greater than the first threshold and whose chrominance parameter values are greater than the second threshold to the ninth chrominance sample set.
[0164] S602: Based on at least two sets of chroma samples and at least two sets of corresponding luminance samples of the current block, determine at least two sets of filter coefficients for the current block;
[0165] In some embodiments, determining at least two sets of filtering coefficients for the current block based on at least two chroma sample sets and at least two corresponding luminance sample sets includes: determining a first set of filtering coefficients corresponding to the first chroma sample set based on the first chroma sample value of the chroma sample in the first chroma sample set and the luminance sample value of the luminance sample in the first luminance sample set, wherein the position of the chroma sample in the first chroma sample set corresponds to the position of the luminance sample in the first luminance sample set.
[0166] The first chromaticity sample set is any one of at least two chromaticity sample sets. In some embodiments, the first luminance sample set may include luminance samples downsampled from the co-located luminance blocks of the current block, and the first luminance sample set may also include luminance samples from the boundary-filled regions. In other embodiments, the first luminance sample set may include luminance samples from the co-located luminance blocks of the current block, and the first luminance sample set may also include luminance samples from the boundary-filled regions.
[0167] For example, when deriving the CCCM model filter coefficients, the downsampled luminance sample values in the second luminance sample set are used as input, and the first chromaticity sample values in the first chromaticity sample set are used as output. The Wiener-Hopp equation is constructed, and the first set of filter coefficients corresponding to the first chromaticity sample set is solved.
[0168] The first chromaticity sample value can be the chromaticity sample value before CCCM filtering. Specifically, the first chromaticity sample value can be the chromaticity reconstructed value, or it can be the chromaticity sample value before loop filtering (LF), before deblocking filtering (DB), before sample offset compensation (SAO), before cross-component sample offset compensation (CCSAO), before adaptive loop filtering (ALF), or before the cross-component convolution model of adaptive filter (ALF-CCCM filtering), etc.
[0169] The first chromaticity sample value can also be the chromaticity sample value input or output of any module such as the LF filter module, DB filter module, SAO filter module, CCSAO filter module, ALF filter module, etc.
[0170] S603: Based on the first set of filter coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set, determine the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set, wherein the first chromaticity sample set is any one of at least two chromaticity sample sets.
[0171] Each chroma sample set is filtered based on its corresponding set of filtering coefficients to obtain the first filtered chroma sample value of the chroma sample in each chroma sample set, thereby improving the filtering performance of the CCCM model and thus improving the encoding and decoding performance.
[0172] In some embodiments, the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set is determined based on the first set of filtering coefficients corresponding to the first chromaticity sample set and the downsampled luminance sample value in the first luminance sample set.
[0173] In some embodiments, the method may further include: averaging the first chromaticity sample value and the first filtered chromaticity sample value of the chromaticity samples in the first chromaticity sample set to obtain the second filtered chromaticity sample value of the chromaticity samples in the first chromaticity sample set. That is, using the luminance sample value of the first luminance sample set corresponding to the first chromaticity sample set as input, filtering calculations are performed using a set of filtering coefficients corresponding to the first chromaticity sample set to determine the first filtered chromaticity sample value corresponding to the chromaticity sample in the first chromaticity sample set, i.e., the chromaticity sample value after CCCM filtering of the first chromaticity sample set. Further, the chromaticity sample values after CCCM filtering at the same position can be averaged with the first chromaticity sample value to obtain the second filtered chromaticity sample value corresponding to the first chromaticity sample set.
[0174] In some embodiments, based on the first set of filtering coefficients and the downsampled luminance sample values of the first luminance sample set, the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set is determined; the first chromaticity sample value and the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set are averaged to obtain the second filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set.
[0175] In some embodiments, the method may further include: determining the chromaticity prediction value or chromaticity reconstruction value of the chromaticity samples in the first chromaticity sample set based on the first filtered chromaticity sample value of the chromaticity samples in the first chromaticity sample set.
[0176] In other embodiments, the method may further include: determining a chromaticity prediction value or a chromaticity reconstruction value of the chromaticity samples in the first chromaticity sample set based on a first filtered chromaticity sample value and a first chromaticity sample value of the chromaticity samples in the first chromaticity sample set.
[0177] In other words, the multi-model filtering method provided in this application can be applied to the prediction stage to determine the chromaticity prediction value, or it can be applied to the loop filtering stage to determine the chromaticity reconstruction value.
[0178] For example, the method may further include: determining the chromaticity reconstruction value of the first chromaticity sample in the first chromaticity sample set based on the first filtered chromaticity sample value and the first chromaticity sample value of the first chromaticity sample in the first chromaticity sample set, wherein the first chromaticity sample is a chromaticity sample in which the difference between the first filtered chromaticity sample value and the first chromaticity sample value is less than or equal to a third threshold, and the first chromaticity sample may also be a chromaticity sample in which the difference between the first filtered chromaticity sample value and the first chromaticity sample value is less than the third threshold.
[0179] For example, the method may further include: when the first condition is met, determining the chromaticity reconstruction value of the first chromaticity sample in the first chromaticity sample set based on the first filtered chromaticity sample value and the first chromaticity sample value of the first chromaticity sample in the first chromaticity sample set, wherein the first chromaticity sample is a chromaticity sample in which the difference between the first filtered chromaticity sample value and the first chromaticity sample value is less than or equal to a third threshold, and the first chromaticity sample may also be a chromaticity sample in which the difference between the first filtered chromaticity sample value and the first chromaticity sample value is less than the third threshold.
[0180] The first chromaticity sample can be referred to as a normal or good sample with the expected CCCM filtering performance. CCCM filtering on the first chromaticity sample can be understood as using the CCCM-filtered first chromaticity sample value and the first chromaticity sample value to determine the chromaticity reconstructed value or chromaticity predicted value, which can ensure the CCCM filtering performance.
[0181] The third threshold is used to distinguish between normal and abnormal samples. The third threshold can be a fixed value, determined based on the feature parameters of the block to be filtered, or determined by syntax elements in the bitstream. For example, the third threshold can be set to 32, or it can be any other value; no restrictions are placed here.
[0182] In some embodiments, the method may further include: if the first condition is not met, determining the chromaticity reconstructed value of the chromaticity sample in the first chromaticity sample set based on the first chromaticity sample value of the chromaticity sample in the first chromaticity sample set. This can be understood as determining the chromaticity reconstructed value using the chromaticity sample value before CCCM filtering if the first condition is not met; that is, not performing CCCM filtering on the chromaticity samples in the first chromaticity sample set, ignoring the first filtered chromaticity sample value after CCCM filtering corresponding to the first chromaticity sample set, or ignoring the first filtered chromaticity sample value after CCCM filtering of abnormal samples.
[0183] The first condition is a detection condition for CCCM filtering performance. In some embodiments, the first condition may include a number of second chromaticity samples in a first chromaticity sample set that is less than or equal to a first number. The second chromaticity sample can be a chromaticity sample whose difference from the first filtered chromaticity sample value is greater than a third threshold. The second chromaticity sample can be referred to as an abnormal sample or a bad sample that does not possess the expected CCCM filtering performance.
[0184] In other embodiments, the first condition may also include that the number of second chromaticity samples in the first chromaticity sample set is less than the first number.
[0185] For example, the first quantity can be 0. When the first chromaticity sample set includes at least one second chromaticity sample, then all chromaticity samples in the first chromaticity sample set are not subjected to CCCM filtering; that is, the chromaticity reconstruction value of all chromaticity samples in the first chromaticity sample set is determined based on the first chromaticity sample value of all chromaticity samples in the first chromaticity sample set. Otherwise, CCCM filtering is performed on all chromaticity samples in the first chromaticity sample set; that is, the chromaticity reconstruction value of the first chromaticity sample in the first chromaticity sample set is determined based on the first filtered chromaticity sample value and the first chromaticity sample value of the first chromaticity sample in the first chromaticity sample set.
[0186] For example, the first quantity can also be greater than 0, such as 1, 2, 3, 4, 8, etc. When there is at least one second chromaticity sample in the first chromaticity sample set, CCCM filtering can be performed on the first chromaticity sample in the first chromaticity sample set. That is, the chromaticity reconstruction value of the first chromaticity sample in the first chromaticity sample set is determined based on the first filtered chromaticity sample value and the first chromaticity sample value of the first chromaticity sample in the first chromaticity sample set. The second chromaticity sample in the first chromaticity sample set is not subject to CCCM filtering. That is, the chromaticity reconstruction value of all chromaticity samples in the second chromaticity sample set is determined based on the first chromaticity sample value of the second chromaticity sample in the first chromaticity sample set.
[0187] In some embodiments, the first condition may further include: the number of second chroma samples in the tenth chroma sample set is less than or equal to a second number; wherein, the tenth chroma sample set is part or all of the chroma sample sets other than the first chroma sample set in at least two chroma sample sets. That is, the first condition includes: the number of second chroma samples in the first chroma sample set is less than or equal to a first number, and the number of second chroma samples in the tenth chroma sample set is less than or equal to a second number.
[0188] In other embodiments, the first condition may also include that the number of second chromaticity samples in the tenth chromaticity sample set is less than the second number.
[0189] In other words, whether to perform CCCM filtering on the first chroma sample set depends not only on the abnormal samples in the first chroma sample set, but also on the abnormal samples in other chroma sample sets in the current block.
[0190] The first quantity and the second quantity can be the same or different. For example, the first quantity and the second quantity can be 0, that is, when there is at least one second chromaticity sample in the first chromaticity sample set or the tenth chromaticity sample set, then all chromaticity samples in the first chromaticity sample set are not subject to CCCM filtering. Otherwise, CCCM filtering is performed on all chromaticity samples in the first chromaticity sample set.
[0191] For example, at least two chroma sample sets include a first chroma sample set and a tenth chroma sample set. When there is at least one outlier sample in the first chroma sample set or the tenth chroma sample set, the first chroma sample set and the second chroma sample set are not subjected to CCCM filtering, that is, the entire block is not subjected to CCCM filtering; when there are no outlier samples in the first chroma sample set and the tenth chroma sample set, the first chroma sample set and the second chroma sample set are subjected to CCCM filtering, that is, the entire block is subjected to CCCM filtering.
[0192] For example, the first quantity and the second quantity can also be greater than 0. When there is at least one second chromaticity sample in the first chromaticity sample set or the tenth chromaticity sample set, CCCM filtering can be performed on the first chromaticity sample in the first chromaticity sample set, and CCCM filtering is not performed on the second chromaticity sample in the first chromaticity sample set.
[0193] For example, at least two chromaticity sample sets include a first chromaticity sample set and a tenth chromaticity sample set. When there is at least one abnormal sample in the first chromaticity sample set, the normal samples in the first chromaticity sample set can be filtered, while the abnormal samples in the first chromaticity sample set are not filtered. When there is at least one abnormal sample in the tenth chromaticity sample set, the normal samples in the tenth chromaticity sample set can be filtered, while the abnormal samples in the tenth chromaticity sample set are not filtered.
[0194] In some embodiments, determining the chromaticity reconstruction value of the first chromaticity sample in the first chromaticity sample set based on the first filtered chromaticity sample value and the first chromaticity sample value in the first chromaticity sample set may include: summing the difference between the first filtered chromaticity sample value and the first chromaticity sample value and the second chromaticity sample value in the first chromaticity sample set to determine the chromaticity reconstruction value of the first chromaticity sample in the first chromaticity sample set.
[0195] In some embodiments, determining the chromaticity reconstruction value of a first chromaticity sample in a first chromaticity sample set based on a first filtered chromaticity sample value and a first chromaticity sample value in a first chromaticity sample set may include: averaging the first chromaticity sample value and the first filtered chromaticity sample value in the first chromaticity sample set to obtain a second filtered chromaticity sample value in the first chromaticity sample set; and summing the difference between the second filtered chromaticity sample value and the first chromaticity sample value in the first chromaticity sample set and the second chromaticity sample value to determine the chromaticity reconstruction value of the first chromaticity sample in the first chromaticity sample set.
[0196] The first chromaticity sample value and the second chromaticity sample value are output by different modules. For example, the second chromaticity sample value can be the chromaticity sample value output by any of the following modules: DB filter module, SAO filter module, CCSAO filter module, BIF filter module, ALF filter module, etc.
[0197] For example, if the current block uses ALF-CCCM for filtering, the difference between the first filtered chromaticity sample value after ALF-CCCM filtering and the first chromaticity sample value output by the previous module is used to obtain the compensation value after ALF-CCCM filtering. This compensation value is then summed with the second chromaticity sample value output by the ALF / CCALF filter and output to the reconstructed image.
[0198] By adopting the above technical solution, when the current block uses the cross-component convolutional model (CCCM) for filtering, the current block directly uses a multi-model scheme to filter the chroma samples of the current block. Specifically, the chroma samples in the current block are accurately divided into at least two chroma sample sets according to the sample division threshold of the current block; a corresponding set of filtering coefficients is used for filtering different chroma sample sets of the current block, thereby improving the filtering performance of the CCCM model and thus improving the decoding performance.
[0199] In another embodiment of this application, referring to FIG7, a flowchart of an encoding method provided by an embodiment of this application is shown. As shown in FIG7, the method may include:
[0200] S701: When filtering is performed using a cross-component convolution model in the current block, determine at least two chroma sample sets for the current block based on the sample partitioning threshold of the current block.
[0201] S702: Based on at least two sets of chroma samples and at least two sets of corresponding luminance samples of the current block, determine at least two sets of filter coefficients for the current block;
[0202] S703: Based on the first set of filter coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set, determine the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set, wherein the first chromaticity sample set is any one of at least two chromaticity sample sets.
[0203] It should be noted that the encoding method provided in this application embodiment is applied to the encoder. Specifically, the encoding method can refer to a filtering method, and more specifically, it can refer to a filtering method based on multiple cross-component convolution models (which can be referred to as a multi-model filtering method). Using this method can improve the filtering performance of the CCCM model, thereby improving the encoding performance.
[0204] For example, the multi-model filtering method provided in this application is applicable to CCCM filters. Specifically, it can be applied to LF-CCCM filters, ALF-CCCM filters, etc. in the loop filtering stage, and also to Inter-CCCM filters, Intra-CCCM filters, etc. in the prediction stage.
[0205] In some embodiments, the method may further include: performing a first processing on the brightness parameters of the current block to determine M first thresholds for the current block, where M is a positive integer.
[0206] In some embodiments, determining at least two chromaticity sample sets for the current block based on the sample partitioning threshold of the current block includes: determining M+1 chromaticity sample sets for the current block based on M first thresholds and the luminance parameters of the current block.
[0207] In some embodiments, the method may further include: when M is 1, determining M+1 chromaticity sample sets for the current block based on M first thresholds and the luminance parameters of the current block, which may include: assigning chromaticity samples in the current block whose luminance parameters are less than the first thresholds to a second chromaticity sample set; and assigning chromaticity samples in the current block whose luminance parameters are greater than or equal to the first thresholds to a third chromaticity sample set.
[0208] In some embodiments, the brightness parameters of the current block are first processed to determine M first thresholds of the current block, including: determining a first value range of the brightness parameters of the current block; and determining M first thresholds of the current block based on the first value range of the brightness parameters of the current block and the number of first sub-ranges.
[0209] In some embodiments, the method further includes: determining the number of first sub-intervals based on the brightness parameters of the current block.
[0210] In some embodiments, the number of the first sub-intervals is M+1.
[0211] In some embodiments, the method further includes: performing a second processing on the chromaticity parameters of the current block to determine N second thresholds for the current block, where N is a positive integer.
[0212] In some embodiments, determining at least two chroma sample sets for the current block based on the sample partitioning threshold of the current block includes: determining N+1 chroma sample sets for the current block based on N second thresholds and the chroma parameters of the current block.
[0213] In some embodiments, when N is 1, based on N second thresholds and the chromaticity parameters of the current block, N+1 chromaticity sample sets of the current block are determined, including: assigning chromaticity samples in the current block whose chromaticity parameters are less than the second thresholds to the fourth chromaticity sample set; and assigning chromaticity samples in the current block whose chromaticity parameters are greater than or equal to the second thresholds to the fifth chromaticity sample set.
[0214] In some embodiments, the method further includes: performing a second processing on the chromaticity parameters of the current block to determine N second thresholds for the current block, where N is a positive integer.
[0215] In some embodiments, determining at least two chromaticity sample sets for the current block based on the sample partitioning threshold of the current block may include: determining a (M+1)×(N+1) chromaticity sample set for the current block based on M first thresholds, N second thresholds, the luminance parameter of the current block, and the chromaticity parameter of the current block.
[0216] In some embodiments, when M and N are both 1, based on M first thresholds, N second thresholds, the luminance parameter of the current block, and the chrominance parameter of the current block, a (M+1)×(N+1) chrominance sample set for the current block is determined, including: assigning chrominance samples in the current block whose luminance parameter values are less than the first threshold and whose chrominance parameter values are less than the second threshold to the sixth chrominance sample set; assigning chrominance samples in the current block whose luminance parameter values are greater than or equal to the first threshold and whose chrominance parameter values are less than the second threshold to the seventh chrominance sample set; assigning chrominance samples in the current block whose luminance parameter values are less than the first threshold and whose chrominance parameter values are greater than or equal to the second threshold to the eighth chrominance sample set; and assigning chrominance samples in the current block whose luminance parameter values are greater than or equal to the first threshold and whose chrominance parameter values are greater than or equal to the second threshold to the ninth chrominance sample set.
[0217] In some embodiments, a second processing is performed on the chroma parameters of the current block to determine N second thresholds for the current block, including: determining a second value range of the chroma parameters of the current block; and determining N second thresholds for the current block based on the number of the second value range and the number of the second sub-ranges of the chroma parameters of the current block.
[0218] In some embodiments, the method further includes: determining the number of second sub-intervals based on the chromaticity parameters of the current block.
[0219] In some embodiments, the number of second sub-intervals is N+1.
[0220] In some embodiments, the brightness parameter includes a brightness sample value or a brightness residual value.
[0221] In some embodiments, the luminance sample value is the downsampled luminance sample value, and the luminance residual value is the downsampled luminance residual value.
[0222] In some embodiments, the luminance sample value includes one of the following: luminance sample value before loop filtering, luminance sample value before deblocking filtering, luminance sample value before sample offset compensation, luminance sample value before cross-component sample offset compensation, luminance sample value before adaptive loop filtering, luminance sample value before cross-component convolution model filtering of adaptive filter, and luminance sample value after cross-component convolution model filtering of adaptive filter.
[0223] In some embodiments, the chromaticity parameter includes chromaticity sample values or chromaticity residual values.
[0224] In some embodiments, the chroma sample values include one of the following: chroma sample values before loop filtering, chroma sample values before deblocking filtering, chroma sample values before sample offset compensation filtering, chroma sample values before cross-component sample offset compensation filtering, chroma sample values before adaptive loop filtering, and chroma sample values before cross-component convolution model filtering of an adaptive filter.
[0225] In some embodiments, determining at least two sets of filtering coefficients for the current block based on at least two chroma sample sets and at least two corresponding luminance sample sets includes: determining a first set of filtering coefficients corresponding to the first chroma sample set based on the first chroma sample value of the chroma sample in the first chroma sample set and the luminance sample value of the luminance sample in the first luminance sample set, wherein the position of the chroma sample in the first chroma sample set corresponds to the position of the luminance sample in the first luminance sample set.
[0226] In some embodiments, the method may further include: averaging the first chromaticity sample value and the first filtered chromaticity sample value of the chromaticity samples in the first chromaticity sample set to obtain the second filtered chromaticity sample value of the chromaticity samples in the first chromaticity sample set.
[0227] In some embodiments, the method further includes: when a first condition is met, determining a chromaticity reconstruction value of a first chromaticity sample in the first chromaticity sample set based on a first filtered chromaticity sample value and a first chromaticity sample value in the first chromaticity sample set, wherein the first chromaticity sample is a chromaticity sample whose difference between the first filtered chromaticity sample value and the first chromaticity sample value is less than or equal to a third threshold.
[0228] In some embodiments, the method further includes: if the first condition is not met, determining the chromaticity reconstruction value of the chromaticity sample in the first chromaticity sample set based on the first chromaticity sample value of the chromaticity sample in the first chromaticity sample set.
[0229] In some embodiments, the first condition includes: the number of second chromaticity samples in the first chromaticity sample set is less than or equal to a first number; wherein, the second chromaticity sample is a chromaticity sample whose difference between the first filtered chromaticity sample value and the first chromaticity sample value is greater than a third threshold.
[0230] In some embodiments, the first condition further includes: the number of second chromaticity samples in the tenth chromaticity sample set is less than or equal to the second number; wherein, the tenth chromaticity sample set is a subset or all of the chromaticity sample sets other than the first chromaticity sample set in at least two chromaticity sample sets.
[0231] In some embodiments, determining the chromaticity reconstruction value of a first chromaticity sample in a first chromaticity sample set based on a first filtered chromaticity sample value and a first chromaticity sample value in a first chromaticity sample set includes: averaging the first chromaticity sample value and the first filtered chromaticity sample value in the first chromaticity sample set to obtain a second filtered chromaticity sample value in the first chromaticity sample set; and summing the difference between the second filtered chromaticity sample value and the first chromaticity sample value in the first chromaticity sample set and the second chromaticity sample value to determine the chromaticity reconstruction value of the first chromaticity sample in the first chromaticity sample set.
[0232] By adopting the above technical solution, when the current block uses the cross-component convolutional model (CCCM) for filtering, the current block directly uses a multi-model scheme to filter the chroma samples of the current block. Specifically, the chroma samples in the current block are accurately divided into at least two chroma sample sets according to the sample division threshold of the current block; and a corresponding set of filtering coefficients are used to filter different chroma sample sets of the current block, thereby improving the filtering performance of the CCCM model and thus improving the coding performance.
[0233] It should be noted that the encoding end and the decoding end perform the same filtering method in the embodiments of this application. For more detailed embodiments of the encoding end and the technical effects of the embodiments, please refer to the technical effects of the corresponding technical solutions in the decoding end. The encoding end will not be described in detail here.
[0234] In another embodiment of this application, based on the encoding and decoding method of the foregoing embodiments, a technical solution for optimizing CCCM filters is proposed. By improving the multi-model scheme of CCCM, when the current block is filtered using the cross-component convolution model (CCCM), the current block is directly filtered using the multi-model scheme. Specifically, the block to be filtered is accurately divided, and appropriate filtering coefficients are used to filter different samples within the block, thereby improving the CCCM filtering performance.
[0235] In one specific embodiment, at the encoding end, this application optimizes the sixth step in the CCCM filter coefficient derivation process and the seventh step in the ALF-CCCM filter process. For example, at the encoding end, the multi-model scheme provided by this application embodiment may include at least:
[0236] Example 1
[0237] CCCM filter coefficient derivation:
[0238] Step 6: Calculate the average value of the downsampled luminance sample values of the input sample to be filtered. All samples within the sample block whose downsampled luminance sample values at the corresponding chromaticity position are less than this average value are used to solve for filter coefficient group 1. Using the downsampled luminance sample as input and the chromaticity sample output from the previous module as output, construct the Wiener-Hope equation and solve for filter coefficient group 1 of the CCCM filter used by the sample. All samples within the sample block whose downsampled luminance sample values at the corresponding chromaticity position are greater than or equal to this average value are used to solve for filter coefficient group 2. Using the downsampled luminance sample as input and the chromaticity sample output from the previous module as output, construct the Wiener-Hope equation and solve for filter coefficient group 2 of the CCCM filter used by the sample.
[0239] ALF-CCCM filtering process:
[0240] Step 7: Compare the chroma sample values after ALF-CCCM filtering with the chroma sample values output by the previous stage. If there is at least one sample in the block whose difference between the filtered chroma sample value and the chroma sample value output by the previous stage is greater than a preset threshold, then the entire block will not be CCCM filtered.
[0241] Example 2
[0242] The seventh step of the ALF-CCCM filtering process differs from that in Example 1.
[0243] ALF-CCCM filtering process:
[0244] Step 7: Compare the chromaticity sample values after ALF-CCCM filtering with the chromaticity sample values output from the previous stage. If the difference between the chromaticity sample values after ALF-CCCM filtering and the chromaticity sample values output from the previous stage in some samples used to calculate filter array 1 is greater than a preset threshold, then these samples will not be subjected to ALF-CCCM filtering. If the difference between the chromaticity sample values after ALF-CCCM filtering and the chromaticity sample values output from the previous stage in some samples used to calculate filter array 2 is greater than a preset threshold, then these samples will not be subjected to ALF-CCCM filtering.
[0245] Example 3
[0246] The number of models is no longer limited to 2, but can be N. N intervals are determined based on the value range of the downsampled brightness samples of all input samples within the block, and all samples in each interval are filtered using the same filter coefficients. The difference lies in step six of the CCCM filter coefficient solution process compared to Examples 1 and 2.
[0247] CCCM filter coefficient derivation:
[0248] Step 6: Determine the value range [A, B] of the downsampled luminance sample value of the input sample to be filtered, and divide this range into N intervals. All samples whose downsampled luminance sample values at the corresponding chromaticity positions within the sample block to be filtered fall into the i-th interval are used to solve for the filter coefficient set i. Using the downsampled luminance sample as input and the chromaticity sample output from the previous module as output, construct the Wiener-Hoppt equation and solve for the filter coefficient set i of the CCCM filter used by the sample. Here, N can be a fixed constant or adaptively determined according to the sample distribution of the sample block.
[0249] ALF-CCCM filtering process:
[0250] Step 7: Compare the chroma sample values after ALF-CCCM filtering with the chroma sample values output by the previous stage. If there is at least one chroma sample in the block whose difference between the filtered chroma sample value and the output chroma sample value of the previous stage is greater than a preset threshold, then the entire block will not be filtered by ALF-CCCM.
[0251] Alternatively, the ALF-CCCM filtering process:
[0252] Step 7: Compare the chromaticity sample values after ALF-CCCM filtering with the chromaticity sample values output from the previous stage. If the difference between the chromaticity sample values after ALF-CCCM filtering and the chromaticity sample values output from the previous stage in some chromaticity samples used to calculate filter array i is greater than a preset threshold, then these chromaticity samples will not be subjected to ALF-CCCM filtering.
[0253] Example 4
[0254] The multi-model approach is no longer limited to two models; it can be N. N intervals are determined based on the value range of all chromaticity samples within a block, and all samples in each interval are filtered using the same filter coefficients. Unlike Example 3, the classification uses the chromaticity sample values of the chromaticity samples. This is specifically reflected in step six of the CCCM filter coefficient calculation process.
[0255] CCCM filter coefficient derivation:
[0256] Step 6: Determine the value range [A, B] of the chromaticity sample values of the sample to be filtered, and divide this range into N intervals. All samples within the sample block whose chromaticity sample values fall within the i-th interval are used to solve for the filter coefficient set i. Using the downsampled luminance sample as input and the chromaticity sample output from the previous module as output, construct the Wiener-Hopp equation and solve for the filter coefficient set i of the CCCM filter used by the sample. Here, N can be a fixed constant or adaptively determined based on the sample distribution of the sample block.
[0257] ALF-CCCM filtering process:
[0258] Step 7: Compare the chroma sample values after ALF-CCCM filtering with the chroma sample values output by the previous stage. If there is at least one chroma sample in the block whose difference between the filtered chroma sample value and the output chroma sample value of the previous stage is greater than a preset threshold, then the entire block will not be filtered by ALF-CCCM.
[0259] Alternatively, the ALF-CCCM filtering process:
[0260] Step 7: Compare the chromaticity sample values after ALF-CCCM filtering with the chromaticity sample values output from the previous stage. If the difference between the chromaticity sample values after ALF-CCCM filtering and the chromaticity sample values output from the previous stage in some chromaticity samples used to calculate filter array i is greater than a preset threshold, then these chromaticity samples will not be subjected to ALF-CCCM filtering.
[0261] In another specific embodiment, no changes are made to the syntax and semantics or the bitstream parsing process at the decoding end. The decoding process only optimizes step six in the CCCM filter coefficient derivation process and step seven in the ALF-CCCM filter process. For example, at the decoding end, the multi-model scheme provided in this application embodiment may include at least:
[0262] Example 1
[0263] CCCM filter coefficient derivation:
[0264] Step 6: Calculate the average value of the downsampled luminance sample values of the input sample to be filtered. All samples within the sample block whose downsampled luminance sample values at the corresponding chromaticity position are less than this average value are used to solve for filter coefficient group 1. Using the downsampled luminance sample as input and the chromaticity sample output from the previous module as output, construct the Wiener-Hope equation and solve for filter coefficient group 1 of the CCCM filter used by the sample. All samples within the sample block whose downsampled luminance sample values at the corresponding chromaticity position are greater than or equal to this average value are used to solve for filter coefficient group 2. Using the downsampled luminance sample as input and the chromaticity sample output from the previous module as output, construct the Wiener-Hope equation and solve for filter coefficient group 2 of the CCCM filter used by the sample.
[0265] ALF-CCCM filtering process:
[0266] Step 7: Compare the chroma sample values after ALF-CCCM filtering with the chroma sample values output by the previous stage. If there is at least one sample in the block whose difference between the filtered chroma sample value and the chroma sample value output by the previous stage is greater than a preset threshold, then the entire block will not be CCCM filtered.
[0267] Example 2
[0268] The seventh step of the ALF-CCCM filtering process differs from that in Example 1.
[0269] ALF-CCCM filtering process:
[0270] Step 7: Compare the chromaticity sample values after ALF-CCCM filtering with the chromaticity sample values output from the previous stage. If the difference between the chromaticity sample values after ALF-CCCM filtering and the chromaticity sample values output from the previous stage in some samples used to calculate filter array 1 is greater than a preset threshold, then these samples will not be subjected to ALF-CCCM filtering. If the difference between the chromaticity sample values after ALF-CCCM filtering and the chromaticity sample values output from the previous stage in some samples used to calculate filter array 2 is greater than a preset threshold, then these samples will not be subjected to ALF-CCCM filtering.
[0271] Example 3
[0272] The number of models is no longer limited to 2, but can be N. N intervals are determined based on the value range of the downsampled brightness samples of all input samples within the block, and all samples in each interval are filtered using the same filter coefficients. The difference lies in step six of the CCCM filter coefficient solution process compared to Examples 1 and 2.
[0273] CCCM filter coefficient derivation:
[0274] Step 6: Determine the value range [A, B] of the downsampled luminance sample value of the input sample to be filtered, and divide this range into N intervals. All samples whose downsampled luminance sample values at the corresponding chromaticity positions within the sample block to be filtered fall into the i-th interval are used to solve for the filter coefficient set i. Using the downsampled luminance sample as input and the chromaticity sample output from the previous module as output, construct the Wiener-Hoppt equation and solve for the filter coefficient set i of the CCCM filter used by the sample. Here, N can be a fixed constant or adaptively determined according to the sample distribution of the sample block.
[0275] ALF-CCCM filtering process:
[0276] Step 7: Compare the chroma sample values after ALF-CCCM filtering with the chroma sample values output by the previous stage. If there is at least one chroma sample in the block whose difference between the filtered chroma sample value and the output chroma sample value of the previous stage is greater than a preset threshold, then the entire block will not be filtered by ALF-CCCM.
[0277] Alternatively, the ALF-CCCM filtering process:
[0278] Step 7: Compare the chromaticity sample values after ALF-CCCM filtering with the chromaticity sample values output from the previous stage. If the difference between the chromaticity sample values after ALF-CCCM filtering and the chromaticity sample values output from the previous stage in some chromaticity samples used to calculate filter array i is greater than a preset threshold, then these chromaticity samples will not be subjected to ALF-CCCM filtering.
[0279] Example 4
[0280] The multi-model approach is no longer limited to two models; it can be N. N intervals are determined based on the value range of all chromaticity samples within a block, and all samples in each interval are filtered using the same filter coefficients. Unlike Example 3, the classification uses the chromaticity sample values of the chromaticity samples. This is specifically reflected in step six of the CCCM filter coefficient calculation process.
[0281] CCCM filter coefficient derivation:
[0282] Step 6: Determine the value range [A, B] of the chroma sample values of the sample to be filtered, and divide this range into N intervals. All samples whose chroma sample values within the sample block to be filtered fall into the i-th interval are used to solve for the filter coefficient set i. Using the downsampled luminance sample of the sample as input and the chroma sample output from the previous module as output, construct the Wiener-Hopp equation and solve for the filter coefficient set i of the CCCM filter used by the sample. Here, N can be a fixed constant or can be adaptively determined according to the sample distribution of the sample block.
[0283] ALF-CCCM filtering process:
[0284] Step 7: Compare the chroma sample values after ALF-CCCM filtering with the chroma sample values output by the previous stage. If there is at least one chroma sample in the block whose difference between the filtered chroma sample value and the output chroma sample value of the previous stage is greater than a preset threshold, then the entire block will not be filtered by ALF-CCCM.
[0285] Alternatively, the ALF-CCCM filtering process:
[0286] Step 7: Compare the chromaticity sample values after ALF-CCCM filtering with the chromaticity sample values output from the previous stage. If the difference between the chromaticity sample values after ALF-CCCM filtering and the chromaticity sample values output from the previous stage in some chromaticity samples used to calculate filter array i is greater than a preset threshold, then these chromaticity samples will not be subjected to ALF-CCCM filtering.
[0287] According to Embodiment 2 of this application, under ECM-Master version 0414 (versions prior to ECM-17), the following performance data are available compared to related technologies:
[0288] Table 4
[0289] Table 5
[0290] Table 6
[0291] As can be seen from the table above, Example 2 has improved filtering performance compared to the current version of ECM.
[0292] This application proposes a CCCM filter optimization technique, which removes the single-model scheme of CCCM and directly adopts a multi-model scheme to filter the block to be filtered. The block to be filtered is accurately divided, and appropriate filtering coefficients are used to filter different chromaticity samples within the current block, thereby improving the CCCM filtering performance.
[0293] In another embodiment of this application, based on the same inventive concept as the foregoing embodiments, referring to FIG8, a schematic diagram of the composition structure of an encoder provided in an embodiment of this application is shown. As shown in FIG8, the encoder 80 may include a first determining unit 801 and a first filtering unit 802; wherein:
[0294] The first determining unit is configured to determine at least two chroma sample sets of the current block based on the sample partitioning threshold of the current block when filtering is performed using a cross-component convolution model in the current block.
[0295] The first determining unit is further configured to determine at least two sets of filter coefficients for the current block based on at least two sets of chroma samples and at least two sets of corresponding luminance samples.
[0296] The first filtering unit is configured to determine the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set based on the first set of filtering coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set. The first chromaticity sample set is any one of at least two chromaticity sample sets.
[0297] In some embodiments, the encoder 80 may further include an encoding unit 803 configured to encode the information to be encoded to obtain a bitstream.
[0298] Understandably, each functional unit of the encoder also performs the encoding method of any of the foregoing embodiments.
[0299] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.
[0300] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0301] Therefore, embodiments of this application provide a computer-readable storage medium applied to an encoder 80, the computer-readable storage medium storing a computer program that, when executed by a first processor, implements the method of any of the foregoing embodiments.
[0302] This application provides a computer-readable storage medium that stores a bitstream generated by the encoding method described above.
[0303] Based on the composition of encoder 80 and the computer-readable storage medium, see Figure 9, which shows a schematic diagram of the specific hardware structure of encoder 80 provided in this embodiment. As shown in Figure 9, encoder 80 may include: a first communication interface 901, a first memory 902, and a first processor 903; the various components are coupled together through a first bus system 904. It is understood that the first bus system 904 is used to realize the connection and communication between these components. In addition to a data bus, the first bus system 904 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as the first bus system 904 in Figure 9.
[0304] The first communication interface 901 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;
[0305] The first memory 902 is used to store computer programs that can run on the first processor 903;
[0306] The first processor 903 is used to execute the following when running computer programs:
[0307] When filtering is performed using a cross-component convolution model in the current block, at least two chroma sample sets for the current block are determined based on the sample partitioning threshold of the current block.
[0308] Based on at least two sets of chroma samples and at least two sets of corresponding luminance samples of the current block, determine at least two sets of filter coefficients for the current block;
[0309] Based on the first set of filter coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set, the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set is determined. The first chromaticity sample set is any one of at least two chromaticity sample sets.
[0310] It is understood that the first memory 902 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The first memory 902 of the system and method described in this application is intended to include, but is not limited to, these and any other suitable types of memory.
[0311] The first processor 903 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the first processor 903 or by instructions in software form. The first processor 903 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the first memory 902. The first processor 903 reads the information in the first memory 902 and, in conjunction with its hardware, completes the steps of the above method.
[0312] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of this application, or combinations thereof. For software implementation, the technology of this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions of this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.
[0313] Alternatively, as another embodiment, the first processor 903 is also configured to execute any of the methods in the foregoing embodiments when running a computer program.
[0314] This embodiment provides an encoder in which, when the current block is filtered using a cross-component convolutional model (CCCM), the current block directly uses a multi-model scheme to filter the chroma samples of the current block. Specifically, the chroma samples in the current block are accurately divided into at least two chroma sample sets according to the sample division threshold of the current block; a corresponding set of filtering coefficients is used to filter different chroma sample sets of the current block, thereby improving the filtering performance of the CCCM model and thus improving the encoding performance.
[0315] In another embodiment of this application, based on the same inventive concept as the foregoing embodiments, referring to FIG10, a schematic diagram of the composition structure of a decoder 100 provided in an embodiment of this application is shown. As shown in FIG10, the decoder 100 may include a second determining unit 1001 and a second filtering unit 1002; wherein:
[0316] The second determining unit is configured to determine at least two chroma sample sets of the current block based on the sample partitioning threshold of the current block when the current block is filtered using a cross-component convolution model.
[0317] The second determining unit is further configured to determine at least two sets of filter coefficients for the current block based on at least two sets of chroma samples and at least two sets of corresponding luminance samples.
[0318] The second filtering unit is configured to determine the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set based on the first set of filtering coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set. The first chromaticity sample set is any one of at least two chromaticity sample sets.
[0319] In some embodiments, the decoder 100 may further include a decoding unit 1003 configured to parse the bitstream to obtain decoding information.
[0320] Understandably, each functional unit of the decoder also performs the decoding method described in any of the foregoing embodiments.
[0321] Based on the composition of the decoder 100 and the computer-readable storage medium, Figure 11 illustrates a schematic diagram of the specific hardware structure of the decoder 100 provided in this embodiment. As shown in Figure 11, the decoder 100 may include: a second communication interface 1101, a second memory 1102, and a second processor 1103; the various components are coupled together through a second bus system 1104. It is understood that the second bus system 1104 is used to realize the connection and communication between these components. In addition to a data bus, the second bus system 1104 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as the second bus system 1104 in Figure 11.
[0322] The second communication interface 1101 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;
[0323] The second memory 1102 is used to store computer programs that can run on the second processor 1103;
[0324] The second processor 1103 is used to execute the following when running computer programs:
[0325] When filtering is performed using a cross-component convolution model in the current block, at least two chroma sample sets for the current block are determined based on the sample partitioning threshold of the current block.
[0326] Based on at least two sets of chroma samples and at least two sets of corresponding luminance samples of the current block, determine at least two sets of filter coefficients for the current block;
[0327] Based on the first set of filter coefficients corresponding to the first chromaticity sample set and the luminance sample value of the luminance sample in the first luminance sample set, the first filtered chromaticity sample value of the chromaticity sample in the first chromaticity sample set is determined. The first chromaticity sample set is any one of at least two chromaticity sample sets.
[0328] Alternatively, as another embodiment, the second processor 1103 is also configured to execute any of the methods in the foregoing embodiments when running a computer program.
[0329] It is understood that the second memory 1102 has similar hardware functions to the first memory 902, and the second processor 1103 has similar hardware functions to the first processor 903; these will not be described in detail here.
[0330] This embodiment provides a decoder in which, when the current block is filtered using a cross-component convolutional model (CCCM), the current block directly uses a multi-model scheme to filter the chroma samples of the current block. Specifically, the chroma samples in the current block are accurately divided into at least two chroma sample sets according to the sample division threshold of the current block; a corresponding set of filtering coefficients is used to filter the different chroma sample sets of the current block, thereby improving the filtering performance of the CCCM model and thus improving the decoding performance.
[0331] In another embodiment of this application, referring to FIG12, a schematic diagram of the composition structure of an encoding / decoding system provided in an embodiment of this application is shown. As shown in FIG12, the encoding / decoding system 120 may include an encoder 1201 and a decoder 1202.
[0332] In this embodiment, encoder 1201 can be any of the encoders described in the foregoing embodiments, and decoder 1202 can be any of the decoders described in the foregoing embodiments.
[0333] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0334] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0335] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict. The features disclosed in the several product embodiments provided in this application can be arbitrarily combined to obtain new product embodiments without conflict. The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined to obtain new method embodiments or device embodiments without conflict.
[0336] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims. Industrial applicability
[0337] This application provides an encoding / decoding method, encoder, decoder, and storage medium. Whether at the encoding or decoding end, when the current block uses a cross-component convolutional model (CCCM) for filtering, the current block directly uses a multi-model scheme to filter the chroma samples of the current block. Specifically, based on the sample partitioning threshold of the current block, the chroma samples in the current block are accurately divided into at least two chroma sample sets; a corresponding set of filtering coefficients is used for filtering different chroma sample sets of the current block, thereby improving the filtering performance of the CCCM model and thus improving the encoding / decoding performance.
Claims
1. A decoding method applied to a decoder, the method comprising: in a case where a current block is filtered using a cross-component convolution model, determining at least two sets of chroma samples of the current block based on a sample division threshold of the current block; determining at least two groups of filter coefficients of the current block based on the at least two sets of chroma samples of the current block and at least two sets of corresponding luma samples of the current block; determining a first filtered chroma sample value of a chroma sample in a first set of chroma samples based on a first group of filter coefficients corresponding to the first set of chroma samples and a luma sample value of a luma sample in a first set of luma samples, the first set of chroma samples being any one of the at least two sets of chroma samples.
2. The method of claim 1, wherein, The method further comprises: performing a first processing on a luma parameter of the current block to determine M first threshold values of the current block, M being a positive integer.
3. The method of claim 2, wherein, The determining the at least two sets of chroma samples of the current block based on the sample division threshold of the current block comprises: determining M+1 sets of chroma samples of the current block based on the M first threshold values and the luma parameter of the current block.
4. The method of claim 3, wherein, When M is 1, the determining the M+1 sets of chroma samples of the current block based on the M first threshold values and the luma parameter of the current block comprises: dividing chroma samples in the current block with a value of the luma parameter less than the first threshold value into a second set of chroma samples; dividing chroma samples in the current block with a value of the luma parameter greater than or equal to the first threshold value into a third set of chroma samples.
5. The method of claim 2, wherein, The performing the first processing on the luma parameter of the current block to determine the M first threshold values of the current block comprises: determining a first value interval of the luma parameter of the current block; determining the M first threshold values of the current block based on the first value interval of the luma parameter of the current block and a first sub-interval number.
6. The method of claim 5, wherein, The method further comprises: determining the first sub-interval number based on the luma parameter of the current block.
7. The method of claim 5, wherein, The first sub-interval number is M+1.
8. The method of claim 1, wherein, The method further comprises: performing a second processing on a chroma parameter of the current block to determine N second threshold values of the current block, N being a positive integer.
9. The method of claim 8, wherein, The determining the at least two sets of chroma samples of the current block based on the sample division threshold of the current block comprises: determining N+1 sets of chroma samples of the current block based on the N second threshold values and the chroma parameter of the current block.
10. The method of claim 9, wherein, When N is 1, the determining the N+1 sets of chroma samples of the current block based on the N second threshold values and the chroma parameter of the current block comprises: dividing chroma samples in the current block with a value of the chroma parameter less than the second threshold value into a fourth set of chroma samples; dividing chroma samples in the current block with a value of the chroma parameter greater than or equal to the second threshold value into a fifth set of chroma samples.
11. The method of claim 2, wherein, The method further comprises: performing the second processing on the chroma parameter of the current block to determine the N second threshold values of the current block, N being a positive integer.
12. The method of claim 11, wherein, The determining the at least two sets of chroma samples of the current block based on the sample division threshold of the current block comprises: determining the (M+1)×(N+1) chroma sample sets of the current block based on the M first threshold values, the N second threshold values, the luma parameter of the current block and the chroma parameter of the current block.
13. The method of claim 12, wherein, when M and N are both 1, the determining the (M+1)×(N+1) chroma sample sets of the current block based on the M first threshold values, the N second threshold values, the luma parameter of the current block and the chroma parameter of the current block comprises: dividing the chroma sample whose luma parameter value in the current block is less than the first threshold value and whose chroma parameter value in the current block is less than the second threshold value into a sixth chroma sample set; dividing the chroma sample whose luma parameter value in the current block is greater than or equal to the first threshold value and whose chroma parameter value in the current block is less than the second threshold value into a seventh chroma sample set; dividing the chroma sample whose luma parameter value in the current block is less than the first threshold value and whose chroma parameter value in the current block is greater than or equal to the second threshold value into an eighth chroma sample set; dividing the chroma sample whose luma parameter value in the current block is greater than or equal to the first threshold value and whose chroma parameter value in the current block is greater than or equal to the second threshold value into a ninth chroma sample set.
14. The method of claim 8 or 11, wherein, the second processing the chroma parameter of the current block to determine the N second threshold values of the current block comprises: determining a second value interval of the chroma parameter of the current block; determining the N second threshold values of the current block based on the second value interval of the chroma parameter of the current block and a second sub-interval number.
15. The method of claim 14, wherein, the method further comprises: determining the second sub-interval number based on the chroma parameter of the current block.
16. The method of claim 14, wherein, the second sub-interval number is N+1.
17. The method of any one of claims 2-13, wherein, the luma parameter comprises a luma sample value or a luma residual value.
18. The method of claim 17, wherein, the luma sample value is a down-sampled luma sample value, and the luma residual value is a down-sampled luma residual value.
19. The method of claim 17, wherein, the luma sample value comprises one of: a luma sample value before loop filtering, a luma sample value before deblocking filtering, a luma sample value before sample offset compensation, a luma sample value before cross-component sample offset compensation, a luma sample value before adaptive loop filtering, a luma sample value before cross-component convolution model filtering of an adaptive filter, and a luma sample value after cross-component convolution model filtering of an adaptive filter.
20. The method of any one of claims 8-15, wherein, the chroma parameter comprises a chroma sample value or a chroma residual value.
21. The method of claim 20, wherein, the chroma sample value comprises one of: a chroma sample value before loop filtering, a chroma sample value before deblocking filtering, a chroma sample value before sample offset compensation filtering, a chroma sample value before cross-component sample offset compensation filtering, a chroma sample value before adaptive loop filtering, and a chroma sample value before cross-component convolution model filtering of an adaptive filter.
22. The method of any one of claims 1-21, wherein, the determining the at least two groups of filter coefficients of the current block based on the at least two chroma sample sets of the current block and the corresponding at least two luma sample sets comprises: determine a first set of filter coefficients corresponding to the first set of chroma samples based on first chroma sample values of the chroma samples in the first set of chroma samples and luma sample values of the luma samples in the first set of luma samples, the positions of the chroma samples in the first set of chroma samples corresponding to the positions of the luma samples in the first set of luma samples.
23. The method of claim 1, wherein, The method further includes: average the first chroma sample values of the chroma samples in the first set of chroma samples and the first filtered chroma sample values to obtain second filtered chroma sample values of the chroma samples in the first set of chroma samples.
24. The method of claim 1, wherein, The method further includes: in a case where a first condition is met, determine chroma reconstruction values of the first chroma samples in the first set of chroma samples based on the first filtered chroma sample values and the first chroma sample values of the first chroma samples in the first set of chroma samples, the first chroma samples being chroma samples whose differences between the first filtered chroma sample values and the first chroma sample values are less than or equal to a third threshold value.
25. The method of claim 24, wherein, The method further includes: in a case where the first condition is not met, determine chroma reconstruction values of the chroma samples in the first set of chroma samples based on the first chroma sample values of the chroma samples in the first set of chroma samples.
26. The method of claim 24 or 25, wherein, The first condition includes: a number of second chroma samples in the first set of chroma samples being less than or equal to a first number; wherein the second chroma samples are chroma samples whose differences between the first filtered chroma sample values and the first chroma sample values are greater than the third threshold value.
27. The method of claim 26, wherein, The first condition further includes: a number of second chroma samples in a tenth set of chroma samples being less than or equal to a second number; wherein the tenth set of chroma samples is part or all of the set of chroma samples in the at least two sets of chroma samples other than the first set of chroma samples.
28. The method of claim 24, wherein, The determining the chroma reconstruction values of the first chroma samples in the first set of chroma samples based on the first filtered chroma sample values and the first chroma sample values of the first chroma samples in the first set of chroma samples includes: averaging the first chroma sample values of the chroma samples in the first set of chroma samples and the first filtered chroma sample values to obtain second filtered chroma sample values of the chroma samples in the first set of chroma samples; summing the differences between the second filtered chroma sample values of the first chroma samples in the first set of chroma samples and the first chroma sample values and second chroma sample values to determine the chroma reconstruction values of the first chroma samples in the first set of chroma samples.
29. A method of encoding, applied to an encoder, the method comprising: in a case where a current block is filtered using a cross-component convolution model, determining at least two sets of chroma samples of the current block based on a sample division threshold of the current block; determining at least two sets of filter coefficients of the current block based on the at least two sets of chroma samples of the current block and at least two sets of luma samples corresponding to the at least two sets of chroma samples; Determine a first filtered chroma sample value of a chroma sample in the first chroma sample set based on a first group of filter coefficients corresponding to the first chroma sample set and a luma sample value of a luma sample in the first luma sample set, the first chroma sample set being any one of the at least two chroma sample sets.
30. The method of claim 29, wherein, The method further comprises: Performing first processing on the luma parameter of the current block to determine M first threshold values of the current block, M being a positive integer.
31. The method of claim 30, wherein, The determining of the at least two chroma sample sets of the current block based on the sample division threshold of the current block comprises: Determining M+1 chroma sample sets of the current block based on the M first threshold values and the luma parameter of the current block.
32. The method of claim 31, wherein, When M is 1, the determining of the M+1 chroma sample sets of the current block based on the M first threshold values and the luma parameter of the current block comprises: dividing chroma samples in the current block with a luma parameter value less than the first threshold value into a second chroma sample set; dividing chroma samples in the current block with a luma parameter value greater than or equal to the first threshold value into a third chroma sample set.
33. The method of claim 30, wherein, The performing of the first processing on the luma parameter of the current block to determine the M first threshold values of the current block comprises: determining a first value interval of the luma parameter of the current block; determining the M first threshold values of the current block based on the first value interval and a first sub-interval number of the luma parameter of the current block.
34. The method of claim 33, wherein, The method further comprises: determining the first sub-interval number based on the luma parameter of the current block.
35. The method of claim 33, wherein, The first sub-interval number is M+1.
36. The method of claim 29, wherein, The method further comprises: Performing second processing on the chroma parameter of the current block to determine N second threshold values of the current block, N being a positive integer.
37. The method of claim 36, wherein, The determining of the at least two chroma sample sets of the current block based on the sample division threshold of the current block comprises: Determining N+1 chroma sample sets of the current block based on the N second threshold values and the chroma parameter of the current block.
38. The method of claim 37, wherein, When N is 1, the determining of the N+1 chroma sample sets of the current block based on the N second threshold values and the chroma parameter of the current block comprises: dividing chroma samples in the current block with a chroma parameter value less than the second threshold value into a fourth chroma sample set; dividing chroma samples in the current block with a chroma parameter value greater than or equal to the second threshold value into a fifth chroma sample set.
39. The method of claim 30, wherein, The method further comprises: Performing second processing on the chroma parameter of the current block to determine N second threshold values of the current block, N being a positive integer.
40. The method of claim 39, wherein, The determining of the at least two chroma sample sets of the current block based on the sample division threshold of the current block comprises: Determining (M+1)×(N+1) chroma sample sets of the current block based on the M first threshold values, the N second threshold values, the luma parameter of the current block and the chroma parameter of the current block.
41. The method of claim 40, wherein, When M and N are both 1, the determining of the (M+1)×(N+1) chroma sample sets of the current block based on the M first threshold values, the N second threshold values, the luma parameter of the current block and the chroma parameter of the current block comprises: divide the chroma sample in the current block, whose value of the luma parameter is less than the first threshold value and whose value of the chroma parameter is less than the second threshold value, into a sixth chroma sample set; divide the chroma sample in the current block, whose value of the luma parameter is greater than or equal to the first threshold value and whose value of the chroma parameter is less than the second threshold value, into a seventh chroma sample set; divide the chroma sample in the current block, whose value of the luma parameter is less than the first threshold value and whose value of the chroma parameter is greater than or equal to the second threshold value, into an eighth chroma sample set; divide the chroma sample in the current block, whose value of the luma parameter is greater than or equal to the first threshold value and whose value of the chroma parameter is greater than or equal to the second threshold value, into a ninth chroma sample set.
42. The method of claim 36 or 39, wherein, The second processing of the chroma parameter of the current block comprises: determining a second value interval of the chroma parameter of the current block; determining the N second threshold values of the current block based on the second value interval and the second subinterval number of the chroma parameter of the current block.
43. The method of claim 42, wherein, The method further comprises: determining the second subinterval number based on the chroma parameter of the current block.
44. The method of claim 42, wherein, The second subinterval number is N+1.
45. The method of any one of claims 30-41, wherein, The luma parameter comprises a luma sample value or a luma residual value.
46. The method of claim 45, wherein, The luma sample value is a down-sampled luma sample value, and the luma residual value is a down-sampled luma residual value.
47. The method of claim 45, wherein, The luma sample value comprises one of the following: a luma sample value before loop filtering, a luma sample value before deblocking filtering, a luma sample value before sample offset compensation, a luma sample value before cross-component sample offset compensation, a luma sample value before adaptive loop filtering, a luma sample value before cross-component convolution model filtering of an adaptive filter, and a luma sample value after cross-component convolution model filtering of an adaptive filter.
48. The method of any one of claims 36-43, wherein, The chroma parameter comprises a chroma sample value or a chroma residual value.
49. The method of claim 48, wherein, The chroma sample value comprises one of the following: a chroma sample value before loop filtering, a chroma sample value before deblocking filtering, a chroma sample value before sample offset compensation filtering, a chroma sample value before cross-component sample offset compensation filtering, and a chroma sample value before adaptive loop filtering.
50. The method of any one of claims 29-49, wherein, The determining of the at least two groups of filter coefficients of the current block based on the at least two chroma sample sets and the corresponding at least two luma sample sets of the current block comprises: determining a first group of filter coefficients corresponding to the first chroma sample set based on the first chroma sample value of the chroma sample in the first chroma sample set and the luma sample value of the luma sample in the first luma sample set, the position of the chroma sample in the first chroma sample set corresponding to the position of the luma sample in the first luma sample set.
51. The method of claim 29, wherein, The method further comprises: averaging the first chroma sample value of the chroma sample in the first chroma sample set and the first filtered chroma sample value to obtain a second filtered chroma sample value of the chroma sample in the first chroma sample set.
52. The method of claim 29, wherein, The method further comprises: In a case where the first condition is met, a chroma reconstruction value of a first chroma sample in the first set of chroma samples is determined based on a first filtered chroma sample value of the first chroma sample in the first set of chroma samples and a first chroma sample value of the first chroma sample, the first chroma sample being a chroma sample whose difference between the first filtered chroma sample value and the first chroma sample value is less than or equal to a third threshold.
53. The method of claim 52, wherein, The method further comprises: In a case where the first condition is not met, a chroma reconstruction value of a chroma sample in the first set of chroma samples is determined based on a first chroma sample value of the chroma sample in the first set of chroma samples.
54. The method of claim 52 or 53, wherein, The first condition comprises: a number of second chroma samples in the first set of chroma samples is less than or equal to a first number; The second chroma sample is a chroma sample whose difference between the first filtered chroma sample value and the first chroma sample value is greater than the third threshold.
55. The method of claim 54, wherein, The first condition further comprises: a number of second chroma samples in a tenth set of chroma samples is less than or equal to a second number; The tenth set of chroma samples is a part or all of the set of chroma samples other than the first set of chroma samples in the at least two sets of chroma samples.
56. The method of claim 52, wherein, The determination of the chroma reconstruction value of the first chroma sample in the first set of chroma samples based on the first filtered chroma sample value of the first chroma sample in the first set of chroma samples and the first chroma sample value comprises: averaging the first chroma sample value and the first filtered chroma sample value of the chroma sample in the first set of chroma samples to obtain a second filtered chroma sample value of the chroma sample in the first set of chroma samples; summing a difference between the second filtered chroma sample value of the first chroma sample in the first set of chroma samples and the first chroma sample value and a second chroma sample value to determine the chroma reconstruction value of the first chroma sample in the first set of chroma samples.
57. An encoder, comprising a first determining unit and a first filtering unit; wherein: the first determining unit is configured to, in a case where a current block is filtered using a cross-component convolution model, determine at least two sets of chroma samples of the current block based on a sample division threshold of the current block; the first determining unit is further configured to determine at least two groups of filter coefficients of the current block based on the at least two sets of chroma samples of the current block and at least two sets of corresponding luma samples; the first filtering unit is configured to determine a first filtered chroma sample value of a chroma sample in a first set of chroma samples based on a first group of filter coefficients corresponding to the first set of chroma samples and a luma sample value of a luma sample in a first set of luma samples, the first set of chroma samples being any one of the at least two sets of chroma samples.
58. An encoder, comprising a first memory and a first processor; wherein: the first memory is configured to store a computer program capable of running on the first processor; the first processor is configured to, when running the computer program, perform the method according to any one of claims 29 to 56.
59. A decoder comprising a second determining unit and a second filtering unit; wherein: the second determining unit is configured to determine at least two sets of chroma samples of a current block based on a sample division threshold of the current block, in a case that the current block is filtered using a cross-component convolution model; the second determining unit is further configured to determine at least two groups of filter coefficients of the current block based on the at least two sets of chroma samples and corresponding at least two sets of luma samples of the current block; and the second filtering unit is configured to determine a first filtered chroma sample value of a chroma sample in a first set of chroma samples based on a first group of filter coefficients corresponding to the first set of chroma samples and a luma sample value of a luma sample in a first set of luma samples, the first set of chroma samples being any one of the at least two sets of chroma samples.
60. A decoder comprising a second memory and a second processor; wherein: the second memory is configured to store a computer program capable of running on the second processor; and the second processor is configured to execute the method of any one of claims 1-28 when running the computer program.
61. A computer readable storage medium storing a computer program which, when executed, implements the method of any one of claims 1-28, or implements the method of any one of claims 29-56.
62. A computer readable storage medium storing a bitstream generated by the method of any one of claims 29-56. 61. A computer readable storage medium, wherein, 62. A computer readable storage medium, wherein,
