Image information processing method, electronic device, storage medium and program product

Abstract

An image information processing method, comprising: determining a candidate reference block list for a target coding block, and constructing a fusion candidate on the basis of the candidate reference block list; on the basis of a fusion classification threshold value, respectively classifying pixel points at corresponding different positions in a template region of the target coding block and a template region of the fusion candidate, so as to obtain at least one pixel classification set; for each pixel classification set, determining a fusion model on the basis of at least two groups of pixel points at corresponding different positions in the pixel classification set; and on the basis of each position of the target coding block, acquiring corresponding target reference pixels in all candidate reference blocks in the fusion candidate, determining, on the basis of each target reference pixel, a matching target fusion model from among fusion models, and on the basis of the target fusion model and the target reference pixel, determining a predicted pixel value of each pixel point in the target coding block.

Description

Image information processing methods, electronic devices, storage media and program products Technical Field

[0001] This application relates to the field of image processing technology, such as an image information processing method, electronic device, storage medium, and program product. Background Technology

[0002] Currently, one approach in Intra Template Match Prediction (IntraTMP) technology is a fusion prediction method. This method sequentially selects multiple block vectors from a candidate list, predicts each candidate individually, and then generates the final prediction result through weighted fusion. However, due to the complexity and diversity of image content, the fusion model determined by a single template cannot effectively improve the accuracy and quality of fusion prediction. Therefore, how to derive multiple fusion models based on template regions and adaptively select the most suitable fusion model for weighted fusion according to the specific content in the reference block has become a pressing problem to be solved in order to improve the performance of fusion prediction. Summary of the Invention

[0003] This application provides an image information processing method, electronic device, storage medium, and program product. The aim is to construct a corresponding fusion model by determining different pixel classifications within image information, and to select the corresponding fusion model to predict pixels based on the classification of pixels within the target coding block. This can improve the diversity of fusion models used in the pixel prediction process. By selecting an appropriate fusion model for pixel prediction based on the actual pixel situation, the image processing quality can be improved and the image effect enhanced.

[0004] This application provides an image information processing method, including:

[0005] Determine a list of candidate reference blocks for the target coding block, and construct fusion candidates based on the list of candidate reference blocks;

[0006] Based on the fusion classification threshold, the pixels at different positions within the template regions of the target coding block and the fusion candidate are classified to obtain at least one pixel classification set.

[0007] For each pixel classification set, a fusion model is determined based on at least two groups of pixels corresponding to different positions within the pixel classification set;

[0008] Based on the position of the target coding block, obtain the corresponding target reference pixel in all candidate reference blocks within the fusion candidate. Based on each target reference pixel, determine the matching target fusion model in each fusion model. Based on the target fusion model and the target reference pixel, determine the predicted pixel value of each pixel in the target coding block.

[0009] This application also provides an image information processing method applied at an encoding end. The image information processing method includes:

[0010] The instruction information of the image information processing method as described in any of the embodiments of this application is transmitted to the decoding end.

[0011] This application also provides an image information processing method applied at a decoding end. The image information processing method includes:

[0012] Obtain the indication information transmitted by the encoding end;

[0013] The image information processing method described in any of the embodiments of this application is executed according to the indicated information.

[0014] This application also provides an electronic device, wherein the electronic device includes:

[0015] One or more processors;

[0016] Memory, configured to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the image information processing method as described in any of the embodiments of this application.

[0018] This application also provides a computer-readable storage medium storing one or more programs that are executed by one or more processors to implement the image information processing method as described in any of the embodiments of this application.

[0019] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the image information processing method provided in this application.

[0020] This application also provides a non-transitory computer-readable storage medium for storing a bit stream, wherein the bit stream includes: indication information indicating an image information processing method, wherein the image information processing method includes any of the image information processing methods described in the embodiments of this application.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0022] Figure 1 is a fusion-based video coding framework provided in an embodiment of this application;

[0023] Figure 2 is a fusion-based video decoding framework provided in an embodiment of this application;

[0024] Figure 3 is an example diagram of intra-frame template matching prediction provided in an embodiment of this application;

[0025] Figure 4 is an example diagram of a weighted fusion prediction based on Wiener filter provided in an embodiment of this application;

[0026] Figure 5 is a flowchart of an image information processing method provided in an embodiment of this application;

[0027] Figure 6 is a flowchart of another image information processing method provided in an embodiment of this application;

[0028] Figure 7 is a flowchart of another image information processing method provided in an embodiment of this application;

[0029] Figure 8 is a flowchart of another image information processing method provided in an embodiment of this application;

[0030] Figure 9 is a flowchart of another image information processing method provided in an embodiment of this application;

[0031] Figure 10 is a flowchart of another image information processing method provided in an embodiment of this application;

[0032] Figure 11 is a schematic diagram of an intra-frame template matching prediction fusion based on multiple models provided in an embodiment of this application;

[0033] Figure 12 is an example diagram of image information processing provided in an embodiment of this application;

[0034] Figure 13 is a schematic diagram of the structure of an image information processing device provided in an embodiment of this application;

[0035] Figure 14 is a schematic diagram of another image information processing device provided in an embodiment of this application;

[0036] Figure 15 is a schematic diagram of another image information processing device provided in an embodiment of this application;

[0037] Figure 16 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0038] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0039] In the following description, the use of suffixes such as “module,” “part,” or “unit” to denote elements is solely for the purpose of illustration in this application and has no particular meaning in itself. Therefore, “module,” “part,” or “unit” may be used interchangeably.

[0040] Referring to Figure 1, in the video coding framework of intra-frame template matching fusion prediction, the video coding framework can include modules such as intra-frame prediction, inter-frame prediction, transform, quantization, loop filtering, and entropy coding. The process of the video coding framework can include the following steps:

[0041] (1) The input video is first divided into frames, and then the frames are divided into blocks;

[0042] (2) The divided blocks are sent to the inter-frame / intra-frame selection module for predictive coding. The intra-frame prediction module is mainly used to remove the spatial correlation of the image; the inter-frame prediction module is mainly used to remove the temporal correlation of the image.

[0043] (3) Subtract the predicted value from the original block to obtain the residual value. Then transform and quantize the residual value to remove frequency domain correlation and perform lossy compression on the data.

[0044] (4) Finally, all the encoding parameters and residual values ​​are entropy encoded to form a binary stream for storage or transmission. The encoded data output by the entropy encoding module is the original video compressed bitstream.

[0045] (5) The predicted value and the residual after inverse transformation and inverse quantization are added together to obtain the block reconstruction value, and finally the reconstructed image is formed.

[0046] (6) The reconstructed image is filtered by a loop filter and stored in the image buffer as a reference image for the future.

[0047] Referring to Figure 2, the video decoding framework may include the following process:

[0048] (1) Parse the bitstream to obtain the prediction pattern and get the prediction value;

[0049] (2) Perform inverse transformation and inverse quantization on the residual obtained from the code stream parsing;

[0050] (3) The predicted value and the residual after inverse quantization and inverse transformation are added together to obtain the block reconstruction value, and finally the reconstructed image is formed.

[0051] (4) The reconstructed image is filtered by a loop filter and stored in the image buffer as a reference image for the future.

[0052] In this embodiment, intra-frame template matching prediction is a special intra-frame prediction mode. It copies the best reference block from the reconstructed portion of the current frame, and the L-shaped template of the best reference block matches the current template of the current block. For a predefined search range, the encoder searches for the template most similar to the current template in the reconstructed portion of the current frame and uses the corresponding block as the reference block. Then, the encoder signals the use of this mode and performs the same prediction operation at the decoder.

[0053] The signal emitted by the encoder is generated by matching the L-shaped, top-only, or left-only adjacent template of the current block with another block in the predefined search area in Figure 3. The predefined search area can include six predefined search areas, namely areas R1 to R6 in Figure 3. These six predefined search areas contain reconstructed samples from the top and left coding tree units (CTUs), as well as some reconstructed samples within the current CTU located above, to the left, at the bottom left corner, and at the top right corner of the current block.

[0054] For example, a given search order of 6 predefined search regions is used, namely R4, R5, R6, R1, R2, and R3. Within each predefined search region, the decoder constructs a candidate list of up to 19 template-matching block vectors, which are ordered in ascending order of template cost.

[0055] Intra-frame template matching prediction supports the following modes:

[0056] Single predictor: Select a predictor from the candidate list.

[0057] Fusion of multiple predictors: Multiple predictors are fused multiple times to obtain the final reference block. The fusion weights can be calculated based on the template matching cost of each predictor, or using a weight derivation method based on Wiener filters.

[0058] Subpixel precision: Subpixel precision is supported when using a single predictor. A new candidate list is constructed by including the selected integer block vector and the surrounding 1 / 2-pixel and 1 / 4-pixel subpixel positions. The candidate list is sorted according to the same cost function used for integer BV search. Afterward, the top two candidates are allowed to be selected, and a single flag is sent from the encoder to the decoder.

[0059] Linear filter model: A linear filter can be learned between the reference template and the current template, and the linear model is applied to the reference block. This mode can be used for a single predictor when sub-pixel precision is not used.

[0060] In addition, when using an IntraTMP with Local Illumination Compensation (LIC), the following factors should be considered:

[0061] 1. For a given coding unit (CU), LIC is mutually exclusive with the use of filtering methods such as filter-based linear model (FLM) or convolutional cross-component model (CCCM).

[0062] 2. Use LIC and fusion within the internal TMP.

[0063] 3. For screen content encoding, top-only and left-only templates are allowed to determine the LIC model. For camera-captured encoding, only the top-left template is used.

[0064] Similar to Intra Block Copy-Local Illumination Compensation (IBC-LIC), it supports a multi-model linear model (MMLM) for screen content encoding.

[0065] In multi-prediction fusion, two fusion modes can be included: template cost-based weighted fusion and Wiener filter-based weighted fusion.

[0066] Template cost-based weighted fusion: For multiple predictors / models participating in the fusion, the template cost of each predictor / model is first calculated, and then the fusion weights are calculated using the template cost.

[0067] Among them, SAD i Let represent the template cost of the i-th predictor / model, where i is an integer from 1 to n, n is the total number of predictors / models, sum represents the sum of the template costs of all predictors / models, and w i p represents the weight coefficient of the i-th predictor / model. i This represents the pixel information of the reference block within the reconstructed region, while p fusion This indicates the predicted pixel information for the current block.

[0068] The fusion equation for weighted fusion based on Wiener filters can be shown below:

[0069] Among them, refBlock n w represents the pixel information of the nth reference block.n w represents the weight coefficient of the nth reference block. n The weight coefficient of the median value is represented by midValue, which is determined based on a preset bit depth. N represents the number of reference blocks participating in the fusion, and preSamples represents the pixel information of the current block.

[0070] Referring to Figure 4, the weighting coefficients w(n) of the template CurT of the current block curBlock and the template MatchT(n) of the reference block refBlock(n) are derived using a Gaussian solver. The weighting coefficients w(n) derived from the templates are then used to perform fusion prediction on the reference block to obtain the predicted pixel information of the current block. Here, BV(n) represents the block vector between the template CurT of the current block curBlock and the template MatchT(n) of the reference block refBlock(n).

[0071] Figure 5 is a flowchart of an image information processing method provided in an embodiment of this application. This embodiment of the application is applicable to scenarios involving fusion-based intra-frame template matching prediction during video image encoding and decoding. The method can be executed by an image information processing device, which can be implemented by software and / or hardware methods. Referring to Figure 5, the method provided in this embodiment of the application includes the following operations:

[0072] In step 110, a list of candidate reference blocks for the target coding block is determined, and fusion candidates are constructed based on the list of candidate reference blocks.

[0073] The target coding block can be the current block to be predicted by intra-frame template matching. The candidate reference block list can be a list of one or more reference blocks determined by the target coding block in the reconstruction region. The fusion candidate can be constructed from the selected reference blocks for fusion. The reference blocks in the fusion candidate are some or all of the reference blocks in the candidate reference block list. There can be multiple groups of fusion candidates. Each group can include the same or different reference blocks. Multiple groups of fusion candidates can form a fusion candidate list.

[0074] In this embodiment, a predefined search region for the target coding block can be determined, candidate reference blocks can be determined within each predefined search region, and a candidate reference block list constructed from each reference block can be obtained. All or some reference blocks can be selected from the candidate reference block list to form fusion candidates. For example, reference blocks whose pixel values ​​satisfy the median or average value can be selected from the candidate reference block list, and fusion candidates can be constructed using the selected reference blocks.

[0075] In step 120, pixels at different positions within the target coding block and the template region of the fusion candidate are classified according to the fusion classification threshold to obtain at least one pixel classification set.

[0076] The fusion classification threshold can be a critical value used to classify pixels. The fusion classification threshold can be pre-configured or determined according to the pixel distribution of the template region. The fusion classification threshold can be one or more thresholds. The fusion classification threshold can be used to divide the pixel classification set. Different pixel classification sets can include pixels at different positions. The fusion classification threshold can divide pixels at different positions in the template region of different target coding blocks and fusion candidates into at least two pixel classification sets.

[0077] In this embodiment, for the template region of the target coding block and the template region of the fusion candidate, the corresponding pixels in each template region can be compared with the fusion classification threshold at each location, thereby classifying one or more pixels belonging to the same location into the same pixel classification set. Each pixel classification set can have at least one location of the template region of a pixel. It can be understood that a location can refer to the same positional relationship between the pixel in the template region of the target coding block and the template region of the target coding block as the positional relationship between the pixel in the template region of all candidate reference blocks in the fusion candidate and the corresponding template region of the pixel.

[0078] In 130, for each pixel classification set, a fusion model is determined based on at least two sets of pixels corresponding to different positions within the pixel classification set.

[0079] Each pixel location can include a set of pixels. This set of pixels can include one pixel from the template region of the target coding block at the corresponding position in the set of pixels, and one pixel from the template region of each candidate reference block within the fusion candidate at the corresponding position in the set of pixels. The fusion model can be a model that fuses multiple template regions. The fusion model can indicate the mapping relationship between multiple candidate reference blocks and the target coding block. This mapping relationship can be determined by the mapping relationship between pixels at the same position within the template regions of the multiple candidate reference blocks and the template region of the target coding block.

[0080] In this embodiment, at least two groups of pixels are determined within each pixel classification set, with each group having the same position. For each pixel classification set, the fusion model can be determined by the mapping relationship between the template regions of multiple candidate reference blocks in the at least two groups of pixels and the pixels at the same position within the template region of the target coding block. For example, a mapping relationship corresponding to the fusion model can be constructed for each group of pixels. By connecting the mapping relationships corresponding to multiple groups of pixels, the model parameters corresponding to the mapping relationship can be determined, thereby obtaining the fusion model.

[0081] In step 140, the corresponding target reference pixels are obtained in all candidate reference blocks within the fusion candidate based on each position of the target coding block. The matching target fusion model is determined in each fusion model based on each target reference pixel. The predicted pixel value of each pixel in the target coding block is determined based on the target fusion model and the target reference pixel.

[0082] In this embodiment, target reference points can be determined at different positions within the candidate reference blocks of the fusion candidates for different positions of the target coding block. The target fusion model corresponding to the target reference point is then determined from multiple fusion models based on the target reference point. This process can include determining whether the target reference point belongs to a pixel classification set corresponding to a certain fusion model. If it does, the fusion model corresponding to the pixel classification set can be used as the target fusion model corresponding to the target reference point. Determining whether the target reference point belongs to a pixel classification set of a certain fusion model can be achieved by judging the distance between the target reference point and pixels within each pixel classification set, or by classifying the target reference point using a fusion classification threshold. The target reference pixels at each position corresponding to the target coding block are processed using the target fusion model to obtain the predicted pixel values ​​for different pixels within the target coding block.

[0083] In this embodiment, a fusion candidate is constructed from a candidate reference block list of the target coding block. Pixels belonging to the same position within the template region of the target coding block and the template region of the fusion candidate are divided into a pixel classification set according to a fusion classification threshold. At least two groups of pixels corresponding to different positions are determined within each pixel classification set, with each group having the same position. A fusion model is determined using these at least two groups of pixels corresponding to different positions. For each pixel in the target coding block, a target reference pixel corresponding to each position is obtained from all candidate reference blocks within the fusion candidate. A target fusion model matching the target reference pixel is determined among multiple fusion models based on the target reference pixel. The predicted pixel value of each pixel in the target coding block is determined using the target fusion model and the target reference pixel. This embodiment, by determining different pixel classification sets within the image information, constructing fusion models corresponding to the pixel classification sets, and selecting the corresponding fusion model to predict pixels based on the classification of pixels within the target coding block, can improve the diversity of fusion models used in the pixel prediction process. Selecting an appropriate fusion model for pixel prediction based on the actual pixel situation can improve image processing quality and enhance image effects.

[0084] Based on the above-described embodiments, the method further includes: obtaining a fusion classification threshold, wherein obtaining the fusion classification threshold includes at least one of the following:

[0085] Use the average or median of all pixels within the template region of the target coding block as the fusion classification threshold; or

[0086] The average or median of all pixels in the template region of each candidate reference block within the fusion candidate is used as the fusion classification threshold.

[0087] In the embodiments of this application, the average or median of all pixels in the template region of the target coding block can be used as the fusion classification threshold. Alternatively, the average or median of all pixels in the template region of all candidate reference blocks in the fusion candidate can be used as the fusion classification threshold. It is understood that the above-mentioned average or median of all pixels is only an example and is not limited. Other statistical methods for pixels, such as mode, covariance, etc., can also be included.

[0088] Based on the above application embodiments, the fusion classification threshold is obtained, which includes: taking the average or median of all pixels in the template region of the specified candidate reference block in the fusion candidate as the fusion classification threshold.

[0089] In this embodiment, a designated candidate reference block can be determined from the fusion candidates. The average or median of all pixels in the template region of the designated candidate reference block can be used as the fusion classification threshold. The designated candidate reference block can be a reference block configured by the upper-layer signaling, or a reference block that meets specific conditions, such as a reference block with the minimum template cost.

[0090] Based on the above application embodiments, the fusion classification threshold is obtained, which includes:

[0091] All pixels in the template regions of all candidate reference blocks in the fusion candidate are weighted according to the template cost weight of the corresponding candidate reference block and fused into a predicted template region. The average or median of all pixels in the predicted template region is used as the fusion classification threshold.

[0092] The template cost weight can be determined by the ratio between the template cost of each template and the template cost of all templates. For example, the template cost weight can be determined by the difference between the sum of the template costs of all candidate reference blocks and the template cost of its own candidate reference block, and the ratio of the number of remaining candidate reference blocks in the fusion candidate excluding its own candidate reference block.

[0093] In this embodiment, a template cost weight can be determined for the template region of each candidate reference block within the fusion candidate. The template regions of each candidate reference block can be fused according to the template cost weight corresponding to each candidate reference block. The fused template region can be used as the predicted template region, and the average or median of all pixels within the predicted template region can be used as the fusion classification threshold. For example, the candidate reference blocks may include reference block 1, reference block 2, and reference block 3. The template region corresponding to each candidate reference block is a 1*2 pixel region. The pixel values ​​of the first pixel of reference block 1, reference block 2, and reference block 3 can be weighted and summed according to their respective template cost weights. The weighted sum can be used as the first pixel value of the predicted template region. Similarly, the pixel value of the second pixel can be determined, thereby generating a predicted target region including two pixels.

[0094] In some embodiments, the method further includes: performing local illumination compensation prediction on the second template region of the candidate reference block within the fusion candidate.

[0095] In some application embodiments, the template region includes at least one of the left pixel region, the top pixel region, the upper left pixel region, the lower left pixel region, and the upper right pixel region of the target coding block or the candidate reference block within the fusion candidate.

[0096] In some application embodiments, a candidate reference block list for the target coding block is determined, and a fusion candidate is constructed based on the candidate reference block list, including at least one of the following:

[0097] Obtain a list of candidate reference blocks for the target coding block, and divide each candidate reference block in the list into at least one candidate reference block group without repetition, and use each candidate reference block group as a fusion candidate.

[0098] Obtain a list of candidate reference blocks for the target coding block; divide each candidate reference block in the list into at least one candidate reference block group; filter the candidate reference blocks within each candidate reference block group based on given conditions; and use the filtered candidate reference block groups as fusion candidates. The given conditions include at least one of pixel threshold and template cost threshold; or

[0099] Obtain a list of candidate reference blocks for the target coding block, divide each candidate reference block in the list into at least one group of candidate reference blocks, and use each group of candidate reference blocks as a fusion candidate, wherein at least two groups of candidate reference blocks have at least one duplicate candidate reference block.

[0100] In some application embodiments, a list of candidate reference blocks for the target coding block can be obtained, and the list of candidate reference blocks can be divided into different candidate reference block groups. The candidate reference blocks in each candidate reference block group can be unique, and each group of candidate reference blocks can be used as a fusion candidate.

[0101] In other application embodiments, a list of candidate reference blocks for the target coding block can be obtained. The candidate reference blocks in the list can be divided into at least one group of candidate reference blocks. Within each group of candidate reference blocks, the candidate reference blocks can be filtered according to given conditions. For example, reference blocks that do not meet the pixel threshold or template cost threshold can be filtered out. Each group of candidate reference blocks after filtering can be determined as a fusion candidate.

[0102] In other application embodiments, a list of candidate reference blocks for the target coding block is obtained, and each candidate reference block in the candidate reference block list is divided into at least one group of candidate reference blocks. It is understood that when all candidate reference blocks in the candidate reference block list are divided into at least two groups of candidate reference blocks, at least two groups of candidate reference blocks have at least one identical candidate reference block, and each group of candidate reference blocks can be used as a fusion candidate.

[0103] Figure 6 is a flowchart of another image information processing method provided in an embodiment of this application. This embodiment is a refinement based on the above-mentioned embodiment. Referring to Figure 6, the method provided in this embodiment includes the following operations:

[0104] In step 210, a list of candidate reference blocks for the target coding block is determined, and fusion candidates are constructed based on the list of candidate reference blocks.

[0105] In 220, the first template region of the target coding block and the second template region of the candidate reference block within the fusion candidate are determined.

[0106] In this embodiment, the template region of the target coding block can be used as the first template region, and the template region of each candidate reference block within the fusion candidate can be used as the second template region. It is understood that when there are multiple candidate reference blocks in the candidate reference list, there can be multiple corresponding second template regions.

[0107] In step 230, the pixels at the same position in the first template region and the second template region are compared with the fusion classification threshold to classify the pixels at the same position into a pixel classification set.

[0108] In this embodiment, a set of pixels can be determined for each position in both the first template region and the second template region. Each set of pixels can include pixels from the first template region and pixels from each of the second template regions. The set of pixels corresponding to each position can be compared with a fusion classification threshold to group pixels belonging to the same position into a single pixel classification set. The process of comparing each pixel with the fusion classification threshold can include determining a parameter value for each pixel. This parameter value can include the average, median, weighted average, weighted sum, etc., of the set of pixels at each position. The comparison result between the parameter value and the fusion classification threshold can be used to group pixels into a single pixel classification set. For example, when the fusion classification threshold is a critical value, it can be used to divide two pixel classification sets. The group of pixels can be assigned to either the first or second pixel classification set based on the relationship between each set of pixels and the fusion classification threshold.

[0109] In 240, for each pixel classification set, a fusion model is determined based on at least two sets of pixels corresponding to different positions within the pixel classification set.

[0110] In step 250, the corresponding target reference pixels are obtained in all candidate reference blocks within the fusion candidate based on each position of the target coding block. The matching target fusion model is determined in each fusion model based on each target reference pixel. The predicted pixel value of each pixel in the target coding block is determined based on the target fusion model and the target reference pixel.

[0111] Based on the above-described embodiments, pixels at the same position in the first template region and the second template region are compared with a fusion classification threshold, including at least one of the following:

[0112] Compare each pixel within the first template region with the fusion classification threshold; or

[0113] The average value of pixels at each position within the second template region of all candidate reference blocks in the fusion candidate is compared with the fusion classification threshold.

[0114] Based on the above application embodiments, each pixel in the first template area is compared with the fusion classification threshold. By comparing the size relationship between each pixel in the first template area and the fusion classification threshold, a set of pixels at each position can be divided into a pixel classification set.

[0115] In other embodiments, the pixels at each position within the second template region of all candidate reference blocks within the fusion candidate can be obtained. A set of pixels at each position within the second template region can be compared with the fusion classification threshold. Based on the comparison result, a set of pixels at each position can be assigned to a pixel classification set. It is understood that a set of pixels at each position assigned to the pixel classification set includes at least the pixels at the corresponding position within the first template region and the pixels at the corresponding position within the second template region.

[0116] Based on the above-described embodiments, the pixels at the same position in the first template region and the second template region are compared with the fusion classification threshold, including:

[0117] The average value of pixels at each position within the second template region of the specified candidate reference block within the fusion candidate is compared with the fusion classification threshold.

[0118] The designated candidate reference block can be a candidate reference block selected within the fusion candidate. The designated candidate reference block is used to compare with the fusion classification threshold. The pixels at each position within the candidate reference block can be compared with the fusion classification threshold, thereby classifying a set of pixels corresponding to each position into a pixel classification set. The set of pixels corresponding to each position can include the pixels at the corresponding position in the first template region and the pixels at the corresponding position in the second template region. The designated candidate reference block can be a reference block configured by higher-layer signaling or a reference block that meets specific conditions, such as the reference block with the minimum template cost within the fusion candidate.

[0119] In this embodiment, the pixels at each location within the second template region of the specified candidate reference block can be compared with the fusion classification threshold, thereby classifying a set of pixels at each location into a pixel classification set.

[0120] Based on the above application embodiments, the designated candidate reference block includes at least the reference block with the minimum template cost among the fusion candidates.

[0121] In some embodiments, pixels at the same position within the first template region and the second template region are compared with a fusion classification threshold, including:

[0122] The pixels at each position of the template region of all candidate reference blocks in the fusion candidate are weighted according to the template cost weight of the corresponding candidate reference block and fused into a predicted template region. The pixels at each position in the predicted template region are compared with the fusion classification threshold.

[0123] In this embodiment, a template cost weight corresponding to each candidate reference block can be determined for the template region of each candidate reference block within the fusion candidate. The template regions of each candidate reference block can be fused according to their respective template cost weights, and the fused template region is used as the predicted template region. It is understood that the fused pixel at each position within the predicted template region can be obtained by weighted fusion of the pixels at the corresponding positions within the template regions of each candidate reference block within the fusion candidate. For each position within the predicted template region, the pixels at each position can be compared with a fusion classification threshold. Based on the comparison result, a set of pixels corresponding to each position can be assigned to a corresponding pixel classification set. This set of pixels can include pixels at corresponding positions within a first template region and pixels at corresponding positions within a second template region.

[0124] Optionally, based on the above application embodiments, the template cost weight of each candidate reference block is obtained. The template cost weight is the difference between the sum of the template costs of each candidate reference block and the template cost of the candidate reference block, and the ratio between the product of the number of remaining candidate reference blocks (excluding the candidate reference block) and the sum of the template costs within the fusion candidate.

[0125] In this embodiment, the template cost weight of each candidate reference block can be determined by the ratio of the template cost weight of each candidate reference block to the template cost weights of other candidates. For example, the template cost weights of all candidate reference blocks are obtained, and the sum of the template cost weights of all candidate reference blocks is determined as the template cost sum. For each candidate reference block, the difference between the template cost sum and its respective template cost is determined. The difference for each candidate reference block can be obtained. The difference can be the sum of the template costs of all other candidate reference blocks excluding itself within each candidate reference block. Alternatively, the product of the number of remaining candidate reference blocks within the fusion candidate (excluding itself) and the template cost sum can be obtained. The ratio of the difference to the product can be used as the template cost weight of each candidate reference block.

[0126] For example, the template cost weight can be determined by the following formula:

[0127] Among them, SAD i Let w represent the template cost of the i-th candidate reference block within the fusion candidate, n represent the total number of candidate reference blocks within the fusion candidate, and w represent the template cost of the i-th candidate reference block within the fusion candidate. i This represents the template cost weight of the i-th candidate reference block within the fusion candidate.

[0128] For example, the predicted template region can be determined based on the template cost weight using the following formula:

[0129] Among them, w ip represents the template cost weight of the i-th candidate reference block within the fusion candidate. i p represents the template region of the i-th candidate reference block within the fusion candidate. fusion This indicates the prediction template region.

[0130] Figure 7 is a flowchart of another image information processing method provided in an embodiment of this application. This embodiment describes the process of determining the fusion model. Referring to Figure 7, the method provided in this embodiment includes the following operations:

[0131] In step 310, a list of candidate reference blocks for the target coding block is determined, and fusion candidates are constructed based on the list of candidate reference blocks.

[0132] In step 320, pixels at different positions within the target coding block and the template region of the fusion candidate are classified according to the fusion classification threshold to obtain at least one pixel classification set.

[0133] In 330, for each pixel classification set, the first pixel and the second pixel at the same position are extracted sequentially in the first template region of the target coding block and the second template region of the candidate reference block in the fusion candidate to form a pixel group.

[0134] In this embodiment of the application, within each pixel classification set, pixels can be extracted according to each position in the first template region of the target coding block and the second template region of the candidate reference block in the fusion candidate. The pixels extracted at each position can be regarded as a group of pixels, and multiple groups of pixels corresponding to all positions can be obtained. The pixels in each group of pixels can be located at the same position. The same position can mean that the position of the first pixel in the first template region is the same as the position of the second pixel in the second template region.

[0135] In 340, the model parameters of the fusion model are determined according to at least two pixel groups.

[0136] In this embodiment, at least two pixel groups can be obtained. The mapping relationship between pixels in the first template region and pixels in each second template region at the same position can be determined using each pixel group. The weight parameters of the fusion model can be solved using the mapping relationships corresponding to at least two pixel groups. The model parameters at least include weight coefficients indicating the mapping relationship between pixels in the first template region and pixels in each second template region within a pixel group. In some embodiments, the fusion model may also include the mapping relationship between pixels in the first template region, pixels in each second template region, and intermediate values ​​determined by a preset bit depth. The preset bit depth can be pre-configured, and correspondingly, the model weights may also include weight coefficients for the intermediate values.

[0137] In step 350, the corresponding target reference pixels are obtained in all candidate reference blocks within the fusion candidate based on each position of the target coding block. The matching target fusion model is determined in each fusion model based on each target reference pixel. The predicted pixel value of each pixel in the target coding block is determined based on the target fusion model and the target reference pixel.

[0138] Optionally, based on the above-described embodiments, the fusion model includes at least:

[0139] Where predSamples represents the pixel values ​​at the target location within the target coding block, and the target location can be any location within the target coding block, refBlock n This represents the pixel at the target location of the nth candidate reference block within the fusion candidate list, where N represents the number of all candidate reference blocks within the fusion candidate list, and midValue represents the median value determined based on a preset bit depth, which is a fixed value. n This represents the nth model parameter of the fusion model, where n ranges from 0 to N.

[0140] Figure 8 is a flowchart of another image information processing method provided in an embodiment of this application. This embodiment describes the fusion prediction process based on a fusion model and multiple templates. Referring to Figure 8, the method provided in this embodiment includes the following operations:

[0141] In step 410, a list of candidate reference blocks for the target coding block is determined, and fusion candidates are constructed based on the list of candidate reference blocks.

[0142] In step 420, pixels at different positions within the target coding block and the template region of the fusion candidate are classified according to the fusion classification threshold to obtain at least one pixel classification set.

[0143] In 430, for each pixel classification set, a fusion model is determined based on at least two sets of pixels corresponding to different positions within the pixel classification set.

[0144] In 440, for each position within the target coding block, obtain the target reference pixel at the corresponding position within all candidate reference blocks in the fusion candidate.

[0145] In the embodiments of this application, for each position within the target coding block, the corresponding pixel point can be obtained in all candidate reference blocks of the fusion candidate. The pixel points of all candidate reference blocks at the corresponding positions can be used as target reference pixels. It is understood that each position within the target coding block can have its own corresponding target reference pixel.

[0146] In step 450, the target reference pixels at each position are compared with the fusion classification threshold. The target classification set to which the target reference pixels at each position belong is determined in the pixel classification set. The fusion model corresponding to the target classification set is used as the target fusion model.

[0147] For example, the target reference pixel at each location can be compared with the fusion classification threshold. The target classification set to which the target reference pixel at each location belongs can be determined within all pixel classification sets through the comparison results. The fusion model corresponding to the target classification set at each location can be used as the target fusion model at each location. It can be understood that the target fusion model can correspond to different locations.

[0148] In 460, each target reference pixel is input into the corresponding target fusion model to obtain the predicted pixel value of the target coding block at the specified location.

[0149] In the embodiments of this application, for each target reference pixel, the target reference pixel can be input into the target fusion model corresponding to the target reference pixel, and the predicted pixel value of the target coding block at each position can be predicted by the target fusion model.

[0150] Based on the above-described embodiments, the target reference pixels at each location are compared with the fusion classification threshold, including at least one of the following:

[0151] The average value of the target reference pixels at each position within all candidate reference blocks in the fusion candidate is compared with the fusion classification threshold;

[0152] Compare the target reference pixels at each position of the specified candidate reference block in the fusion candidate with the fusion classification threshold; or

[0153] The target reference pixels at each position in all candidate reference blocks in the fusion candidate are weighted and fused according to their respective template cost weights, and then compared with the fusion classification threshold.

[0154] In this embodiment of the application, the method of comparing the target reference pixel at each location with the fusion classification threshold may include: determining the average value for the target reference pixel at each location, and comparing the average value with the fusion classification threshold. It is understood that the median or other data statistical parameter values ​​may also be determined for the target reference pixel at each location.

[0155] In other embodiments, the method of comparing the target reference pixels at each location with the fusion classification threshold may include: comparing the target reference pixels at each location in the specified candidate reference block in the fusion list with the fusion classification threshold respectively, thereby determining the target classification set to which a set of target reference pixels at each location belongs.

[0156] In other embodiments, comparing the target reference pixels at each location with the fusion classification threshold may include: determining the template cost weight corresponding to each target reference pixel within all candidate reference blocks; performing weighted fusion on the target reference pixels at each location according to their respective template cost weights; and weighted fusion may include a weighted average or a weighted sum. For each location, the pixel value after weighted fusion can be compared with the fusion classification threshold to determine the pixel classification set to which the target reference pixels at that location belong.

[0157] Based on the above application embodiments, the template cost includes at least one of absolute difference and cost, squared difference and cost, or transform domain absolute difference and cost.

[0158] In some embodiments, the method further includes: determining that the target coded block satisfies preset conditions, wherein the preset conditions include at least one of the following:

[0159] The width of the target coded block is greater than or equal to the first threshold;

[0160] The height of the target coded block is greater than or equal to the second threshold; or

[0161] The area of ​​the target coded block is greater than or equal to the third threshold.

[0162] In the embodiments of this application, before performing fusion prediction on the target coding block, the target coding block can be judged to determine whether it meets preset conditions, such as the width of the target coding block being greater than or equal to a first threshold, the height of the target coding block being greater than or equal to a second threshold, and the area of ​​the target coding block being greater than or equal to a third threshold.

[0163] In some embodiments of this application, the embodiments can be applied to scenarios where fusion-based intra-frame template matching prediction is performed at the encoding end. This method can be executed by an image information processing device, which can be implemented by software and / or hardware methods, and is generally applied at the encoding end. The image information processing method includes:

[0164] The instruction information indicating the image information processing method is transmitted to the decoding end.

[0165] In this application embodiment, the encoding end can transmit indication information to the decoding end, and the indication information can indicate the image information processing method provided in any embodiment of this application.

[0166] Figure 9 is a flowchart of another image information processing method provided in an embodiment of this application. This embodiment of the application is applicable to scenarios where the encoding end performs fusion-based intra-frame template matching prediction. This method can be executed by an image information processing device, which can be implemented by software and / or hardware methods. Referring to Figure 9, the method provided in this embodiment of the application includes the following operations:

[0167] In step 510, a list of candidate reference blocks for the target coding block is determined, and fusion candidates are constructed based on the list of candidate reference blocks.

[0168] In 520, the pixels at different positions in the target coding block and the template region of the fusion candidate are classified according to the fusion classification threshold to obtain at least one pixel classification set.

[0169] In 530, for each pixel classification set, the fusion model is determined based on at least two sets of pixels corresponding to different positions within the pixel classification set.

[0170] In step 540, the corresponding target reference pixels are obtained in all candidate reference blocks within the fusion candidate based on each position of the target coding block. The matching target fusion model is determined in each fusion model based on each target reference pixel. The predicted pixel value of each pixel in the target coding block is determined based on the target fusion model and the target reference pixel.

[0171] In 550, the instruction information indicating the image information processing method is transmitted to the decoding end.

[0172] The indication information can indicate whether the decoding end performs encoding processing according to the image information processing method provided in the embodiments of this application, and the indication information can indicate whether the interface end enables or disables the image information processing method provided in the embodiments of this application.

[0173] In this embodiment, the encoding end can transmit instruction information to the decoding end, and instruct the encoding and decoding processing to be performed according to the image information processing method provided in this embodiment.

[0174] Based on the above-described embodiments, the instruction information includes at least one of the following:

[0175] Intra-frame prediction indication information, wherein the intra-frame prediction indication information indicates whether intra-frame prediction mode is enabled or disabled in image information processing;

[0176] Intra-frame template fusion prediction information, wherein the intra-frame template fusion prediction information indicates whether the image information processing enables or disables the template fusion-based intra-frame prediction mode;

[0177] Intra-frame template fusion type, wherein the intra-frame template fusion type indicates the type of intra-frame template fusion prediction used by the coding unit in image information processing, and the type includes at least one of template cost weighted fusion, single-model Wiener filter weighted fusion, or multi-model Wiener filter weighted fusion; or

[0178] Fusion candidate index, which indicates the fusion candidates involved in the fusion process during image information processing.

[0179] Based on the above application embodiments, the indication level of the indication information includes at least one of the following: video level; sequence level; image level; slice level; or coding unit level.

[0180] In this application embodiment, the first indication information can indicate the use of the image information processing method provided in this application embodiment at the video level; the first indication information can also indicate the use of the image information processing method provided in this application embodiment at the sequence level; the first indication information can also indicate the use of the image information processing method provided in this application embodiment at the image level; the first indication information can also indicate the use of the image information processing method provided in this application embodiment at the slice level; and the first indication information can also indicate the use of the image information processing method provided in this application embodiment at the coding unit level.

[0181] Based on the above-described embodiments, the instruction information includes at least one of the following:

[0182] Intra-prediction indication information (set to enabled), intra-template fusion prediction information (set to enabled), intra-template fusion type and fusion candidate index (set to weighted fusion using multi-model Wiener filter); or

[0183] Intra-prediction indication information when set to enabled, intra-template fusion prediction information when set to enabled, intra-template fusion type when set to weighted fusion using Wiener filter, Wiener filter indication information when set to multi-model Wiener filter, and fusion candidate index.

[0184] In this embodiment, the encoding end transmits intra-frame prediction indication information (set to the enabled state), intra-frame template fusion prediction information (set to the enabled state), intra-frame template fusion type (set to the weighted fusion state of multi-model Wiener filter), and fusion candidate index as indication information to the decoding end, so that the decoding end performs image decoding according to the image information processing method provided in this embodiment. Alternatively, the encoding end transmits intra-frame prediction indication information (set to the enabled state), intra-frame template fusion prediction information (set to the enabled state), intra-frame template fusion type (set to the weighted fusion state of Wiener filter), Wiener filter indication information (set to the multi-model Wiener filter), and fusion candidate index as indication information to the decoding end, so that the decoding end performs image decoding according to the image information processing method provided in this embodiment.

[0185] Figure 10 is a flowchart of another image information processing method provided in an embodiment of this application. This embodiment of the application is applicable to scenarios where the decoding end performs fusion-based intra-frame template matching prediction. This method can be executed by an image information processing device, which can be implemented by software and / or hardware methods. Referring to Figure 10, the method provided in this embodiment of the application includes the following operations:

[0186] In 610, the instruction information of the instruction image information processing method transmitted by the encoding end is obtained.

[0187] In this embodiment, the decoding end can receive instruction information transmitted by the encoding end, and the decoding end can instruct the image information processing method provided in this embodiment to perform encoding and decoding processing according to the instruction information.

[0188] In step 620, a list of candidate reference blocks for the target coding block is determined, and fusion candidates are constructed based on the list of candidate reference blocks.

[0189] In step 630, pixels at different positions within the target coding block and the template region of the fusion candidate are classified according to the fusion classification threshold to obtain at least one pixel classification set.

[0190] In 640, for each pixel classification set, a fusion model is determined based on at least two sets of pixels corresponding to different positions within the pixel classification set.

[0191] In step 650, the corresponding target reference pixels are obtained in all candidate reference blocks within the fusion candidate based on each position of the target coding block. The matching target fusion model is determined in each fusion model based on each target reference pixel. The predicted pixel value of each pixel in the target coding block is determined based on the target fusion model and the target reference pixel.

[0192] Based on the above application embodiments, the indication level of the indication information includes at least one of the following: video level; sequence level; image level; slice level; or coding unit level.

[0193] Based on the above-described embodiments, the instruction information includes at least one of the following:

[0194] Intra-prediction indication information (set to enabled), intra-template fusion prediction information (set to enabled), intra-template fusion type and fusion candidate index (set to weighted fusion using multi-model Wiener filter); or

[0195] Intra-prediction indication information when set to enabled, intra-template fusion prediction information when set to enabled, intra-template fusion type when set to weighted fusion using Wiener filter, Wiener filter indication information when set to multi-model Wiener filter, and fusion candidate index.

[0196] In this embodiment of the application, when the decoding end receives the following indication information: intra-frame prediction indication information indicating that the intra-frame prediction mode is enabled, intra-frame template fusion prediction information indicating that the intra-frame template fusion prediction mode is enabled, intra-frame template fusion type indicating weighted fusion through multi-model Wiener filter, and fusion candidate index indicating which set of fusion candidates to select, the decoding end can execute any of the image information processing methods provided in this embodiment of the application.

[0197] Alternatively, when the decoding end receives the following indication information: intra-prediction indication information indicating that the intra-prediction mode is enabled; intra-template fusion prediction information indicating that the intra-template fusion prediction mode is enabled; intra-template fusion type indicating weighted fusion through Wiener filter; Wiener filter indication information indicating that the Wiener filter used is a multi-model Wiener filter; and fusion candidate index indicating which set of fusion candidates to select, the decoding end may execute any of the image information processing methods provided in the embodiments of this application.

[0198] Figure 11 is a schematic diagram of intra-frame template matching prediction fusion based on multiple models provided in an embodiment of this application. Referring to Figure 12, intra-frame template matching prediction fusion based on multiple models can include the following process: constructing fusion candidates, deriving a first threshold for grouping the first fusion candidate in the candidate list, comparing the pixel set of the template region with the first threshold, grouping the pixel set according to the comparison result, deriving a set of model coefficients according to the pixels in each group, comparing the pixel set of the reference region with the first threshold, and selecting the model coefficients of the corresponding group to perform fusion prediction on the current block.

[0199] In one exemplary embodiment, an intra-frame prediction method for video blocks may include the following operations:

[0200] In section 1.1, we construct fusion candidates to participate in the fusion process.

[0201] For example, a candidate list is constructed by means of searching, implicit merging, etc. Multiple candidates are selected in the candidate list, each candidate points to a reference block, and the selected multiple candidates are combined into a fusion candidate. That is, a fusion candidate includes multiple candidates. The fusion candidate is used for fusion prediction, and multiple fusion candidates can construct a fusion candidate list.

[0202] Optionally, based on the above-described embodiments, the application may further include:

[0203] Divide the candidates in the candidate list into multiple groups, each group containing multiple candidates, and combine all candidates in each group to form a fusion candidate; or

[0204] The candidates in the candidate list are divided into multiple groups, each group containing multiple candidates. Candidates that meet the requirements in each group are selected to form fusion candidates based on given conditions. Candidates in each group may be repeated or may be repeated differently.

[0205] The candidates in the candidate list are divided into multiple groups, each group includes multiple candidates, and candidates in each group can be repeated. All candidates in each group are combined to form a fusion candidate.

[0206] In section 1.2, the fusion model is derived.

[0207] First, determine one or more first thresholds.

[0208] The method for determining the first threshold includes at least one of the following:

[0209] 1) Use the average or median of all pixels in the template region of the current block as the first threshold;

[0210] 2) Use the average or median of all pixels within the template region of the reference block pointed to by all candidates within the fusion candidate as the first threshold;

[0211] 3) Use the average or median of all pixels within the template region of the reference block pointed to by the specified candidate within the fusion candidate as the first threshold; or

[0212] 4) The pixels of the template regions of all candidate reference blocks within the fusion candidate are weighted and fused according to the template cost calculation weights to obtain a predicted template region. The average or median of all pixels within the predicted template region is used as the first threshold. The determination and use of the template cost calculation weights can be as follows:

[0213] Among them, SAD iLet w represent the template cost of the reference block pointed to by the i-th candidate within the fusion candidate, n represent the total number of reference blocks pointed to by all candidates within the fusion candidate, and w represent the template cost of the reference block pointed to by all candidates within the fusion candidate. i p represents the template cost weight of the reference block pointed to by the i-th candidate within the fusion candidate. i p represents the template region of the reference block pointed to by the i-th candidate within the fusion candidate. fusion This indicates the prediction template region.

[0214] Then, the pixel sets of all locations in the template region are classified according to the first threshold. Here, the pixel set of a location represents the set of pixels of the templates of all candidate reference blocks within the fusion candidate at that location, and the pixel set of the current template of the current block at that location. After comparing the pixel sets of all locations with the first threshold, the pixel sets of all locations are divided into multiple groups based on the comparison results, with each group including at least 0 pixels.

[0215] The comparison between the pixel set and the first threshold includes at least one of the following:

[0216] 1) Compare the pixels of the set of pixels at a given position within the template region of the current block with the first threshold;

[0217] 2) Compare the average or median of the set of pixels at a given position within the template region of the reference block pointed to by all candidates within the fusion candidate with a first threshold;

[0218] 3) Compare the average or median of the pixel set at a given location within the template region of the reference block pointed to by the specified candidate within the fusion candidate with a first threshold; or

[0219] 4) After weighted fusion of the template region pixels of the reference block pointed to by the specified candidate within the fusion candidate according to the template cost, a predicted template region is obtained. The average or median of the set of pixels at a given position in the predicted template region is compared with the first threshold.

[0220] The process of determining and using template cost calculation weights can be summarized as follows:

[0221] Among them, SAD i Let w represent the template cost of the reference block pointed to by the i-th candidate within the fusion candidate, n represent the total number of reference blocks pointed to by all candidates within the fusion candidate, and w represent the template cost of the reference block pointed to by all candidates within the fusion candidate. i p represents the template cost weight of the reference block pointed to by the i-th candidate within the fusion candidate. i p represents the template region of the reference block pointed to by the i-th candidate within the fusion candidate. fusion This indicates the prediction template region.

[0222] Finally, a set of fusion model parameters is derived for each pixel set using the Gaussian solver method. Taking two fusion models as an example, the fusion model can be expressed as follows:

[0223] Model:

[0224] Model:

[0225] Where predSamples represents the pixel value of the target position in the current block, and the target position can include any position within the current block, refBlock n w1 is the pixel value of the reference block pointed to by the nth candidate at the target location. n and w2 n These represent the coefficients of the first and second fusion models, respectively. The value of n ranges from 0 to N, where N is the total number of all candidates within the fusion candidate pool. The midValue represents the intermediate value determined based on the preset bit depth, which is a fixed value.

[0226] In section 1.3, we perform fusion prediction.

[0227] Iterate through all positions of the current block, taking each position as a given position, and obtain the pixels at the same position of the reference block pointed to by all candidates in the fusion candidate to form a pixel set. Compare the pixel sets of all given positions with the first threshold. Based on the comparison result, group the pixel sets of each given position. Input the pixel sets of all reference blocks at each given position into the fusion model corresponding to each given position group to obtain the predicted value of the given position.

[0228] The comparison of the set of pixels at all given locations with the first threshold includes at least one of the following:

[0229] 1) Compare the average or median of the set of pixels at a given position in the reference block pointed to by all candidates within the fusion candidate with a first threshold;

[0230] 2) Compare the average or median of the set of pixels at a given location within the reference block pointed to by the specified candidate within the fusion candidate with a first threshold, wherein the reference block pointed to by the specified candidate includes the reference block with the minimum template cost; or

[0231] 3) The pixels of all candidate reference blocks within the fusion candidate are weighted and fused according to the template cost to obtain a prediction region. The pixels at a given position in the prediction region are compared with the first threshold, where the template cost is calculated as described above.

[0232] In one exemplary embodiment, an intra-frame prediction method for video blocks may include the following operations:

[0233] In section 2.1, we construct fusion candidates to participate in the fusion process.

[0234] For example, a candidate list is constructed by means of searching, implicit merging, etc. Multiple candidates are selected in the candidate list, each candidate points to a reference block, and the selected multiple candidates are combined into a fusion candidate. That is, a fusion candidate includes multiple candidates. The fusion candidate is used for fusion prediction, and multiple fusion candidates can construct a fusion candidate list.

[0235] Optionally, based on the above-described embodiments, the application may further include:

[0236] Divide the candidates in the candidate list into multiple groups, each group containing multiple candidates, and combine all candidates in each group to form a fusion candidate; or

[0237] The candidates in the candidate list are divided into multiple groups, each group containing multiple candidates. Candidates meeting the requirements within each group are selected based on given conditions to form the fusion candidate pool. Candidates within each group may be repeated or may have different repeats.

[0238] The candidates in the candidate list are divided into multiple groups, each group includes multiple candidates, and candidates in each group can be repeated. All candidates in each group are combined to form a fusion candidate.

[0239] In section 2.2, the fusion model is derived.

[0240] The average or median of all pixels in the template region of the current block is used as the first threshold. Then, the pixel sets at all positions in the template region are classified according to the first threshold. Classification involves comparing the pixels at a given position in the template region of the current block with the first threshold. It can be understood that the given position can be determined by the position corresponding to the pixel set. Here, the pixel set at a position represents the set of pixels of the templates of all candidate reference blocks within the fusion candidate at that position, and the set of pixels of the current template of the current block at that position. After comparing the pixel sets at all positions with the first threshold, the pixel sets at all positions are divided into multiple groups based on the comparison results, with each group including at least zero pixels. Finally, a set of fusion model parameters is derived for each set of pixels using a Gaussian solver method. Taking two fusion models as an example, the fusion model can be expressed as follows:

[0241] Model:

[0242] Model:

[0243] Where predSamples represents the pixel value of the target position in the current block, and the target position can include any position within the current block, refBlockn w1 is the pixel value of the reference block pointed to by the nth candidate at the target location. n and w2 n These represent the coefficients of the first and second fusion models, respectively. The value of n ranges from 0 to N, where N is the total number of all candidates within the fusion candidate pool. The midValue represents the intermediate value determined based on the preset bit depth, which is a fixed value.

[0244] In section 2.3, we perform fusion prediction.

[0245] Iterate through all positions of the current block, taking each position as a given position, and obtain the pixels at the same position of the reference block pointed to by all candidates in the fusion candidate to form a pixel set. Compare the average value of the pixel set corresponding to the given position in the reference block pointed to by all candidates in the fusion candidate with the first threshold. Based on the comparison result, group the pixel set of each given position. Input the pixel set of all reference blocks at each given position into the fusion model corresponding to each given position group to obtain the predicted value of the given position.

[0246] In one exemplary embodiment, an intra-frame prediction method for video blocks may include the following operations:

[0247] In section 3.1, we construct fusion candidates to participate in the fusion process.

[0248] For example, a candidate list is constructed by means of searching, implicit merging, etc. Multiple candidates are selected in the candidate list, each candidate points to a reference block, and the selected multiple candidates are combined into a fusion candidate. That is, a fusion candidate includes multiple candidates. The fusion candidate is used for fusion prediction, and multiple fusion candidates can construct a fusion candidate list.

[0249] Optionally, based on the above-described embodiments, the application may further include:

[0250] Divide the candidates in the candidate list into multiple groups, each group containing multiple candidates, and combine all candidates in each group to form a fusion candidate; or

[0251] The candidates in the candidate list are divided into multiple groups, each group containing multiple candidates. Candidates meeting the requirements within each group are selected based on given conditions to form the fusion candidate pool. Candidates within each group may be repeated or may have different repeats.

[0252] The candidates in the candidate list are divided into multiple groups, each group includes multiple candidates, and candidates in each group can be repeated. All candidates in each group are combined to form a fusion candidate.

[0253] In section 3.2, the fusion model is derived.

[0254] The first threshold is set by using the average or median of all pixels within the reference template region of the reference block pointed to by all candidates within the fusion candidate. Then, the pixel sets at all positions within the template region are classified according to the first threshold. Classification involves comparing the average or median of the pixel sets at a given position within the reference template region of the reference block pointed to by all candidates within the fusion candidate with the first threshold. It can be understood that a given position can be determined by the position corresponding to the pixel set. Here, the pixel set at a position represents the set of pixels of the template of the reference blocks of all candidates within the fusion candidate at that position, and the set of pixels of the current template of the current block at that position. After comparing the pixel sets at all positions with the first threshold, the pixel sets at all positions are divided into multiple groups based on the comparison results, with each group including at least zero pixels. Finally, a set of fusion model parameters is derived for each group of pixel sets using a Gaussian solver method. Taking two fusion models as an example, the fusion model can be expressed as follows:

[0255] Model:

[0256] Model:

[0257] Where predSamples represents the pixel value of the target position in the current block, and the target position can include any position within the current block, refBlock n w1 is the pixel value of the reference block pointed to by the nth candidate at the target location. n and w2 n These represent the coefficients of the first and second fusion models, respectively. The value of n ranges from 0 to N, where N is the total number of all candidates within the fusion candidate pool. The midValue represents the intermediate value determined based on the preset bit depth, which is a fixed value.

[0258] In section 3.3, we perform fusion prediction.

[0259] Iterate through all positions of the current block, taking each position as a given position, and obtain the pixels at the same position of the reference block pointed to by all candidates in the fusion candidate to form a pixel set. Compare the average value of the pixel set corresponding to the given position in the reference block pointed to by all candidates in the fusion candidate with the first threshold. Based on the comparison result, group the pixel set of each given position. Input the pixel set of all reference blocks at each given position into the fusion model corresponding to each given position group to obtain the predicted value of the given position.

[0260] In one exemplary embodiment, an intra-frame prediction method for video blocks may include the following operations:

[0261] In section 4.1, we construct fusion candidates to participate in the fusion process.

[0262] For example, a candidate list is constructed by means of searching, implicit merging, etc. Multiple candidates are selected in the candidate list, each candidate points to a reference block, and the selected multiple candidates are combined into a fusion candidate. That is, a fusion candidate includes multiple candidates. The fusion candidate is used for fusion prediction, and multiple fusion candidates can construct a fusion candidate list.

[0263] Optionally, based on the above-described embodiments, the application may further include:

[0264] Divide the candidates in the candidate list into multiple groups, each group containing multiple candidates, and combine all candidates in each group to form a fusion candidate; or

[0265] The candidates in the candidate list are divided into multiple groups, each group containing multiple candidates. Candidates meeting the requirements within each group are selected based on given conditions to form the fusion candidate pool. Candidates within each group may be repeated or may have different repeats.

[0266] The candidates in the candidate list are divided into multiple groups, each group includes multiple candidates, and candidates in each group can be repeated. All candidates in each group are combined to form a fusion candidate.

[0267] In section 4.2, the fusion model is derived.

[0268] The average or median of all pixels within the template region of the reference block pointed to by the specified candidate in the fusion candidate is used as the first threshold. Then, the pixel sets at all positions within the template region are classified according to the first threshold. Classification involves comparing the average or median of the pixel sets at a given position within the template region of the reference block pointed to by the specified candidate in the fusion candidate with the first threshold. It can be understood that a given position can be determined by the position corresponding to the pixel set. Here, the pixel set at a position represents the set of pixels of the template of all candidate reference blocks within the fusion candidate at that position, and the set of pixels of the current template of the current block at that position. After comparing the pixel sets at all positions with the first threshold, the pixel sets at all positions are divided into multiple groups based on the comparison results, with each group including at least zero pixels. Finally, a set of fusion model parameters is derived for each group of pixel sets using a Gaussian solver method. Taking two fusion models as an example, the fusion model can be expressed as follows:

[0269] Model:

[0270] Model:

[0271] Where predSamples represents the pixel value of the target position in the current block, and the target position can include any position within the current block, refBlock nw1 is the pixel value of the reference block pointed to by the nth candidate at the target location. n and w2 n These represent the coefficients of the first and second fusion models, respectively. The value of n ranges from 0 to N, where N is the total number of all candidates within the fusion candidate pool. The midValue represents the intermediate value determined based on the preset bit depth, which is a fixed value.

[0272] In section 4.3, we perform fusion prediction.

[0273] Iterate through all positions in the current block, taking each position as a given position. Obtain the pixels at the same position in the reference block pointed to by all candidates within the fusion candidate and form a pixel set. Compare the average or median of the pixel set at the given position in the reference block pointed to by the specified candidate within the fusion candidate with a first threshold. The reference block pointed to by the specified candidate includes the reference block with the minimum template cost. Based on the comparison result, group the pixel set at each given position. Input the pixel set of all reference blocks at each given position into the fusion model corresponding to the group at each given position to obtain the predicted value at the given position.

[0274] In one exemplary embodiment, an intra-frame prediction method for video blocks may include the following operations:

[0275] In section 5.1, we construct fusion candidates to participate in the fusion process.

[0276] For example, a candidate list is constructed by means of searching, implicit merging, etc. Multiple candidates are selected in the candidate list, each candidate points to a reference block, and the selected multiple candidates are combined into a fusion candidate. That is, a fusion candidate includes multiple candidates. The fusion candidate is used for fusion prediction, and multiple fusion candidates can construct a fusion candidate list.

[0277] Optionally, based on the above-described embodiments, the application may further include:

[0278] Divide the candidates in the candidate list into multiple groups, each group containing multiple candidates, and combine all candidates in each group to form a fusion candidate; or

[0279] The candidates in the candidate list are divided into multiple groups, each group containing multiple candidates. Candidates meeting the requirements within each group are selected based on given conditions to form the fusion candidate pool. Candidates within each group may be repeated or may have different repeats.

[0280] The candidates in the candidate list are divided into multiple groups, each group includes multiple candidates, and candidates in each group can be repeated. All candidates in each group are combined to form a fusion candidate.

[0281] In section 5.2, the fusion model is derived.

[0282] The pixels of the template region of the reference block pointed to by all candidates within the fusion candidate are weighted and fused according to the template cost calculation weight to obtain a predicted template region. The average or median of all pixels in the predicted template region is used as the first threshold. The determination and use of the template cost calculation weight can be as follows:

[0283] Among them, SAD i Let w represent the template cost of the reference block pointed to by the i-th candidate within the fusion candidate, n represent the total number of reference blocks pointed to by all candidates within the fusion candidate, and w represent the template cost of the reference block pointed to by all candidates within the fusion candidate. i p represents the template cost weight of the reference block pointed to by the i-th candidate within the fusion candidate. i p represents the template region of the reference block pointed to by the i-th candidate within the fusion candidate. fusion This indicates the prediction template region.

[0284] Then, the pixel sets of all positions in the template region are classified according to the first threshold. Classification includes weighted fusion of the template region pixels of the reference blocks pointed to by the specified candidates within the fusion candidate, calculated according to the template cost, to obtain a predicted template region. The average or median of the pixel set at a given position in the predicted template region is compared with the first threshold. It can be understood that a given position can be determined by the position corresponding to the pixel set. Here, the pixel set at a position represents the set of pixels of the templates of all candidate reference blocks within the fusion candidate at that position, and the set of pixels of the current template of the current block at that position. After comparing the pixel sets of all positions with the first threshold, the pixel sets of all positions are divided into multiple groups based on the comparison results, each group including at least zero pixels. Finally, a set of fusion model parameters is derived for the pixel sets in each group using the Gaussian solver method. Taking two fusion models as an example, the fusion model can be expressed as follows:

[0285] Model:

[0286] Model:

[0287] Where predSamples represents the pixel value of the target position in the current block, and the target position can include any position within the current block, refBlock n w1 is the pixel value of the reference block pointed to by the nth candidate at the target location. n and w2 n These represent the coefficients of the first and second fusion models, respectively. The value of n ranges from 0 to N, where N is the total number of all candidates within the fusion candidate pool. The midValue represents the intermediate value determined based on the preset bit depth, which is a fixed value.

[0288] In section 5.3, we perform fusion prediction.

[0289] Iterate through all positions in the current block, taking each position as a given position. Collect pixels at the same position in the reference blocks pointed to by all candidates within the fusion candidate, forming a pixel set. Calculate the weights of the reference blocks executed by all candidates within the fusion candidate according to the template cost and perform weighted fusion to obtain a prediction region. Compare the pixels at the given position in this prediction region with a first threshold. The template cost weights are calculated as described above. The reference blocks pointed to by the specified candidates include the reference blocks with the minimum template cost. Based on the comparison results, group the pixel sets at each given position. Input the pixel sets of all reference blocks at each given position into the fusion model corresponding to each group at each given position to obtain the predicted value at the given position.

[0290] In one exemplary embodiment, an intra-frame prediction method for video blocks at the encoding end may include the following operations:

[0291] In section 6.1, we construct fusion candidates to participate in the fusion process.

[0292] For example, a candidate list is constructed through methods such as searching and implicit merging. Multiple candidates are selected from the candidate list, each pointing to a reference block. The selected candidates are combined into a fusion candidate, that is, a fusion candidate includes multiple candidates. This fusion candidate is used for fusion prediction, and multiple fusion candidates can construct a fusion candidate list. The candidates within the fusion candidate are sorted in ascending order according to their corresponding template costs.

[0293] Optionally, based on the above-described embodiments, the application may further include:

[0294] Divide the candidates in the candidate list into multiple groups, each group containing multiple candidates, and combine all candidates in each group to form a fusion candidate; or

[0295] The candidates in the candidate list are divided into multiple groups, each containing multiple candidates. Candidates meeting certain criteria within each group are selected to form the fusion candidates based on given conditions. Candidates within each group can be repeated or distinctly repeated. Given conditions include: if the ratio of the current candidate's template cost to the minimum template cost is greater than a first threshold, the current candidate does not participate in the fusion. Candidates participating in the fusion are selected from each group according to the threshold; or...

[0296] The candidates in the candidate list are divided into multiple groups, each group includes multiple candidates, and candidates in each group can be repeated. All candidates in each group are combined to form a fusion candidate.

[0297] In section 6.2, the fusion model is derived.

[0298] For any set of fusion candidates, the first step is to obtain the template region of the reference block corresponding to each candidate, as well as the template region of the current block.

[0299] If LIC technology is enabled, LIC prediction is performed on the template region of the reference block first.

[0300] First, determine one or more second thresholds.

[0301] The method for determining the second threshold includes at least one of the following:

[0302] 1) Use the average or median of all pixels in the template region of the current block as the second threshold;

[0303] 2) Use the average or median of all pixels within the template region of the reference block pointed to by all candidates within the fusion candidate as the second threshold;

[0304] 3) Use the average or median of all pixels within the template region of the reference block pointed to by the specified candidate within the fusion candidate as the second threshold; or

[0305] 4) The pixels of the template regions of all candidate reference blocks within the fusion candidate are weighted and fused according to the template cost calculation weights to obtain a predicted template region. The average or median of all pixels within the predicted template region is used as the second threshold. The determination and use of the template cost calculation weights can be as follows:

[0306] Among them, SAD i Let w represent the template cost of the reference block pointed to by the i-th candidate within the fusion candidate, n represent the total number of reference blocks pointed to by all candidates within the fusion candidate, and w represent the template cost of the reference block pointed to by all candidates within the fusion candidate. i p represents the template cost weight of the reference block pointed to by the i-th candidate within the fusion candidate. i p represents the template region of the reference block pointed to by the i-th candidate within the fusion candidate. fusion This indicates the prediction template region.

[0307] Then, the pixel sets at all locations within the template region are classified according to the second threshold. Here, the pixel set at a location represents the set of pixels of the templates of all candidate reference blocks within the fusion candidate at that location, and the pixel set of the current template of the current block at that location. After comparing all pixel sets at all locations with the second threshold, based on the comparison results, all pixel sets at all locations are divided into multiple groups, each group including at least 0 pixels.

[0308] The comparison between the pixel set and the second threshold includes at least one of the following:

[0309] 1) Compare the pixels of the set of pixels at a given position within the template region of the current block with the second threshold;

[0310] 2) Compare the average or median of the pixel set at a given position within the template region of the reference block pointed to by all candidates within the fusion candidate with the second threshold.

[0311] 3) Compare the average or median of the pixel set at a given location within the template region of the reference block pointed to by the specified candidate within the fusion candidate, and the second threshold; or

[0312] 4) After weighted fusion of the template region pixels of the reference block pointed to by the specified candidate within the fusion candidate according to the template cost, a predicted template region is obtained. The average or median of the set of pixels at a given position in the predicted template region is compared with the second threshold.

[0313] The process of determining and using template cost calculation weights can be summarized as follows:

[0314] Among them, SAD i Let w represent the template cost of the reference block pointed to by the i-th candidate within the fusion candidate, n represent the total number of reference blocks pointed to by all candidates within the fusion candidate, and w represent the template cost of the reference block pointed to by all candidates within the fusion candidate. i p represents the template cost weight of the reference block pointed to by the i-th candidate within the fusion candidate. i p represents the template region of the reference block pointed to by the i-th candidate within the fusion candidate. fusion This indicates the prediction template region.

[0315] Finally, a set of fusion model parameters is derived for each pixel set using the Gaussian solver method. Taking two fusion models as an example, the fusion model can be expressed as follows:

[0316] Model:

[0317] Model:

[0318] Where predSamples represents the pixel value of the target position in the current block, and the target position can include any position within the current block, refBlock n w1 is the pixel value of the reference block pointed to by the nth candidate at the target location. n and w2 n These represent the coefficients of the first and second fusion models, respectively. The value of n ranges from 0 to N, where N is the total number of all candidates within the fusion candidate pool. The midValue represents the intermediate value determined based on the preset bit depth, which is a fixed value.

[0319] In section 6.3, we perform fusion prediction.

[0320] Iterate through all positions of the current block, taking each position as a given position, and obtain the pixels at the same position of the reference block pointed to by all candidates in the fusion candidate to form a pixel set. Compare the pixel sets of all given positions with the second threshold. Based on the comparison result, group the pixel sets of each given position. Input the pixel sets of all reference blocks at each given position into the fusion model corresponding to each given position group to obtain the predicted value of the given position.

[0321] The comparison of the set of pixels at all given locations with the second threshold includes at least one of the following:

[0322] 1) Compare the average or median of the set of pixels at a given location in the reference block pointed to by all candidates within the fusion candidate with a second threshold;

[0323] 2) Compare the average or median of the set of pixels at a given location within the reference block pointed to by the specified candidate within the fusion candidate with a second threshold, wherein the reference block pointed to by the specified candidate includes the reference block with the minimum template cost; or

[0324] 3) The pixels of the reference blocks of all candidates within the fusion candidate are weighted and fused according to the template cost to obtain a prediction region. The pixels at a given position in the prediction region are compared with the second threshold, where the template cost is calculated as described above.

[0325] In 6.4, the identifier of the fusion mode is transmitted to the decoding end.

[0326] For example, rate-distortion cost can be calculated at the encoding end for the above processing. If the current fusion mode has the lowest mode cost, the identifier of the fusion mode is set to true; otherwise, it is set to false. The identifier of the fusion mode is then encoded into the bitstream and transmitted to the decoding end.

[0327] In this embodiment of the application, the identifier of the fusion mode can be implemented using the coding unit identifier syntax shown in the table below.

[0328] The semantic descriptions of the relevant syntax fields in the table are as follows:

[0329] tmpFlag: Represents the IntraTMP enable flag, used to indicate whether the encoding unit enables the IntraTMP method. A value of 1 indicates that it is enabled, and a value of 0 indicates that it is not enabled. The representation of the syntax fields is not limited to the embodiments of this application.

[0330] tmpFusionFlag: This flag indicates whether IntraTMP fusion technology is enabled. It is used to indicate whether the coding unit enables the IntraTMP fusion prediction method. A value of 1 indicates that it is enabled, and a value of 0 indicates that it is not enabled. The representation of the syntax fields is not limited to the embodiments of this application.

[0331] tmpFusionIdx: Identifies the fusion candidate type used by the IntraTMP fusion technique, indicating the method used by the coding unit for IntraTMP fusion prediction. The value is 0-2. When the value is 0, it means that the weighted fusion uses template cost to calculate the fusion weights. When the value is 1, it means that the single-model Wiener filter is used for weighted fusion. When the value is 2, it means that the multi-model Wiener filter is used for weighted fusion.

[0332] tmpIdx: Identifies the fusion candidate index used by the IntraTMP fusion technique. It indicates the index value of the candidate predicted by the coding unit using IntraTMP fusion. The value is 0-2. When the value is 0, it means that the candidate comes from the 0th to 4th candidate in the candidate list. When the value is 1, it means that the candidate comes from the 5th to 9th candidate in the candidate list. When the value is 2, it means that the candidate comes from the 10th to 14th candidate in the candidate list.

[0333] In this embodiment of the application, the identifier of the fusion mode can also be implemented using the coding unit identifier syntax shown in the table below.

[0334] The semantic descriptions of the relevant syntax fields in the table are as follows:

[0335] tmpFlag: Represents the IntraTMP enable flag, used to indicate whether the encoding unit enables the IntraTMP method. A value of 1 indicates that it is enabled, and a value of 0 indicates that it is not enabled. The representation of the syntax fields is not limited to the embodiments of this application.

[0336] tmpFusionFlag: This flag indicates whether IntraTMP fusion technology is enabled. It is used to indicate whether the coding unit enables the IntraTMP fusion prediction method. A value of 1 indicates that it is enabled, and a value of 0 indicates that it is not enabled. The representation of the syntax fields is not limited to the embodiments of this application.

[0337] tmpFusionIdx: Identifies the fusion candidate type used by the IntraTMP fusion technique, indicating the method used by the coding unit for IntraTMP fusion prediction. The value is 0-1; a value of 0 indicates that weighted fusion uses template cost to calculate the fusion weights; a value of 1 indicates that Wiener filters are used for weighted fusion.

[0338] tmpIdx: Identifies the fusion candidate index used by the IntraTMP fusion technique. It indicates the index value of the candidate predicted by the coding unit using IntraTMP fusion. The value is 0-2. When the value is 0, it means that the candidate comes from the 0th to 4th candidate in the candidate list. When the value is 1, it means that the candidate comes from the 5th to 9th candidate in the candidate list. When the value is 2, it means that the candidate comes from the 10th to 14th candidate in the candidate list.

[0339] tmpMmFusionFlag: Indicates whether IntraTMP uses a multi-model Wiener filter for weighted fusion, with a value of 0-1. When the value is 0, it means that IntraTMP uses a single-model Wiener filter for weighted fusion; when the value is 1, it means that IntraTMP uses a multi-model Wiener filter for weighted fusion.

[0340] In one exemplary embodiment, an intra-frame prediction method for video blocks at the encoding end may include the following operations:

[0341] Section 7.1 parses the IntraTMP related syntax.

[0342] For example, the code parses syntax symbols from the bitstream indicating whether IntraTMP is enabled and whether IntraTMP fusion prediction is enabled. If both are enabled, it continues parsing the fusion prediction type and fusion candidate index. If it belongs to the multi-model Wiener filter fusion type, it proceeds to the next step; otherwise, it terminates.

[0343] In section 7.2, we construct fusion candidates to participate in the fusion process.

[0344] For example, a candidate list is constructed through methods such as searching and implicit merging. Multiple candidates are selected from the candidate list, each pointing to a reference block. The selected candidates are combined into a fusion candidate, that is, a fusion candidate includes multiple candidates. The fusion candidate is used for fusion prediction, and multiple fusion candidates can construct a fusion candidate list. All candidates within the fusion candidate are sorted in ascending order according to their corresponding template costs.

[0345] Optionally, based on the above-described embodiments, the application may further include:

[0346] The candidates in the candidate list are divided into multiple groups, each group includes multiple candidates, and all candidates in each group are combined to form a fusion candidate.

[0347] The candidates in the candidate list are divided into multiple groups, each containing multiple candidates. Candidates meeting certain criteria within each group are selected to form the fusion candidates based on given conditions. Candidates within each group can be repeated or distinctly repeated. Given conditions include: if the ratio of the current candidate's template cost to the minimum template cost is greater than a first threshold, the current candidate does not participate in the fusion. Candidates participating in the fusion are selected from each group according to the threshold; or...

[0348] The candidates in the candidate list are divided into multiple groups, each group includes multiple candidates, and candidates in each group can be repeated. All candidates in each group are combined to form a fusion candidate.

[0349] In section 7.3, the fusion model is derived.

[0350] For any set of fusion candidates, first obtain the template region of the reference block corresponding to each candidate, and the template region of the current block. The template region includes the surrounding reconstruction region of the current block or the reference block. The surrounding reconstruction region includes at least one of the left pixel region, the top pixel region, the upper left pixel region, the lower left pixel region, and the upper right pixel region.

[0351] If LIC technology is enabled, LIC prediction is performed on the template region of the reference block first.

[0352] First, determine one or more second thresholds.

[0353] The method for determining the second threshold includes at least one of the following:

[0354] 1) Use the average or median of all pixels in the template region of the current block as the second threshold;

[0355] 2) Use the average or median of all pixels within the template region of the reference block pointed to by all candidates within the fusion candidate as the second threshold;

[0356] 3) Use the average or median of all pixels within the template region of the reference block pointed to by the specified candidate within the fusion candidate as the second threshold; or

[0357] 4) The pixels of the template regions of all candidate reference blocks within the fusion candidate are weighted and fused according to the template cost calculation weights to obtain a predicted template region. The average or median of all pixels within the predicted template region is used as the second threshold. The determination and use of the template cost calculation weights can be as follows:

[0358] Among them, SAD i Let w represent the template cost of the reference block pointed to by the i-th candidate within the fusion candidate, n represent the total number of reference blocks pointed to by all candidates within the fusion candidate, and w represent the template cost of the reference block pointed to by all candidates within the fusion candidate. ip represents the template cost weight of the reference block pointed to by the i-th candidate within the fusion candidate. i p represents the template region of the reference block pointed to by the i-th candidate within the fusion candidate. fusion This indicates the prediction template region.

[0359] Then, the pixel sets at all locations within the template region are classified according to the second threshold. Here, the pixel set at a location represents the set of pixels of the templates of all candidate reference blocks within the fusion candidate at that location, and the pixel set of the current template of the current block at that location. After comparing all pixel sets at all locations with the second threshold, based on the comparison results, all pixel sets at all locations are divided into multiple groups, each group including at least 0 pixels.

[0360] The comparison between the pixel set and the second threshold includes at least one of the following:

[0361] 1) Compare the pixels of the set of pixels at a given position within the template region of the current block with the second threshold;

[0362] 2) Compare the average or median of the pixel set at a given position within the template region of the reference block pointed to by all candidates within the fusion candidate with the second threshold.

[0363] 3) Compare the average or median of the pixel set at a given location within the template region of the reference block pointed to by the specified candidate within the fusion candidate, and the second threshold; or

[0364] 4) After weighted fusion of the template region pixels of the reference block pointed to by the specified candidate within the fusion candidate according to the template cost, a predicted template region is obtained. The average or median of the set of pixels at a given position in the predicted template region is compared with the second threshold.

[0365] The process of determining and using template cost calculation weights can be summarized as follows:

[0366] Among them, SAD i Let w represent the template cost of the reference block pointed to by the i-th candidate within the fusion candidate, n represent the total number of reference blocks pointed to by all candidates within the fusion candidate, and w represent the template cost of the reference block pointed to by all candidates within the fusion candidate. i p represents the template cost weight of the reference block pointed to by the i-th candidate within the fusion candidate. i p represents the template region of the reference block pointed to by the i-th candidate within the fusion candidate. fusion This indicates the prediction template region.

[0367] Finally, a set of fusion model parameters is derived for each pixel set using the Gaussian solver method. Taking two fusion models as an example, the fusion model can be expressed as follows:

[0368] Model:

[0369] Model:

[0370] Where predSamples represents the pixel value of the target position in the current block, and the target position can include any position within the current block, refBlock n w1 is the pixel value of the reference block pointed to by the nth candidate at the target location. n and w2 n These represent the coefficients of the first and second fusion models, respectively. The value of n ranges from 0 to N, where N is the total number of all candidates within the fusion candidate pool. The midValue represents the intermediate value determined based on the preset bit depth, which is a fixed value.

[0371] Based on the above application embodiments, the template cost includes at least one of the following: absolute difference and cost, squared difference and cost, and transform domain absolute difference and cost.

[0372] In section 7.4, we perform fusion prediction.

[0373] Iterate through all positions of the current block, taking each position as a given position, and obtain the pixels at the same position of the reference block pointed to by all candidates in the fusion candidate to form a pixel set. Compare the pixel sets of all given positions with the second threshold. Based on the comparison result, group the pixel sets of each given position. Input the pixel sets of all reference blocks at each given position into the fusion model corresponding to each given position group to obtain the predicted value of the given position.

[0374] The comparison of the set of pixels at all given locations with the second threshold includes at least one of the following:

[0375] 1) Compare the average or median of the set of pixels at a given location in the reference block pointed to by all candidates within the fusion candidate with a second threshold;

[0376] 2) Compare the average or median of the set of pixels at a given location within the reference block pointed to by the specified candidate within the fusion candidate with a second threshold, wherein the reference block pointed to by the specified candidate includes the reference block with the minimum template cost; or

[0377] 3) The pixels of the reference blocks of all candidates within the fusion candidate are weighted and fused according to the template cost to obtain a prediction region. The pixels at a given position in the prediction region are compared with the second threshold, where the template cost is calculated as described above.

[0378] Based on the above application embodiments, this application embodiment is applied only when the width or height of the current block is greater than or equal to a given value 1, or the area is greater than or equal to a given value 2.

[0379] Optionally, based on the above application embodiments, each embodiment can be controlled by syntax element symbols to determine whether to enable them.

[0380] In the video layer of the encoded bitstream, i.e., the Video Parameter Set (VPS), the enable flag selected by the method of the embodiments of this application is notified by signaling.

[0381] Table 1 is a syntax table of the video parameter set at the encoded bitstream video level, as shown below. It should be noted that the value of each identifier in the table of this application embodiment is only an exemplary illustration. In practical applications, other types of values ​​can also be taken. For example, "true" and "false" can be used to replace "1" and "0", or other types of characters or symbols can be used to replace "1" and "0", etc.

[0382] Table 1. Syntax table of video parameter sets for encoded bitstream video layers.

[0383] The semantic descriptions of the relevant syntax fields in Table 1 are as follows:

[0384] `vps_itmp_fusion_mm_filter_enabled_flag`: This flag indicates the enabling function of the method in this embodiment of the application. It is used to indicate whether the method of this embodiment is enabled at the video layer level. A value of 1 indicates that the method is enabled, and a value of 0 indicates that the method is not enabled. The representation of the syntax fields is not limited to this embodiment of the application.

[0385] In some embodiments of this application, the enable flag for using the method of this application is notified by signaling at the sequence level of the encoded bit stream, i.e., the sequence parameter set (SPS).

[0386] Table 2 is a syntax table for the video parameter set at the encoded bitstream sequence level, as shown below.

[0387] Table 2 Syntax table of sequence parameter sets for encoded bitstream sequence levels

[0388] The semantic descriptions of the relevant syntax fields in Table 2 are as follows:

[0389] sps_itmp_fusion_mm_filter_enabled_flag: This flag indicates the enabling status of the method in this embodiment of the application. It is used to indicate whether the method in this embodiment of the application is enabled at the sequence level. A value of 1 indicates that the method is enabled, and a value of 0 indicates that the method is not enabled. The representation of the syntax fields is not limited to this embodiment of the application.

[0390] In some embodiments of this application, the enable flag selected by the method of this application is notified by signaling at the image level of the encoded bitstream, i.e., the Picture Parameter Set (PPS) or the Picture Header (PH).

[0391] Table 3 is the syntax table for the video parameter set at the encoded bitstream image level, as shown below.

[0392] Table 3. Syntax table of image parameter sets for encoded bitstream image levels.

[0393] The semantic descriptions of the relevant syntax fields in Table 3 are as follows:

[0394] ph_itmp_fusion_mm_filter_enabled_flag: This flag indicates whether the method of this embodiment is enabled at the image layer level. A value of 1 indicates that the method is enabled, and a value of 0 indicates that the method is not enabled. The representation of the syntax fields is not limited to the embodiments of this application.

[0395] In some embodiments of this application, the enable flag for using the method of this application is notified by signaling at the slice level of the encoded bitstream, i.e., the slice header (SH).

[0396] Table 4 is the syntax table for the chip header (SH) at the bitstream level, as shown below.

[0397] Table 4 Syntax table for the header of the encoded bitstream slice level

[0398] The semantic descriptions of the relevant syntax fields in Table 4 are as follows:

[0399] sh_itmp_fusion_mm_filter_enabled_flag: This flag indicates the enabling flag of the method in this embodiment of the application. It is used to indicate whether the method of this embodiment is enabled at the slice level. A value of 1 indicates that it is enabled, and a value of 0 indicates that it is not enabled. The representation of the syntax fields is not limited to this embodiment of the application.

[0400] In some embodiments of this application, the enable flag for using the method selected in the embodiments of this application is notified by signaling at the coding unit level of the encoded bitstream, i.e., in the coding unit syntax.

[0401] Table 5 is the syntax table of coding units at the coding unit level of the coding bitstream, as shown below.

[0402] Table 5 Syntax table of encoded bitstream coding units

[0403] The semantic descriptions of the relevant syntax fields in Table 5 are as follows:

[0404] `itmp_fusion_mm_filter_flag`: This flag indicates the enable flag for the method in this embodiment of the application. It is used to determine whether the encoding unit enables the method in this embodiment of the application. A value of 1 indicates that the method is enabled, and a value of 0 indicates that the method is not enabled. The representation of the syntax fields is not limited to this embodiment of the application.

[0405] Figure 13 is a schematic diagram of an image information processing apparatus provided in an embodiment of this application. This apparatus can execute the image information processing method provided in any embodiment of this application, and possesses the corresponding functional modules and beneficial effects of executing the method. This apparatus can be implemented by software and / or hardware. As shown in Figure 13, the apparatus provided in this embodiment includes:

[0406] The fusion list module 710 is set to determine the candidate reference block list of the target coding block and construct fusion candidates based on the candidate reference block list.

[0407] The pixel classification module 720 is configured to classify pixels at different positions within the target coding block and the template region of the fusion candidate according to the fusion classification threshold, so as to obtain at least one pixel classification set.

[0408] The model determination module 730 is configured to determine the fusion model based on at least two sets of pixels at different positions within each pixel classification set.

[0409] The information prediction module 740 is configured to obtain the corresponding target reference pixel in all candidate reference blocks within the fusion candidate based on each position of the target coding block, determine the matching target fusion model in each fusion model based on each target reference pixel, and determine the predicted pixel value of each pixel in the target coding block based on the target fusion model and the target reference pixel.

[0410] Based on the above-described embodiments, the pixel classification module 720 obtains the fusion classification threshold, including at least one of the following:

[0411] Use the average or median of all pixels within the template region of the target coding block as the fusion classification threshold;

[0412] The average or median of all pixels in the template region of each candidate reference block within the fusion candidate is used as the fusion classification threshold.

[0413] Use the average or median of all pixels in the template region of the specified candidate reference block in the fusion candidate as the fusion classification threshold; or

[0414] All pixels in the template regions of all candidate reference blocks in the fusion candidate are weighted according to the template cost weight of the corresponding candidate reference block and fused into a predicted template region. The average or median of all pixels in the predicted template region is used as the fusion classification threshold.

[0415] Based on the above-mentioned application embodiments, it further includes: a local illumination prediction module, configured to perform local illumination compensation prediction on the second template region of the candidate reference block within the fusion candidate.

[0416] In some application embodiments, the template region includes at least one of the left pixel region, the top pixel region, the upper left pixel region, the lower left pixel region, and the upper right pixel region of the target coding block or the candidate reference block within the fusion candidate.

[0417] In some application embodiments, the fusion list module 710 is configured with at least one of the following:

[0418] Obtain a list of candidate reference blocks for the target coding block, and divide each candidate reference block in the list into at least one candidate reference block group without repetition, and use each candidate reference block group as a fusion candidate.

[0419] Obtain a list of candidate reference blocks for the target coding block; divide each candidate reference block in the list into at least one candidate reference block group; filter the candidate reference blocks within each candidate reference block group based on given conditions; and use the filtered candidate reference block groups as fusion candidates. The given conditions include at least one of pixel threshold and template cost threshold; or

[0420] Obtain a list of candidate reference blocks for the target coding block, divide each candidate reference block in the list into at least one group of candidate reference blocks, and use each group of candidate reference blocks as a fusion candidate, wherein at least two groups of candidate reference blocks have at least one duplicate candidate reference block.

[0421] In some embodiments, the pixel classification module 720 includes:

[0422] The template determination unit is configured to determine the first template region of the target coding block and the second template region of the candidate reference block within the fusion candidate.

[0423] The pixel classification unit is configured to compare the pixels at the same position in the first template region and the second template region with the fusion classification threshold respectively, so as to classify the pixels at the same position into the pixel classification set.

[0424] Based on the above-described embodiments, the pixel classification unit is configured with at least one of the following:

[0425] Each pixel within the first template region is compared with the fusion classification threshold.

[0426] The average value of each pixel in the second template region of all candidate reference blocks within the fusion candidate is compared with the fusion classification threshold.

[0427] Compare the pixels at each position within the second template region of the specified candidate reference block within the fusion candidate with the fusion classification threshold; or

[0428] The pixels at each position of the template region of all candidate reference blocks in the fusion candidate are weighted according to the template cost weight of the corresponding candidate reference block and fused into a predicted template region. The pixels at each position in the predicted template region are compared with the fusion classification threshold.

[0429] Based on the above application embodiments, the designated candidate reference block includes the reference block with the minimum template cost among the fusion candidates.

[0430] Based on the above application embodiments, it further includes: a weight acquisition module, configured to acquire the template cost weight of each candidate reference block, wherein the template cost weight is the difference between the sum of the template costs of each candidate reference block and the template cost of the candidate reference block, and the ratio between the product of the number of remaining candidate reference blocks (excluding the candidate reference block) and the sum of the template costs within the fusion candidate.

[0431] Based on the above-described embodiments, the model determination module 730 includes:

[0432] The pixel pair unit is configured to extract the first and second pixels at the same position in the first template region of the target coding block and the second template region of the candidate reference block in the fusion candidate for each pixel classification set, forming a pixel group.

[0433] The parameter determination unit is configured to determine the model parameters of the fusion model based on at least two pixel groups.

[0434] Based on the above-described embodiments, the information prediction module 740 includes:

[0435] The pixel group unit is set to obtain the target reference pixel at the corresponding position in all candidate reference blocks within the fusion candidate for each position within the target coding block.

[0436] The target model unit is configured to compare the target reference pixel at each location with the fusion classification threshold, determine the target classification set to which the target reference pixel at each location belongs in the pixel classification set, and use the fusion model corresponding to the target classification set as the target fusion model.

[0437] The fusion prediction unit is configured to input each target reference pixel into the corresponding target fusion model to obtain the predicted pixel value of the target coding block at the specified location.

[0438] Based on the above-described embodiments, the target model unit compares the target reference pixel at each location with a fusion classification threshold, including at least one of the following:

[0439] The average value of the target reference pixels at each position within all candidate reference blocks in the fusion candidate is compared with the fusion classification threshold;

[0440] Compare the target reference pixels at each position of the specified candidate reference block in the fusion candidate with the fusion classification threshold; or

[0441] The target reference pixels at each position in all candidate reference blocks in the fusion candidate are weighted and fused according to their respective template cost weights, and then compared with the fusion classification threshold.

[0442] Based on the above application embodiments, the template cost includes at least one of the following: absolute difference and cost, squared difference and cost, or transform domain absolute difference and cost.

[0443] Based on the above-described embodiments, the application further includes: a condition determination module, configured to determine that the target coding block satisfies preset conditions, wherein the preset conditions include at least one of the following:

[0444] The width of the target coded block is greater than or equal to the first threshold;

[0445] The height of the target coded block is greater than or equal to the second threshold; or

[0446] The area of ​​the target coded block is greater than or equal to the third threshold.

[0447] Figure 14 is a schematic diagram of another image information processing device provided in an embodiment of this application. This device can execute the image information processing method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method. This device can be implemented by software and / or hardware, and is generally used in the encoding end. As shown in Figure 14, the device provided in this embodiment includes:

[0448] The fusion list module 810 is set to determine the candidate reference block list of the target coding block and construct fusion candidates based on the candidate reference block list.

[0449] The pixel classification module 820 is configured to classify pixels at different positions within the target coding block and the template region of the fusion candidate according to the fusion classification threshold, so as to obtain at least one pixel classification set.

[0450] The model determination module 830 is configured to determine the fusion model based on at least two sets of pixels at different positions within each pixel classification set.

[0451] The information prediction module 840 is configured to obtain the corresponding target reference pixel in all candidate reference blocks within the fusion candidate based on each position of the target coding block, determine the matching target fusion model in each fusion model based on each target reference pixel, and determine the predicted pixel value of each pixel in the target coding block based on the target fusion model and the target reference pixel.

[0452] The instruction transmission module 850 is configured to transmit instruction information indicating the image information processing method to the decoding end.

[0453] Based on the above-described embodiments, the instruction information includes at least one of the following:

[0454] Intra-frame prediction indication information, wherein the intra-frame prediction indication information indicates whether intra-frame prediction mode is enabled or disabled in image information processing;

[0455] Intra-frame template fusion prediction information, wherein the intra-frame template fusion prediction information indicates whether the image information processing enables or disables the template fusion-based intra-frame prediction mode;

[0456] Intra-frame template fusion type, wherein the intra-frame template fusion type indicates the type of intra-frame template fusion prediction used by the coding unit in image information processing, and the type includes at least one of template cost weighted fusion, single-model Wiener filter weighted fusion, or multi-model Wiener filter weighted fusion;

[0457] Fusion candidate index, wherein the fusion candidate index indicates the fusion candidates participating in the fusion process during image information processing; or

[0458] Wiener filter indication information, wherein the Wiener filter indication information indicates the type of Wiener filter used in image information processing.

[0459] Based on the above application embodiments, the indication level of the indication information includes at least one of the following: video level; sequence level; image level; slice level; or coding unit level.

[0460] Based on the above-described embodiments, the instruction information includes at least one of the following:

[0461] Intra-prediction indication information (set to enabled), intra-template fusion prediction information (set to enabled), intra-template fusion type and fusion candidate index (set to weighted fusion using multi-model Wiener filter); or

[0462] Intra-prediction indication information when set to enabled, intra-template fusion prediction information when set to enabled, intra-template fusion type when set to weighted fusion using Wiener filter, Wiener filter indication information when set to multi-model Wiener filter, and fusion candidate index.

[0463] Figure 15 is a schematic diagram of another image information processing device provided in an embodiment of this application. This device can execute the image information processing method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method. This device can be implemented by software and / or hardware, and is generally used at the decoding end. As shown in Figure 15, the device provided in this embodiment includes:

[0464] The instruction receiving module 910 is configured to acquire instruction information of the instruction image information processing method transmitted by the encoding end.

[0465] The fusion list module 920 is set to determine the candidate reference block list of the target coding block and construct fusion candidates based on the candidate reference block list.

[0466] The pixel classification module 930 is configured to classify pixels at different positions within the target coding block and the template region of the fusion candidate according to the fusion classification threshold, so as to obtain at least one pixel classification set.

[0467] The model determination module 940 is configured to determine the fusion model based on at least two sets of pixels at different positions within each pixel classification set.

[0468] The information prediction module 950 is configured to obtain the corresponding target reference pixel in all candidate reference blocks within the fusion candidate based on each position of the target coding block, determine the matching target fusion model in each fusion model based on each target reference pixel, and determine the predicted pixel value of each pixel in the target coding block based on the target fusion model and the target reference pixel.

[0469] Based on the above application embodiments, the indication level of the indication information includes at least one of the following: video level; sequence level; image level; slice level; or coding unit level.

[0470] Based on the above-described embodiments, the instruction information includes at least one of the following:

[0471] Intra-prediction indication information (set to enabled), intra-template fusion prediction information (set to enabled), intra-template fusion type and fusion candidate index (set to weighted fusion using multi-model Wiener filter); or

[0472] Intra-prediction indication information when set to enabled, intra-template fusion prediction information when set to enabled, intra-template fusion type when set to weighted fusion using Wiener filter, Wiener filter indication information when set to multi-model Wiener filter, and fusion candidate index.

[0473] Figure 16 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device includes a processor 10, a memory 11, an input device 12, and an output device 13. The number of processors 10 in the electronic device can be one or more. Figure 16 shows one processor 10 as an example. The processor 10, memory 11, input device 12, and output device 13 in the electronic device can be connected by a bus or other means. Figure 16 shows a connection via a bus as an example.

[0474] The memory 11, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the modules corresponding to the image information processing device in the embodiments of this application (fusion list module 710, pixel classification module 720, model determination module 730, and information prediction module 740; or fusion list module 810, pixel classification module 820, model determination module 830, information prediction module 840, and instruction transmission module 850; or instruction receiving module 910, fusion list module 920, pixel classification module 930, model determination module 940, and information prediction module 950). The processor 10 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 11, thereby implementing the above-described method.

[0475] The memory 11 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 11 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 11 may further include memory remotely located relative to the processor 10, which can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0476] Input device 12 is configured to receive input digital or character information and generate key signal inputs related to user settings and function control of the electronic device. Output device 13 may include display devices such as a display screen.

[0477] This application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform an image information processing method, including:

[0478] Determine a list of candidate reference blocks for the target coding block, and construct fusion candidates based on the list of candidate reference blocks;

[0479] Based on the fusion classification threshold, the pixels at different positions within the template regions of the target coding block and the fusion candidate are classified to obtain at least one pixel classification set.

[0480] For each pixel classification set, a fusion model is determined based on at least two groups of pixels corresponding to different positions within the pixel classification set;

[0481] Based on the position of the target coding block, obtain the corresponding target reference pixel in all candidate reference blocks within the fusion candidate. Based on each target reference pixel, determine the matching target fusion model in each fusion model. Based on the target fusion model and the target reference pixel, determine the predicted pixel value of each pixel in the target coding block.

[0482] Alternatively, image information processing methods include:

[0483] Determine a list of candidate reference blocks for the target coding block, and construct fusion candidates based on the list of candidate reference blocks;

[0484] Based on the fusion classification threshold, the pixels at different positions within the template regions of the target coding block and the fusion candidate are classified to obtain at least one pixel classification set.

[0485] For each pixel classification set, a fusion model is determined based on at least two groups of pixels corresponding to different positions within the pixel classification set;

[0486] Based on the position of the target coding block, obtain the corresponding target reference pixel in all candidate reference blocks within the fusion candidate; determine the matching target fusion model in each fusion model based on each target reference pixel; and determine the predicted pixel value of each pixel in the target coding block based on the target fusion model and the target reference pixel.

[0487] The instruction information indicating the image information processing method is transmitted to the decoding end.

[0488] Alternatively, image information processing methods include:

[0489] Obtain the instruction information of the image information processing method transmitted from the encoding end;

[0490] Determine a list of candidate reference blocks for the target coding block, and construct fusion candidates based on the list of candidate reference blocks;

[0491] Based on the fusion classification threshold, the pixels at different positions within the template regions of the target coding block and the fusion candidate are classified to obtain at least one pixel classification set.

[0492] For each pixel classification set, a fusion model is determined based on at least two groups of pixels corresponding to different positions within the pixel classification set;

[0493] Based on the position of the target coding block, obtain the corresponding target reference pixel in all candidate reference blocks within the fusion candidate. Based on each target reference pixel, determine the matching target fusion model in each fusion model. Based on the target fusion model and the target reference pixel, determine the predicted pixel value of each pixel in the target coding block.

[0494] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this application can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0495] In some embodiments, the image processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as memory. In some embodiments, part or all of the computer program may be loaded and / or mounted on an electronic device via ROM 12 and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the image processing method by any other suitable means (e.g., by means of firmware). The computer program for the method in the embodiments of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus such that, when executed by the processor, the computer program causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program may be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine as a standalone software package, or entirely on a remote machine or server.

[0496] In the context of embodiments of this application, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0497] It is worth noting that in the embodiments of the above-mentioned device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of this application.

[0498] Those skilled in the art will understand that all or some of the steps, apparatuses, or functional modules / units in the methods disclosed above can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0499] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. The corresponding software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0500] The above description, with reference to the accompanying drawings, illustrates embodiments of this application but does not limit the scope of this application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of this application shall be within the scope of this application.

Claims

1. An image information processing method, comprising: Determine a list of candidate reference blocks for the target coding block, and construct fusion candidates based on the list of candidate reference blocks; Based on the fusion classification threshold, the pixels at different positions within the template regions of the target coding block and the fusion candidate are classified to obtain at least one pixel classification set. For each pixel classification set, a fusion model is determined based on at least two groups of pixels corresponding to different positions within the pixel classification set; Based on the position of the target coding block, obtain the corresponding target reference pixel in all candidate reference blocks within the fusion candidate. Based on each target reference pixel, determine the matching target fusion model in each fusion model. Based on the target fusion model and the target reference pixel, determine the predicted pixel value of each pixel in the target coding block.

2. The method according to claim 1, further comprising: Obtaining a fusion classification threshold, wherein obtaining the fusion classification threshold includes at least one of the following: The average or median of all pixels within the template region of the target coding block is used as the fusion classification threshold; The average or median of all pixels in the template region of each candidate reference block within the fusion candidate is used as the fusion classification threshold; The average or median of all pixels in the template region of the specified candidate reference block in the fusion candidate is used as the fusion classification threshold; or All pixels in the template region of all candidate reference blocks in the fusion candidate are weighted and fused into a predicted template region according to the template cost weight of the corresponding candidate reference block. The average or median of all pixels in the predicted template region is used as the fusion classification threshold.

3. The method according to claim 1, further comprising: Local illumination compensation prediction is performed on the second template region of the candidate reference block within the fusion candidate.

4. The method according to claim 1, wherein, The template region includes at least one of the left pixel region, the top pixel region, the upper left pixel region, the lower left pixel region, and the upper right pixel region of the target coding block or the candidate reference block within the fusion candidate.

5. The method according to claim 1, wherein, The process of determining a candidate reference block list for the target coding block and constructing a fusion candidate based on the candidate reference block list includes at least one of the following: Obtain a list of candidate reference blocks for the target coding block, and divide each candidate reference block in the candidate reference block list into at least one candidate reference block group without repetition, and use each candidate reference block group as the fusion candidate; Obtain a list of candidate reference blocks for the target coding block; divide each candidate reference block in the candidate reference block list into at least one candidate reference block group; filter the candidate reference blocks in each candidate reference block group based on given conditions; and use the filtered candidate reference block groups as the fusion candidates. The given conditions include at least one of a pixel threshold and a template cost threshold. Obtain a list of candidate reference blocks for the target coding block, divide each candidate reference block in the list into at least one group of candidate reference blocks, and use each group of candidate reference blocks as the fusion candidate, wherein at least two groups of candidate reference blocks have at least one duplicate candidate reference block.

6. The method according to claim 1, wherein, The process involves classifying pixels at different positions within the template regions of the target coding block and the fusion candidate based on a fusion classification threshold, resulting in at least one pixel classification set, including: Determine the first template region of the target coding block and the second template region of the candidate reference block within the fusion candidate; The pixels at the same position in the first template region and the second template region are compared with the fusion classification threshold to classify the pixels at the same position into the pixel classification set.

7. The method according to claim 6, wherein, The step of comparing the pixels at the same position in the first template region and the second template region with the fusion classification threshold includes at least one of the following: The pixels at each position within the first template region are compared with the fusion classification threshold. The average value of the pixels at each position within the second template region of all candidate reference blocks in the fusion candidate is compared with the fusion classification threshold; The pixels at each position within the second template region of the specified candidate reference block in the fusion candidate are compared with the fusion classification threshold. or The pixels at each position of the template region of all candidate reference blocks in the fusion candidate are weighted and fused into a predicted template region according to the template cost weight of the corresponding candidate reference block. The pixels at each position of the predicted template region are compared with the fusion classification threshold.

8. The method according to claim 2 or 7, wherein, The designated candidate reference block includes the reference block with the minimum template cost among the fusion candidates.

9. The method according to claim 2 or 7, further comprising: Obtain the template cost weight of each candidate reference block. The template cost weight is the difference between the sum of the template costs of each candidate reference block and the template cost of the candidate reference block, and the ratio between the product of the number of remaining candidate reference blocks (excluding the candidate reference block) and the sum of the template costs in the fusion candidate.

10. The method according to claim 1, wherein, The step of determining a fusion model for each pixel classification set based on at least two groups of pixels corresponding to different positions within the pixel classification set includes: For each pixel classification set, first and second pixels at the same position are extracted sequentially in the first template region of the target coding block and the second template region of the candidate reference block in the fusion candidate to form a pixel group. The model parameters of the fusion model are determined according to at least two groups of pixels.

11. The method according to claim 1, wherein, The step of obtaining corresponding target reference pixels in all candidate reference blocks within the fusion candidate based on the positions of the target coding block, determining a matching target fusion model within each fusion model based on each target reference pixel, and determining the predicted pixel value of each pixel in the target coding block based on the target fusion model and the target reference pixels includes: For each position within the target coding block, obtain the target reference pixel corresponding to that position within all candidate reference blocks in the fusion candidate; The target reference pixel at each of the aforementioned locations is compared with the fusion classification threshold, and the target classification set to which the target reference pixel at each of the aforementioned locations belongs is determined in each of the aforementioned pixel classification sets. The fusion model corresponding to the target classification set is then used as the target fusion model. Each of the target reference pixels is input into the corresponding target fusion model to obtain the predicted pixel value of the pixel point of the target coding block at the position.

12. The method according to claim 11, wherein, The step of comparing the target reference pixel at each of the aforementioned locations with the fusion classification threshold includes at least one of the following: The average value of the target reference pixels at each position within all candidate reference blocks in the fusion candidate is compared with the fusion classification threshold; The target reference pixel at each position of the specified candidate reference block in the fusion candidate is compared with the fusion classification threshold; or The target reference pixels at each position within all candidate reference blocks in the fusion candidate are weighted and fused according to their respective template cost weights, and then compared with the fusion classification threshold.

13. The method according to claim 12, wherein, The template cost includes at least one of the following: absolute difference and cost, squared difference and cost, or transform domain absolute difference and cost.

14. The method according to claim 1, further comprising: The target coded block is determined to satisfy a preset condition, which includes at least one of the following: The width of the target coded block is greater than or equal to the first threshold; The height of the target coding block is greater than or equal to the second threshold; or The area of ​​the target coded block is greater than or equal to the third threshold.

15. An image information processing method, applied at an encoding end, the method comprising: The instruction information for the image information processing method is transmitted to the decoding end, wherein the image information processing method includes any one of the image information processing methods described in claims 1-14.

16. The method of claim 15, further comprising: Determine a list of candidate reference blocks for the target coding block, and construct fusion candidates based on the list of candidate reference blocks; Based on the fusion classification threshold, the pixels at different positions within the template regions of the target coding block and the fusion candidate are classified to obtain at least one pixel classification set. For each pixel classification set, the fusion model is determined based on at least two groups of pixels corresponding to different positions within the pixel classification set; Based on the position of the target coding block, obtain the corresponding target reference pixel in all candidate reference blocks within the fusion candidate. Based on each target reference pixel, determine the matching target fusion model in each fusion model. Based on the target fusion model and the target reference pixel, determine the predicted pixel value of each pixel in the target coding block.

17. The method according to claim 15, wherein, The indication information includes at least one of the following: Intra-frame prediction indication information, wherein the intra-frame prediction indication information indicates whether intra-frame prediction mode is enabled or disabled in image information processing; Intra-frame template fusion prediction information, wherein the intra-frame template fusion prediction information indicates whether the image information processing enables or disables the template fusion-based intra-frame prediction mode; Intra-frame template fusion type, wherein the intra-frame template fusion type indicates the type of intra-frame template fusion prediction used by the coding unit in image information processing, the type including at least one of template cost weighted fusion, single-model Wiener filter weighted fusion, or multi-model Wiener filter weighted fusion; A fusion candidate index, wherein the fusion candidate index indicates fusion candidates participating in fusion during image information processing; or Wiener filter indication information, wherein the Wiener filter indication information indicates the type of Wiener filter used in image information processing.

18. The method according to claim 15, wherein, The indication level of the indication information includes at least one of the following: video level; sequence level; image level; slice level; or coding unit level.

19. An image information processing method, applied at a decoding end, the method comprising: Obtain instruction information of an image information processing method transmitted from the encoding end, wherein the image information processing method includes the image information processing method according to any one of claims 1-14.

20. The method according to claim 19, wherein, The indication information includes at least one of the following: Intra-prediction indication information (set to enabled), intra-template fusion prediction information (set to enabled), intra-template fusion type and fusion candidate index (set to weighted fusion using multi-model Wiener filter); or Intra-prediction indication information when set to enabled, intra-template fusion prediction information when set to enabled, intra-template fusion type when set to weighted fusion using Wiener filter, Wiener filter indication information when set to multi-model Wiener filter, and fusion candidate index.

21. An electronic device, the electronic device comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image information processing method as described in any one of claims 1-20.

22. A computer-readable storage medium storing one or more programs, said one or more programs being executed by one or more processors to implement the image information processing method as described in any one of claims 1-20.

23. A computer program product comprising a computer program that, when executed by a processor, implements the image information processing method according to any one of claims 1-20.

Images