An intra-frame template matching method, apparatus, device and medium

Abstract

The application discloses an intra-frame template matching method, device and equipment and a medium, and relates to the technical field of video coding. The method comprises the following steps: extracting the context features of a current block; inputting the extracted context features into a lightweight predictor, outputting candidate scores from the lightweight predictor, and using the candidate scores to indicate the priority of candidate modes or candidate categories; selecting a predetermined number of target candidates from a candidate set according to the candidate scores, wherein the target candidates are candidate modes or candidate categories that meet preset conditions; when the target candidates are candidate categories, expanding the candidate categories into corresponding candidate modes to obtain a candidate mode set; performing template matching processing only on the candidate modes in the candidate mode set, and determining the prediction mode of the current block according to the template matching result. The method can greatly reduce the calculation complexity of template matching while maintaining the prediction accuracy, and improves the video coding efficiency.

Description

Technical Field

[0001] This invention relates to the field of video coding technology, and in particular to an intra-frame template matching method, apparatus, device, and medium. Background Technology

[0002] In video coding, intra-frame template matching generates a prediction signal by searching for the most similar template to the current block within the reconstructed pixel region, effectively improving the coding efficiency of intra-frame prediction. However, traditional template matching schemes typically require performing a complete template matching calculation on each of a large number of candidate patterns to select the best one, a process with extremely high computational complexity. Some schemes attempt to use large neural networks to predict and filter candidate patterns, but large neural networks themselves contain a large number of parameters, and the overhead of parameter access and multiplication-accumulation operations during inference may approach or even exceed the complexity of a full template matching search, causing the search benefits brought by prediction and filtering to be offset by the overhead of the predictor itself. Summary of the Invention

[0003] The purpose of this invention is to provide an intra-frame template matching method, apparatus, device, and medium that can significantly reduce computational complexity while maintaining template matching prediction accuracy.

[0004] To address the aforementioned technical problems, this invention provides an intra-frame template matching method, comprising: Extract the contextual features of the current block; The extracted context features are input into a lightweight predictor, which outputs candidate scores; the candidate scores are used to indicate the priority of candidate patterns or candidate categories. Based on the candidate scores, a predetermined number of target candidates are selected from the candidate set; the target candidates are candidate patterns or candidate categories whose candidate scores satisfy preset conditions; wherein, when the target candidate is a candidate category, the candidate category is expanded into the corresponding candidate patterns to obtain a candidate pattern set; Template matching is performed only on candidate patterns in the candidate pattern set, and the prediction pattern of the current block is determined based on the template matching result.

[0005] To address the aforementioned technical problems, the present invention also provides an intra-frame template matching device, comprising: The feature extraction module is used to extract the contextual features of the current block; The score prediction module is used to input the extracted context features into a lightweight predictor, which outputs candidate scores; the candidate scores are used to indicate the priority of candidate patterns or candidate categories. The candidate selection module is used to select a predetermined number of target candidates from the candidate set according to the candidate score; the target candidate is a candidate pattern or candidate category whose candidate score satisfies a preset condition; wherein, when the target candidate is a candidate category, the candidate category is expanded into the corresponding candidate pattern to obtain a candidate pattern set; The pattern determination module is used to perform template matching processing only on the candidate patterns in the candidate pattern set, and determine the prediction pattern of the current block based on the template matching result.

[0006] To address the aforementioned technical problems, the present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the above-described intra-frame template matching method.

[0007] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described intra-frame template matching method.

[0008] The beneficial effects of this invention are as follows: The intra-frame template matching method provided by this invention extracts the context features of the current block and inputs them into a lightweight predictor. The lightweight predictor predicts the priority of candidate patterns or candidate categories and selects a predetermined number of target candidates based on the predicted candidate scores. Template matching is performed only within the selected candidate pattern set, avoiding the computational overhead of performing template matching on all candidate patterns one by one. At the same time, since the input of the lightweight predictor is context features and the output is candidate scores, its parameter count and inference computation are much smaller than those of large neural networks, and the overhead of the predictor itself is controlled at a low level. In addition, when outputting candidate scores, the lightweight predictor can output the priority of each candidate pattern or output the priority of each candidate category first and then expand it into the corresponding candidate pattern. It can flexibly adapt to application scenarios with different candidate scales, making candidate selection adaptive to the content characteristics of the current block, and significantly improving the coding efficiency of the intra-frame template matching process.

[0009] In addition, the present invention also provides a corresponding intra-frame template matching device, electronic device and computer-readable storage medium for the intra-frame template matching method, which have the same or corresponding technical features as the intra-frame template matching method mentioned above, and have the same effect. Attached Figure Description

[0010] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart of the intra-frame template matching method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the intra-frame template matching device provided in an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0013] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] Embodiments of this invention provide an intra-frame template matching method, applicable to video encoders or decoders, particularly suitable for application scenarios in video coding or decoding standards that require intra-frame template matching to improve intra-frame prediction accuracy. This method can be implemented independently at either the encoding or decoding end: at the encoding end, it quickly filters out a small number of high-probability candidate modes from multiple candidate modes and performs template matching only on these selected candidate modes to determine the prediction mode of the current block; at the decoding end, since the decoding end can obtain the same reconstructed neighborhood information, block size, quantization parameters, and component types as the encoding end, it can perform the exact same candidate filtering and template matching process, thereby independently reproducing the prediction mode of the current block without relying on the encoding end to transmit additional syntactic information.

[0016] The following section describes the method in detail, taking into account the execution flow of the intra-frame template matching method. Figure 1 A flowchart of the intra-frame template matching method provided in the embodiments of the present invention is shown below. Figure 1 As shown, the method includes: S101. Extract the context features of the current block.

[0017] It should be noted that the current block refers to the image block currently being encoded or decoded in a video frame, and its size can be arbitrary, such as 4×4, 8×8, 16×16, 32×32, or 64×64. Contextual features refer to the available information associated with the current block, reflecting its coding characteristics. Contextual features may include the spatial location of the current block within the video frame, information about its reconstructed neighborhood, its size, the quantization parameters used during encoding, and the component type to which the current block belongs. These contextual features are available at both the encoding and decoding ends and do not depend on the original pixel information.

[0018] When performing step S101, the present invention can extract information such as reconstructed pixels and encoded patterns from the spatial neighborhood of the current block. This information does not depend on samples that have not yet been reconstructed and can be obtained at both the encoding and decoding ends.

[0019] S102. Input the extracted contextual features into the lightweight predictor, which outputs candidate scores; the candidate scores are used to indicate the priority of candidate patterns or candidate categories.

[0020] It should be noted that a lightweight predictor refers to a prediction model with fewer than a first preset threshold of parameters and fewer than a second preset threshold of multiply-accumulate operations required for inference on a single current block. The specific values ​​of the first and second preset thresholds can be pre-set by the encoder according to its configuration level, or adjusted by the user according to the actual application scenario, to ensure that the predictor's inference overhead is much lower than the overhead of a full template matching search. In one specific embodiment, the first preset threshold is 8000 parameters, and the second preset threshold is 1024 multiply-accumulate operations. In another specific embodiment, the second preset threshold is set to 5% of the computational cost of a complete template matching operation. The model parameters of the lightweight predictor can be fixed in the video coding standard or selected by the sequence-level configuration level. The specific form of the lightweight predictor can include a rule lookup table, a linear model, a multilayer perceptron model, or a convolutional neural network model.

[0021] Candidate scores refer to the numerical values ​​output by the lightweight predictor for each candidate mode or candidate category. These values ​​indicate the priority of the corresponding candidate mode or category. A candidate mode refers to the various combinations of prediction parameters available during intra-frame template matching. Candidate modes can include intra-frame orientation prediction modes, template matching displacement vectors, reference frame indices, or reference line indices. A candidate category is a set of categories formed by grouping multiple candidate modes according to predetermined rules; for example, grouping multiple intra-frame orientation prediction modes with similar directional characteristics into the same category.

[0022] In step S102, the present invention inputs the contextual features extracted in step S101 into a lightweight predictor, which estimates the priority of each candidate pattern or candidate category. Since the number of parameters and the amount of inference computation of the lightweight predictor are both kept at a low level, the computational cost of this step itself is much less than that of traditional template matching full search, and it can provide a reliable priority basis for screening with a small additional computational burden.

[0023] S103. Based on the candidate scores, select a predetermined number of target candidates from the candidate set; the target candidates are candidate patterns or candidate categories whose candidate scores meet preset conditions; wherein, when the target candidate is a candidate category, the candidate category is expanded into the corresponding candidate pattern to obtain a candidate pattern set.

[0024] It should be noted that the preset conditions may include candidate scores to indicate the priority of candidate patterns or candidate categories, with higher-priority candidate patterns or categories being selected more frequently. The value of the predetermined quantity K can be adjusted according to the size of the current block or the level of coding complexity. For example, a larger K value can be used for larger blocks to ensure prediction quality, while a smaller K value can be used for smaller blocks to further reduce computational complexity. In one specific embodiment, for a 16×16 luminance block, the predetermined quantity K can be 4; in another specific embodiment, for an 8×8 luminance block, the predetermined quantity K can be 2.

[0025] In step S103, based on the candidate scores output by the lightweight predictor in step S102, the present invention can filter out a predetermined number of candidate patterns or candidate categories with the highest priority from the candidate set. If the selected category is a candidate category, then all candidate patterns contained in each candidate category are expanded according to a preset category classification table, and deduplication is performed on all expanded candidate patterns to finally obtain a candidate pattern set. This reduces the number of candidate patterns from dozens or even hundreds to a predetermined number, thus significantly reducing the computational load of template matching.

[0026] S104. Perform template matching only on the candidate patterns in the candidate pattern set, and determine the prediction pattern of the current block based on the template matching results.

[0027] It should be noted that template matching refers to the process of generating template prediction signals corresponding to candidate patterns and calculating the matching cost. The prediction pattern of the current block refers to the candidate pattern selected from the candidate pattern set after template matching is completed, which is used to generate the final prediction signal of the current block.

[0028] In step S104, this invention only performs template matching on candidate patterns in the candidate pattern set obtained in step S103, and does not perform template matching on unselected candidate patterns. Since the candidate pattern set contains only a small number of candidate patterns (usually single digits), the number of template matching operations is significantly reduced. Furthermore, since the candidate selection in step S103 is based on the context-dependent priority output of the lightweight predictor, the selected candidate patterns have a statistically high probability of containing the optimal candidate. Therefore, this invention can significantly reduce computational complexity while maintaining prediction accuracy.

[0029] In the intra-frame template matching method provided in this embodiment of the invention, the context features of the current block are extracted and input into a lightweight predictor. The lightweight predictor is used to predict the priority of candidate patterns or candidate categories, and a predetermined number of target candidates are selected based on the predicted candidate scores. Template matching is performed only within the selected candidate pattern set, avoiding the computational overhead of performing template matching on all candidate patterns one by one. At the same time, since the input of the lightweight predictor is context features and the output is candidate scores, its parameter count and inference computation are much smaller than those of large neural networks, and the overhead of the predictor itself is controlled at a low level. In addition, when outputting candidate scores, the lightweight predictor can output the priority of each candidate pattern or output the priority of each candidate category first and then expand it into the corresponding candidate pattern. It can flexibly adapt to application scenarios with different candidate scales, making candidate selection adaptive to the content characteristics of the current block, and significantly improving the coding efficiency of the intra-frame template matching process.

[0030] Furthermore, in a specific implementation, in the intra-frame template matching method provided in the embodiments of the present invention, step S101 extracting the context features of the current block may specifically include: extracting neighboring block pattern information from the reconstructed neighborhood of the current block, performing histogram statistics on the neighboring block pattern information to obtain neighboring block pattern histogram features; extracting edge direction information from the reconstructed neighborhood of the current block, performing histogram statistics on the edge direction information to obtain edge direction histogram features; extracting texture variance features or gradient intensity features of the current block; extracting the block size, quantization parameters, and component types of the current block; combining the extracted neighboring block pattern histogram features, edge direction histogram features, texture variance features or gradient intensity features, as well as the block size, quantization parameters, and component types into a feature vector, and determining the feature vector as the context features.

[0031] In implementation, the extraction process of the above-mentioned features relies solely on the reconstructed neighborhood information already obtained at both the encoder and decoder ends, without depending on the original pixel information or samples that have not yet been reconstructed. Specifically, the neighbor block pattern histogram feature reflects the distribution of prediction patterns adopted by neighboring blocks in the current block space; the edge direction histogram feature reflects the distribution of texture direction in the area surrounding the current block; the texture variance feature or gradient intensity feature reflects the texture complexity of the area surrounding the current block; and the block size, quantization parameters, and component type reflect the encoding configuration environment in which the current block is encoded. After combining the above-mentioned features into a feature vector, this feature vector becomes the context feature in step S101, serving as the input to the lightweight predictor. Through the above multi-dimensional context feature extraction, the lightweight predictor can obtain richer and more accurate prior information, thereby improving the accuracy of priority prediction.

[0032] In the above steps, before combining the extracted features into a feature vector, the following steps may be included: quantizing the continuous numerical features among the extracted features and mapping the continuous numerical features to a finite number of quantization levels; filling the missing values ​​in the extracted features according to a predetermined filling rule; and performing fixed-point normalization on the extracted features.

[0033] The quantization process described above refers to mapping continuous numerical features to a finite number of discrete levels according to pre-defined quantization boundary values. Specifically, several boundary values ​​are pre-determined as quantization boundary values ​​based on the distribution range of the continuous numerical features, and samples whose feature values ​​fall within different intervals are mapped to the corresponding discrete levels. For example, texture variance can be divided into 8 levels (0 to 7) or 16 levels (0 to 15) according to the pre-defined quantization boundary values. For values ​​exceeding the pre-defined range, truncation can be performed according to pre-defined maximum and minimum values: when a feature value exceeds the pre-defined maximum value, the feature value is truncated to the quantization level corresponding to the maximum value; when a feature value is lower than the pre-defined minimum value, the feature value is truncated to the quantization level corresponding to the minimum value. The rounding rules in the quantization process can be executed according to pre-defined rounding methods (e.g., rounding to the nearest whole number, rounding up, or rounding down).

[0034] The aforementioned predefined filling rule refers to filling missing values ​​with a preset unavailable flag for unavailable neighborhood information at the current block boundary (for example, when the current block is located at the boundary of a video frame, there may be no reconstructed neighborhood above or to the left of it) according to a predefined filling method. This unavailable flag is different from all normal values ​​to ensure that the predictor can distinguish between the two states of available and unavailable information.

[0035] The aforementioned fixed-point normalization process maps the numerical range of each feature dimension to a unified fixed-point number representation range to facilitate integer inference operations. The feature vectors after fixed-point normalization are implemented using integer multiplication-addition and shift operations during the inference process of the lightweight predictor.

[0036] By quantizing, padding, and fixed-point normalizing the features, the input features of the lightweight predictor can be converted into a uniform representation suitable for integer operations. This helps to improve inference speed, reduce hardware implementation costs, and ensure that the encoder and decoder obtain the exact same feature vector for the same current block.

[0037] Furthermore, in a specific implementation, in the intra-frame template matching method provided in the embodiments of the present invention, step S102 inputs the extracted context features into a lightweight predictor, and the lightweight predictor outputs candidate scores. Specifically, this may include: inputting context features into a lightweight predictor; the lightweight predictor may be a rule lookup table, a linear model, a multi-layer perceptron (MLP) model, or a convolutional neural network (CNN) model; the lightweight predictor outputs the conditional probability p(m|x) of each candidate pattern as a candidate score, or outputs the conditional probability p(c|x) of each candidate category as a candidate score.

[0038] In implementation, lightweight predictors can adopt various implementation forms to adapt to different hardware platforms and complexity constraints. In the implementation using a rule lookup table, the lightweight predictor directly looks up a pre-stored priority list based on the index value formed by combining input context features (e.g., main edge direction category and neighboring block main pattern category). In the implementation using a linear model, the lightweight predictor performs a linear weighted summation of the input context features and the weight parameters of each candidate pattern or candidate category to obtain the score of each candidate pattern or candidate category. Specifically, for a candidate pattern m, its score S(m) = b_m + sum_i w_i f_i, where b_m represents the bias term of candidate pattern m, w_i represents the weight coefficient of the i-th context feature, f_i represents the feature value of the i-th context feature, and sum_i represents the summation operation over all context features. In the implementation using a multilayer perceptron model, the lightweight predictor comprises an input layer, at least one hidden layer, and an output layer, all connected via fully connected layers. The input layer receives contextual feature vectors, and the output layer outputs scores for each candidate pattern or category. In the implementation using a convolutional neural network model, the lightweight predictor comprises at least one convolutional layer and at least one fully connected layer, enabling the extraction of higher-order feature representations from contextual features.

[0039] The lightweight predictor models the priority of each candidate pattern m under a given input feature x, outputting a conditional probability p(m|x) = f_θ(x), where θ represents the model parameters of the lightweight predictor, and f_θ(x) represents the output value of the predictor function defined by the parameters θ under the input x. When the lightweight predictor outputs the conditional probability p(m|x) of each candidate pattern, a higher conditional probability value indicates that the candidate pattern is more likely to become the optimal predictor. When the lightweight predictor outputs the conditional probability p(c|x) of each candidate category, a higher conditional probability value indicates that the candidate category is more likely to contain the optimal predictor. The above two output methods correspond to the two candidate generation paths of direct pattern prediction and category prediction followed by expansion, respectively. In practical applications, the choice can be flexibly made according to the size of the candidate set: when the number of candidate patterns is small, the direct prediction method can meet the requirements; when the number of candidate patterns is large (e.g., containing a large number of displacement vectors), the category prediction followed by expansion method can further reduce the output dimensionality of the predictor.

[0040] The training process of the aforementioned lightweight predictor may include: acquiring training blocks in the training set, performing a full search on the training blocks to obtain the rate-distortion cost corresponding to each candidate mode; constructing soft labels based on the rate-distortion cost corresponding to each candidate mode, with candidate modes having smaller rate-distortion costs being assigned greater weights in the soft labels; training the lightweight predictor with the contextual features of the training blocks as input and the soft labels as supervision signals until the difference between the probability distribution output by the lightweight predictor and the soft labels satisfies the convergence condition.

[0041] In implementation, the training process is completed offline. The training set can contain a large number of training blocks from different types of video sequences, covering rich texture features and coded scenes. For each training block in the training set, a full search is first performed to obtain the rate-distortion cost J(m) of the training block across all candidate modes, where m represents the index of the candidate mode, and the smaller the value of J(m), the better the rate-distortion performance of the candidate mode. Then, a soft label y_m is constructed based on the rate-distortion cost J(m) obtained from the full search, and the candidate mode with the smaller rate-distortion cost is assigned a larger weight in the soft label. Unlike traditional methods that only mark the single mode with the best rate-distortion cost as correct, this invention uses soft labels to simultaneously assign non-zero label weights to multiple candidate modes with similar rate-distortion values, thus allowing candidate modes with performance close to the optimal mode to also have a chance to enter the Top-K list. Finally, using the contextual features x of the training block as input and the soft label y_m as the supervision signal, the parameters θ of the lightweight predictor are trained using the cross-entropy loss function, so that the difference between the probability distribution p_θ(m|x) output by the lightweight predictor and the soft label y_m gradually decreases until the convergence condition is met.

[0042] Specifically, the cross-entropy loss function used during training can be L_soft = -sum_m·y_m·log(p_θ(m|x)), where L_soft represents the soft-label loss function value, y_m represents the soft-label weights of candidate patterns m, p_θ(m|x) represents the conditional probability of outputting candidate patterns m given input features x under the lightweight predictor parameters θ, log represents the natural logarithm function, and sum_m represents the summation operation over all candidate patterns m. The training objective is to minimize the loss function L_soft, so that the probability distribution p_θ(m|x) output by the predictor gradually approaches the soft label y_m. The convergence condition can be set to the loss function value being less than a set threshold (e.g., 0.01), or the training epochs reaching a preset maximum number of epochs (e.g., 100). In practical applications, the set threshold and preset maximum number of epochs can be adjusted according to the model size and the amount of training data. After training, the parameters of the lightweight predictor are fixed as an integer weight table, and only forward inference is performed during online encoding and decoding; the model parameters are no longer updated.

[0043] In the above steps, constructing soft labels based on the rate-distortion cost corresponding to each candidate mode can specifically include: calculating the weight value of each candidate mode based on the rate-distortion cost of each candidate mode; the weight value of a single candidate mode is the exponential function value exp(-J(m) / T) with the natural constant e as the base and the ratio of the negative of the rate-distortion cost J(m) of the single candidate mode to the temperature parameter T as the exponent; normalizing the exponential function value of each candidate mode and using the normalized result as the weight of the corresponding candidate mode in the soft label.

[0044] Specifically, the soft label y_m can be constructed as: y_m = exp(-J(m) / T) / sum_m'·exp(-J(m') / T), where m' represents the index variable for summing all candidate patterns in the candidate set, and T represents the temperature parameter, used to control the smoothness of the soft label. When the temperature parameter T is large, the difference in soft label weights among the candidate patterns becomes smaller, and the soft labels tend to be evenly distributed, which helps the lightweight predictor learn more relative ranking information. When the temperature parameter T is small, the candidate pattern with the best rate-distortion cost receives a significantly higher weight, and the soft label tends to be a hard label. The value of the temperature parameter T can be determined based on the average interval of rate-distortion cost between the best and second-best candidate patterns in the training set. In one specific embodiment, the temperature parameter T is set to the average value of the rate-distortion cost difference between the best and second-best candidate patterns in each training block on the training set; in another specific embodiment, the temperature parameter T is set to a fixed value predetermined based on coding experience. In a specific example, if the rate-distortion costs of the optimal mode 24 and the second-best mode 25 are similar, then both are assigned significant weights in the soft label. Using this soft label construction method, when the rate-distortion costs of the optimal candidate mode and the second-best candidate mode are similar, both are assigned significant weights in the soft label, thereby effectively reducing the risk of missing the optimal mode in the Top-K algorithm.

[0045] In the encoding syntax, this invention can indicate the lightweight predictor configuration used by the current sequence or the current frame through a lightweight predictor configuration identifier. The above syntax elements are merely illustrative and can be merged, split, or implied from the context in actual encoding standards according to standard specifications. All candidate order, quantization, rounding, boundary padding, and tie-breaking rules that need to be reproduced by the decoder are predefined or consistently indicated through higher-level parameters.

[0046] When the computational cost required for the lightweight predictor to infer the current block exceeds the preset budget, the system switches to a rule lookup table or linear model for candidate score output. The preset budget can be the same as the aforementioned preset threshold or can be set separately. For example, the preset budget can be set to no more than 512 multiply-accumulate operations per block. When the contextual features or size of the current block cause the computational cost of the lightweight predictor (e.g., a multilayer perceptron model or a convolutional neural network model) to exceed the preset budget, the system automatically switches to a lower-complexity predictor form such as a rule lookup table or linear model to ensure that the inference overhead of this invention remains controllable even in the worst-case scenario. This switching method ensures the applicability and robustness of this invention across different hardware platforms and application scenarios.

[0047] Furthermore, in a specific implementation, in the intra-frame template matching method provided in the embodiments of the present invention, step S103 selects a predetermined number of target candidates from the candidate set according to the candidate score. Specifically, it may include: when the candidate score is used to indicate the priority of the candidate mode, selecting a predetermined number of candidate modes with higher priority as target candidates according to the priority order indicated by the candidate score; when the candidate score is used to indicate the priority of the candidate category, selecting a predetermined number of candidate categories with higher priority as target candidates according to the priority order indicated by the candidate score, and merging the candidate modes contained in each of the selected candidate categories according to a predetermined category division table, and performing deduplication processing on the merged candidate modes to obtain a candidate mode set.

[0048] In implementation, the priority order indicated by the candidate scores specifically means that the numerical value of the candidate score directly corresponds to the level of priority. When the candidate score is a conditional probability value, the candidate pattern or category with a larger conditional probability value has a higher priority; when the candidate score is a weighted score, the candidate pattern or category with a larger score value has a higher priority. Selecting a predetermined number of candidate patterns or categories with the highest priority is called Top-K selection.

[0049] When the lightweight predictor outputs the conditional probability p(m|x) of each candidate mode, this invention selects the top K candidate modes from high to low according to the conditional probability to obtain the candidate mode set M_prime, that is, M_prime=TopK_m·p(m|x), where TopK_m represents the operation of selecting the top K candidate modes after sorting them from high to low according to the conditional probability p(m|x).

[0050] In one specific embodiment, for a 16×16 luma block, with Top-K=4, and assuming the candidate pattern list output by the lightweight predictor is {24,25,23,18}, then the candidate pattern set is {24,25,23,18}. Only these four candidate patterns are selected for template matching, and finally, pattern 24 is selected as the prediction pattern for the current block. In the traditional full-search scheme, 67 complete matches are required, and the complete pattern index needs to be transmitted; in this embodiment, only 4 complete matches are required, and it is possible to choose not to transmit the index or only transmit log2(K) bits of the list position.

[0051] When the target candidate is a candidate category, the candidate category needs to be expanded into candidate patterns. The category partitioning table is a predefined mapping table used to map candidate patterns to their respective categories. For example, in intra-frame direction prediction, multiple direction patterns with similar directional angles can be grouped into the same category; for instance, the vertical direction category contains multiple direction patterns with angles close to vertical, and the horizontal direction category contains multiple direction patterns with angles close to horizontal. In a specific embodiment, the lightweight predictor outputs 8 direction category probabilities, selects the top-2 categories, such as vertical category A and vertical category B, expands them into approximately 16 fine patterns according to the category partitioning table, and then performs template matching. After merging all candidate patterns contained in each selected candidate category according to the category partitioning table, deduplication is required to ensure that each candidate pattern in the candidate pattern set appears only once, avoiding repeated template matching for the same pattern in subsequent iterations.

[0052] In the above steps, the process of selecting target candidates may specifically include: when multiple candidate patterns or candidate categories have the same candidate score, making it impossible to uniquely determine a predetermined number of target candidates, the selection order of multiple candidate patterns or candidate categories shall be determined according to a predetermined fixed numbering rule.

[0053] In implementation, when candidate scores are the same, a fixed numbering rule is used as the deterministic rule to break ties. The fixed numbering rule refers to a rule that sorts candidate patterns or categories according to their numerical values, such as ascending order of pattern numbers or descending order of pattern numbers. In a specific embodiment, the fixed numbering rule stipulates that when candidate scores are the same, they are selected in ascending order of candidate pattern numbers. For example, the pattern list {24,25,23,18} is sorted and arranged in ascending order of numbers as {18,23,24,25}. The fixed numbering rule is pre-agreed upon and maintained consistently at both the encoding and decoding ends, thereby ensuring that under the same input conditions, the target candidate lists selected by the encoding and decoding ends are exactly the same, avoiding encoding / decoding mismatch.

[0054] After obtaining the candidate pattern set, you can also impose a length limit on the candidate pattern set to control the number of candidate patterns in the candidate pattern set within a preset maximum number (such as 8), so as to control the number of subsequent template matching times.

[0055] Furthermore, in a specific implementation, in the intra-frame template matching method provided in the embodiments of the present invention, step S104 only performs template matching processing on the candidate modes in the candidate mode set, and determines the prediction mode of the current block based on the template matching result. Specifically, it may include: generating template prediction signals for each candidate mode in the candidate mode set, and calculating the template matching cost corresponding to each candidate mode; and selecting the candidate mode with the best cost from the candidate mode set as the prediction mode of the current block based on the template matching cost.

[0056] In implementation, for each candidate pattern in the candidate pattern set, reference samples are extracted from the reconstructed neighborhood of the current block based on the prediction parameters specified by the candidate pattern (such as prediction direction angle, displacement vector, etc.). A template prediction signal for that candidate pattern is generated based on the extracted reference samples. The template matching cost is used to measure the degree of difference between the template prediction signal and the actual neighborhood of the current block.

[0057] Template matching costs can be sums of absolute differences, sums of squared differences, or sums of absolute differences in the transform domain. Those skilled in the art can choose according to actual needs. After calculating the template matching costs corresponding to each candidate pattern, the candidate pattern with the minimum template matching cost is selected as the prediction pattern for the current block.

[0058] Furthermore, in a specific implementation, the intra-frame template matching method provided in the embodiments of the present invention may further include: calculating the completeness rate distortion cost for each candidate mode in the candidate mode set, and selecting the candidate mode with the optimal cost from the candidate mode set as the prediction mode for the current block based on the completeness rate distortion cost.

[0059] In implementation, the completeness rate distortion cost considers not only the distortion between the predicted signal and the original signal, but also the bit rate overhead required to encode the predicted pattern, i.e., J_complete = D + λ·R, where D represents distortion, R represents bit rate, and λ represents Lagrange multipliers. Compared with template matching cost, the calculation of completeness rate distortion cost is more comprehensive, but the computational complexity is also higher. In this invention, since the candidate pattern set contains only a small number of candidate patterns, even if the completeness rate distortion cost is calculated for each candidate pattern, the total computational cost is still much smaller than that in the full search case. Therefore, using completeness rate distortion cost as the selection criterion is one feasible implementation method of this invention.

[0060] Furthermore, in a specific implementation, the intra-frame template matching method provided in the embodiments of the present invention may further include, after determining the prediction mode of the current block, encoding and writing the index position of the prediction mode in the candidate mode set into the bitstream.

[0061] In implementation, when this method is applied to the encoder, the encoder writes the index position of the finally selected prediction mode in the candidate mode set into the bitstream. This allows the decoder to directly determine the prediction mode of the current block from the same candidate mode set based on this index position, without repeating the template matching process. The number of bits required to write the index position into the bitstream is approximately log2(K) bits, where K is a predetermined number, much smaller than the number of bits required to write the complete mode index. Furthermore, when this method is applied to the decoder, the decoder can use the exact same context features and lightweight predictor as the encoder to generate the same candidate mode set and directly determine the final prediction mode according to the same template matching rules, without relying on the encoder to transmit the index position, thus achieving zero-overhead prediction mode reproduction.

[0062] When the difference between the optimal cost obtained after template matching processing of candidate patterns in the candidate pattern set and the neighbor block statistics exceeds a preset difference value, the present invention can expand the predetermined number to reselect target candidates, or perform template matching processing on all candidate patterns.

[0063] In implementation, neighbor block statistics refer to the average or minimum template matching costs of the best neighboring blocks around the current block. The preset difference value can be a multiple or offset of the neighbor block statistics; for example, the preset difference value could be 1.5 times the average cost of neighboring blocks, or the preset difference value could be the minimum cost of neighboring blocks plus a fixed offset. The specific value of the preset difference value can be set based on coding experience or adaptively determined based on the coding statistics of the current frame. When the optimal cost of the current block is much higher than the neighbor block statistics, it indicates that the candidate pattern set selected by the lightweight predictor may not contain the truly optimal candidate. In this case, a fallback mechanism is triggered: one fallback method is to expand the value of a predetermined number K, reselecting more target candidates to cover a wider range of candidates; another fallback method is to directly perform template matching on all candidate patterns, i.e., fall back to the traditional full search scheme. This ensures that even when the predictor's prediction is inaccurate, the prediction quality can still be guaranteed by increasing computational load, thus achieving an adaptive balance between prediction quality and computational complexity. The triggering conditions for the rollback mechanism are agreed upon in advance on the encoding and decoding ends, so that both ends can independently deduce whether rollback needs to be executed under the same conditions.

[0064] Furthermore, when neighborhood information for the current block is unavailable, a predefined default candidate list is used as the candidate pattern set. This predefined default candidate list may include, but is not limited to, the top few highest-priority patterns from the most likely pattern list. This default candidate list differs from the candidate list output by the lightweight predictor; it is used only as a fallback in the extreme case where neighborhood information is completely unavailable, ensuring that the method can still execute correctly in boundary cases without relying on the inference results of the lightweight predictor.

[0065] From the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0066] Embodiments of the present invention also provide an intra-frame template matching device. Figure 2 This is a schematic diagram of the intra-frame template matching device provided in an embodiment of the present invention. This embodiment is based on the perspective of functional modules, such as… Figure 2 As shown, the device includes: Feature extraction module 10 is used to extract contextual features of the current block; The score prediction module 11 is used to input the extracted context features into the lightweight predictor, which outputs candidate scores; the candidate scores are used to indicate the priority of candidate patterns or candidate categories. The candidate selection module 12 is used to select a predetermined number of target candidates from the candidate set based on the candidate scores; the target candidates are candidate patterns or candidate categories whose candidate scores meet preset conditions; wherein, when the target candidate is a candidate category, the candidate category is expanded into the corresponding candidate patterns to obtain a candidate pattern set. The pattern determination module 13 is used to perform template matching processing only on candidate patterns in the candidate pattern set, and determine the prediction pattern of the current block based on the template matching result.

[0067] In the intra-frame template matching apparatus provided in this embodiment of the invention, the feature extraction module 10 extracts the context features of the current block, the score prediction module 11 uses a lightweight predictor to predict the priority of each candidate mode or candidate category, the candidate selection module 12 selects only a predetermined number of target candidates based on the predicted candidate scores, and the mode determination module 13 performs template matching processing only within the selected candidate mode set, avoiding the computational overhead of performing template matching on all candidate modes one by one. At the same time, the number of parameters and inference operations of the lightweight predictor are much smaller than those of large neural networks, and the overhead of the predictor itself is controlled at a low level, which will not offset the search gains. In addition, when outputting candidate scores, the lightweight predictor can output the priority of each candidate mode or output the priority of each candidate category first and then expand it into the corresponding candidate mode, which can flexibly adapt to application scenarios with different candidate scales and significantly improve the coding efficiency of the intra-frame template matching process.

[0068] Since the embodiments of the intra-frame template matching device and the intra-frame template matching method correspond to each other, the description of the features in the embodiment corresponding to the intra-frame template matching device can be found in the relevant description of the embodiment corresponding to the intra-frame template matching method, and will not be repeated here. Furthermore, it has the same beneficial effects as the intra-frame template matching method mentioned above.

[0069] Furthermore, in a specific implementation, in the intra-frame template matching device provided in the embodiments of the present invention, the feature extraction module 10 can be specifically used to extract neighboring block pattern information from the reconstructed neighborhood of the current block, perform histogram statistics on the neighboring block pattern information to obtain neighboring block pattern histogram features; extract edge direction information from the reconstructed neighborhood of the current block, perform histogram statistics on the edge direction information to obtain edge direction histogram features; extract texture variance features or gradient intensity features of the current block; extract the block size, quantization parameters and component types of the current block; combine the extracted features into a feature vector, and determine the feature vector as context features.

[0070] Furthermore, in a specific implementation, in the intra-frame template matching device provided in the embodiments of the present invention, the score prediction module 11 can be specifically used to input context features into a lightweight predictor; the lightweight predictor is a rule lookup table, a linear model, a multilayer perceptron model, or a convolutional neural network model; the lightweight predictor outputs the conditional probability of each candidate mode as a candidate score, or outputs the conditional probability of each candidate category as a candidate score.

[0071] Furthermore, in a specific implementation, in the intra-frame template matching device provided in the embodiments of the present invention, the candidate selection module 12 can be specifically used to select a predetermined number of candidate modes with higher priority as target candidates according to the priority order indicated by the candidate scores when the candidate scores are used to indicate the priority of the candidate modes; when the candidate scores are used to indicate the priority of the candidate categories, select a predetermined number of candidate categories with higher priority as target candidates according to the priority order indicated by the candidate scores, and merge the candidate modes contained in each of the selected candidate categories according to a predetermined category division table, and perform deduplication processing on the merged candidate modes to obtain a candidate mode set.

[0072] Furthermore, in a specific implementation, in the intra-frame template matching device provided in the embodiments of the present invention, the mode determination module 13 can be specifically used to generate template prediction signals for each candidate mode in the candidate mode set, and calculate the template matching cost corresponding to each candidate mode; based on the template matching cost, select the candidate mode with the best cost from the candidate mode set as the prediction mode of the current block.

[0073] Embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the intra-frame template matching method.

[0074] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the above-described intra-frame template matching method embodiments at runtime.

[0075] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0076] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the intra-frame template matching method.

[0077] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described intra-frame template matching method embodiments.

[0078] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-chip (SoC), a complex programmable logic device (CPLD), a microcontroller unit (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0079] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0080] The present invention has provided a detailed description of an intra-frame template matching method, apparatus, device, and medium. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of these embodiments are only intended to aid in understanding the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. An intra-frame template matching method, characterized in that, include: Extract the contextual features of the current block; The extracted context features are input into a lightweight predictor, which then outputs candidate scores. The candidate score is used to indicate the priority of the candidate pattern or candidate category; Based on the candidate scores, a predetermined number of target candidates are selected from the candidate set; The target candidate is a candidate pattern or candidate category whose candidate score satisfies a preset condition; wherein, when the target candidate is a candidate category, the candidate category is expanded into the corresponding candidate pattern to obtain a candidate pattern set; Template matching is performed only on candidate patterns in the candidate pattern set, and the prediction pattern of the current block is determined based on the template matching result.

2. The intra-frame template matching method according to claim 1, characterized in that, The extracted contextual features are input into a lightweight predictor, which outputs candidate scores, including: The contextual features are input into the lightweight predictor; the lightweight predictor is a rule lookup table, a linear model, a multilayer perceptron model, or a convolutional neural network model. The lightweight predictor outputs the conditional probability of each candidate pattern as a candidate score, or outputs the conditional probability of each candidate category as a candidate score.

3. The intra-frame template matching method according to claim 1, characterized in that, Based on the candidate scores, a predetermined number of target candidates are selected from the candidate set, including: When the candidate score is used to indicate the priority of the candidate pattern, a predetermined number of candidate patterns with higher priority are selected as the target candidate according to the priority order indicated by the candidate score. When the candidate score is used to indicate the priority of the candidate category, a predetermined number of candidate categories with higher priority are selected as the target candidates according to the priority order indicated by the candidate score. The candidate patterns contained in each of the selected candidate categories are merged according to the predetermined category division table. The merged candidate patterns are deduplicated to obtain a candidate pattern set.

4. The intra-frame template matching method according to claim 1, characterized in that, Extract the contextual features of the current block, including: Extract neighbor block pattern information from the reconstructed neighborhood of the current block, and perform histogram statistics on the neighbor block pattern information to obtain neighbor block pattern histogram features; Edge direction information is extracted from the reconstructed neighborhood of the current block, and histogram statistics are performed on the edge direction information to obtain edge direction histogram features; Extract the texture variance feature or gradient intensity feature of the current block; Extract the block size, quantization parameters, and component type of the current block; The extracted features are combined into a feature vector, and the feature vector is determined as the context feature.

5. The intra-frame template matching method according to claim 4, characterized in that, Before combining the extracted features into a feature vector, the process also includes: The continuous numerical features among the extracted features are quantized and mapped to a finite number of quantization levels. Missing values ​​in the extracted features are filled according to a predetermined filling rule; The extracted features are subjected to fixed-point normalization.

6. The intra-frame template matching method according to claim 1, characterized in that, The process of selecting the target candidates includes: When multiple candidate patterns or candidate categories have the same candidate score, making it impossible to uniquely determine the predetermined number of target candidates, the selection order of the multiple candidate patterns or candidate categories is determined according to a predetermined fixed numbering rule.

7. The intra-frame template matching method according to claim 1, characterized in that, Template matching is performed only on candidate patterns in the candidate pattern set, and the prediction pattern of the current block is determined based on the template matching results, including: For each candidate mode in the candidate mode set, a template prediction signal is generated, and the template matching cost corresponding to each candidate mode is calculated. Based on the template matching cost, the candidate pattern with the optimal cost is selected from the candidate pattern set as the prediction pattern for the current block.

8. An intra-frame template matching device, characterized in that, include: The feature extraction module is used to extract the contextual features of the current block; The score prediction module is used to input the extracted context features into a lightweight predictor, which outputs candidate scores; the candidate scores are used to indicate the priority of candidate patterns or candidate categories. The candidate selection module is used to select a predetermined number of target candidates from the candidate set based on the candidate scores; The target candidate is a candidate pattern or candidate category whose candidate score satisfies a preset condition; wherein, when the target candidate is a candidate category, the candidate category is expanded into the corresponding candidate pattern to obtain a candidate pattern set; The pattern determination module is used to perform template matching processing only on the candidate patterns in the candidate pattern set, and determine the prediction pattern of the current block based on the template matching result.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the intra-frame template matching method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the intra-frame template matching method as described in any one of claims 1 to 7.

Images