Template matching search range setting method and related equipment

Abstract

The invention provides a template matching search range setting method and related equipment, and belongs to the technical field of video coding and decoding. The template matching search range setting method comprises the following steps: obtaining a target template feature of a template of a current block; obtaining a target region feature of a reference region of the current block; and determining a target search range of the template based on the target template features and the target region features.

Description

Technical Field

[0001] This disclosure relates to the field of video encoding and decoding technology, and more specifically, to a method for setting a template matching search range, an electronic device, and a computer-readable storage medium. Background Technology

[0002] On the one hand, the internet is about to enter the 5G (5th generation mobile networks or 5th generation wireless systems, 5th-Generation, fifth-generation mobile communication technology) era, and images (videos) appearing in various internet applications have become the main consumers of internet bandwidth. In particular, mobile internet image traffic is increasing daily, and will experience explosive growth in the 5G era, injecting a powerful new impetus into the accelerated development of image encoding and decoding technologies. At the same time, this also presents many severe new challenges to image encoding and decoding technologies that have never been encountered before. In the 5G era, with the Internet of Everything, the new internet images generated by various emerging applications are diverse and varied. Therefore, how to research efficient image encoding and decoding technologies to address the characteristics of these diverse and varied new internet images has become an urgent need.

[0003] On the other hand, the amount of video data required to depict even relatively short films can be quite large, which can pose difficulties when the data is streamed or otherwise transmitted over communication networks with limited bandwidth. Therefore, video data is typically compressed before transmission over modern telecommunications networks. Before transmission, video compression devices typically encode the video data using software and / or hardware on the source side, thereby reducing the amount of data required to represent the digital video image. The compressed data is then received at the destination by a video decompression device, which decodes the video data. Given limited network resources and the increasing demand for higher video quality, there is a need for improved compression and decompression techniques that can improve image quality without increasing the bit rate.

[0004] Therefore, a new method for setting the search scope for template matching, an electronic device, and a computer-readable storage medium are needed.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure. Summary of the Invention

[0006] This disclosure provides a method for setting a template matching search range, including: obtaining target template features of a template of the current block; obtaining target region features of a reference region of the current block; and determining the target search range of the template based on the target template features and the target region features.

[0007] In some exemplary embodiments of this disclosure, obtaining the target template features of the template of the current block includes: using at least one of the mean, variance, and gradient of the set of pixel values ​​in the template as the target template features. Specifically, obtaining the sub-region features of each sub-region based on the set of pixel values ​​in each sub-region includes: using at least one of the mean, variance, and gradient of the set of pixel values ​​in the sub-region as the sub-region features of the corresponding sub-region.

[0008] In some exemplary embodiments of this disclosure, each sub-region feature is compared with the target template feature, and sub-regions whose sub-region features do not match the target template features are removed to determine the target search range of the template, including: taking the sub-regions whose absolute value difference between the target template feature and the sub-region feature satisfies a predetermined condition as the target search range.

[0009] This disclosure provides a method for setting a template matching search range, which involves performing at least one of the following to determine the target search range for the template:

[0010] Obtain the hash value of the pixel value set of each matching template in the reference area of ​​the current block, and remove the matching templates that do not match the hash value of the pixel value set of the template of the current block;

[0011] Determine the smallest unit, extract features from the pixel values ​​of each smallest unit within the reference region of the current block, construct the hash value of each smallest unit within the reference region, construct the hash value of each matching template within the reference region based on the hash value of each smallest unit within the reference region, and remove matching templates that do not match the hash value of the pixel value set of the template of the current block.

[0012] Feature extraction is performed on the reference region of the current block after downsampling, and the hash value of the pixel value set of each matching template in the downsampling reference region is constructed. Matching templates that do not match the hash value of the pixel value set of the template of the current block are removed.

[0013] The reference region of the current block is divided into multiple sub-regions. The feature value of each sub-region is obtained based on the set of pixel values ​​contained in each sub-region. The sub-region whose feature value matches the feature value of the template of the current block is selected.

[0014] This disclosure provides a video decoding method, comprising: acquiring a current frame in an encoded video stream, the current frame including a current block and its template; obtaining target template features of the template of the current block; obtaining target region features of a reference region of the current block; determining a target search range of the template based on the target template features and the target region features; searching for a target matching template that matches the template within the target search range; determining a reference block of the current block based on the target matching template; and decoding the current block according to the reference block.

[0015] This disclosure provides a video encoding method, comprising: acquiring a current frame in a video stream to be encoded, the current frame including a current block and its template; obtaining target template features of the template of the current block; obtaining target region features of a reference region of the current block; determining a target search range of the template based on the target template features and the target region features; searching for a target matching template that matches the template within the target search range; determining a reference block of the current block based on the target matching template; and encoding the current block according to the reference block.

[0016] This disclosure provides a template matching search range setting device, including: a processing unit, configured to obtain target template features of a template of a current block; the processing unit is further configured to obtain target region features of a reference region of the current block; the processing unit is further configured to determine the target search range of the template based on the target template features and the target region features.

[0017] In some exemplary embodiments of this disclosure, the template matching search range setting device is disposed in a video encoder and / or a video decoder.

[0018] This disclosure provides a template matching search range setting apparatus, including: a processing unit configured to perform at least one of the following to determine the target search range of a template:

[0019] Obtain the hash value of the pixel value set of each matching template in the reference area of ​​the current block, and remove the matching templates that do not match the hash value of the pixel value set of the template of the current block;

[0020] Determine the smallest unit, extract features from the pixel values ​​of each smallest unit within the reference region of the current block, construct the hash value of each smallest unit within the reference region, construct the hash value of each matching template within the reference region based on the hash value of each smallest unit within the reference region, and remove matching templates that do not match the hash value of the pixel value set of the template of the current block.

[0021] Feature extraction is performed on the reference region of the current block after downsampling, and the hash value of the pixel value set of each matching template in the downsampling reference region is constructed. Matching templates that do not match the hash value of the pixel value set of the template of the current block are removed.

[0022] The reference region of the current block is divided into multiple sub-regions. The feature value of each sub-region is obtained based on the set of pixel values ​​contained in each sub-region. The sub-region whose feature value matches the feature value of the template of the current block is selected.

[0023] This disclosure provides a video decoder, including: a receiving unit, configured to acquire a current frame in an encoded video stream, the current frame including a current block and its template; a processing unit, configured to obtain target template features of the template of the current block; the processing unit is further configured to obtain target region features of a reference region of the current block; the processing unit is further configured to determine a target search range of the template based on the target template features and the target region features; the processing unit is further configured to search for a target matching template that matches the template within the target search range; the processing unit is further configured to determine a reference block of the current block based on the target matching template; and the processing unit is further configured to decode the current block according to the reference block.

[0024] This disclosure provides a video encoder, comprising: a receiving unit, configured to acquire a current frame in a video stream to be encoded, the current frame including a current block and its template; a processing unit, configured to obtain target template features of the template of the current block; the processing unit is further configured to obtain target region features of a reference region of the current block; the processing unit is further configured to determine a target search range of the template based on the target template features and the target region features; the processing unit is further configured to search for a target matching template that matches the template within the target search range; the processing unit is further configured to determine a reference block of the current block based on the target matching template; and the processing unit is further configured to encode the current block according to the reference block.

[0025] This disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of this disclosure.

[0026] This disclosure provides an electronic device, including: at least one processor; and a storage device configured to store at least one program, which, when executed by the at least one processor, causes the at least one processor to perform the method as described in any embodiment of this disclosure.

[0027] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in any embodiment of this disclosure. Attached Figure Description

[0028] Figure 1 A block diagram of an example video codec system according to some embodiments of the present disclosure is illustrated schematically.

[0029] Figure 2 A basic block diagram of an example video encoder according to some embodiments of the present disclosure is illustrated schematically.

[0030] Figure 3 A basic block diagram of an example video decoder according to some embodiments of the present disclosure is illustrated schematically.

[0031] Figure 4 A flowchart illustrating a template matching search range setting method according to some embodiments of the present disclosure is shown.

[0032] Figure 5 The illustration shows a schematic diagram of an example cross-component chromaticity prediction according to some embodiments of the present disclosure.

[0033] Figure 6 The illustration shows a schematic diagram of example intra-frame template matching according to some embodiments of the present disclosure.

[0034] Figure 7 The illustration shows a schematic diagram of example intra-frame template matching according to other embodiments of the present disclosure.

[0035] Figure 8 The illustration shows a schematic diagram of example regional intra-frame template matching according to some embodiments of the present disclosure.

[0036] Figure 9 A flowchart illustrating a template matching search range setting method according to other embodiments of the present disclosure is shown schematically.

[0037] Figure 10 A flowchart illustrating a video decoding method according to an embodiment of the present disclosure is shown schematically.

[0038] Figure 11 A flowchart illustrating a video encoding method according to some embodiments of the present disclosure is shown schematically.

[0039] Figure 12 A block diagram of a template matching search range setting device according to some embodiments of the present disclosure is shown schematically.

[0040] Figure 13 A block diagram of a template matching search range setting device according to other embodiments of the present disclosure is shown schematically.

[0041] Figure 14 A block diagram of a video decoder according to some embodiments of the present disclosure is shown schematically.

[0042] Figure 15 A block diagram of a video encoder according to some embodiments of the present disclosure is shown schematically.

[0043] Figure 16 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0044] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0045] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0046] In the following description, unless otherwise defined, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0047] The terms "an embodiment," "embodiment," "example embodiment," etc., used in this disclosure refer to embodiments that may include specific features, structures, or characteristics, but not every embodiment is required to include that specific feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in conjunction with an example embodiment, whether explicitly described or not, it is believed that such a feature, structure, or characteristic affecting its relation to other embodiments is within the knowledge of those skilled in the art.

[0048] It should be understood that although the terms “first” and “second”, etc., can be used to describe various elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0049] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0050] First, some of the terms used in the embodiments of this disclosure will be explained.

[0051] Intra(picture)Prediction: Intra-frame prediction.

[0052] Inter(picture)Prediction: Inter-frame prediction.

[0053] SCC stands for Screen Content Coding. Screen images are images generated by electronic devices such as computers, mobile phones, and televisions. They mainly consist of two types of content: one is computer-generated non-continuous tone content, containing many small, sharp lines and shapes, such as text, icons, buttons, and grids; the other is content captured by cameras, containing many continuous tones, such as movies, television clips, and natural video images. With the rapid development of cloud computing, mobile communication technology, and wireless display technology, how to achieve high-quality display of screen images on various electronic terminal devices at low bitrates is a problem that SCC needs to solve.

[0054] Loop Filtering: Loop filtering.

[0055] QP: Quantization Parameter.

[0056] LCU: Largest Coding Unit.

[0057] CTU: Coding Tree Unit, which is generally divided from the largest coding unit downwards.

[0058] CU: Coding Unit.

[0059] PU: Prediction Unit.

[0060] MV: Motion Vector.

[0061] BV: Block Vector.

[0062] MVP: Motion Vector Prediction.

[0063] MVD: Motion Vector Difference, the difference between the MVP and the true estimate of MV.

[0064] ME: Motion Estimation. The process of obtaining the motion vector MV is called motion estimation, which is a technique in motion compensation (MC).

[0065] Motion compensation (MC) is the process of estimating the current image based on motion vectors and inter-frame prediction methods. Motion compensation is a method for describing the differences between adjacent frames (adjacent here means adjacent in coding relation, not necessarily adjacent in playback order). Specifically, it describes how each small block in the previous frame moves to a certain position in the current frame. This method is often used by video compression / video codecs to reduce spatial redundancy in video sequences. Adjacent frames are usually very similar, meaning they contain a lot of redundancy. The purpose of using motion compensation is to improve the compression ratio by eliminating this redundancy.

[0066] I Slice: Intra Slice. An image can be divided into one frame or two fields, and a frame can be further divided into one or more slices.

[0067] The method provided in this disclosure can be applied to products using video codecs or video compression, and is applicable to both lossy and lossless data compression encoding and decoding. The data involved in the encoding and decoding process refers to one or a combination of the following examples:

[0068] 1) One-dimensional data;

[0069] 2) Two-dimensional data;

[0070] 3) Multidimensional data;

[0071] 4) Graphics;

[0072] 5) Images;

[0073] 6) Image sequences;

[0074] 7) Video;

[0075] 8) Three-dimensional scene;

[0076] 9) A sequence of continuously changing 3D scenes;

[0077] 10) Virtual reality scenarios;

[0078] 11) A sequence of continuously changing virtual reality scenes;

[0079] 12) Images in pixel format;

[0080] 13) Transform domain data of the image;

[0081] 14) A collection of bytes with two or more dimensions;

[0082] 15) A set of two or more bits;

[0083] 16) A set of pixels;

[0084] 17) A set of three-component pixels (Y, U, V);

[0085] 18) A set of three-component pixels (Y, Cb, Cr);

[0086] 19) A set of three-component pixels (Y, Cg, Co);

[0087] 20) A set of three-component pixels (R, G, B);

[0088] 21) A set of four-component pixels (C, M, Y, K);

[0089] 22) A set of four-component pixels (R, G, B, A);

[0090] 23) A set of four-component pixels (Y, U, V, A);

[0091] 24) A set of four-component pixels (Y, Cb, Cr, A);

[0092] 25) A set of four-component pixels (Y, Cg, Co, A).

[0093] When the data is an image, a sequence of images, or a video as listed above, a coding block is a coding region of an image and should include at least one of the following: a set of images, a predetermined number of images, an image, a frame of an image, a field of an image, a sub-image of an image, a strip, a macroblock, a maximum coding unit (LCU), a coding tree unit (CTU), or a coding unit (CU).

[0094] Figure 1A block diagram of an example video encoding / decoding system that can utilize the techniques of this disclosure is shown. As illustrated, the video encoding / decoding system may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0095] Video source 112 may include sources such as video capture devices. Examples of video capture devices include, but are not limited to, interfaces for receiving video data from video content providers, computer graphics systems for generating video data, and / or combinations thereof.

[0096] Video data may include one or more pictures / images. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may include bit sequences forming a coded and decoded representation of the video data. The bitstream may include coded and decoded pictures and associated data. A coded and decoded picture is a coded and decoded representation of a picture. Associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. I / O interface 116 may include a modulator / demodulator and / or a transmitter. Encoded video data can be directly transmitted to destination device 120 via network 130A through I / O interface 116. Encoded video data may also be stored on storage medium / server 130B for access by destination device 120.

[0097] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may acquire encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to a user. The display device 122 may be integrated with the destination device 120, or it may be external to the destination device 120, which is configured to interface with an external display device.

[0098] The video encoder 114 and the video decoder 124 can operate according to video compression standards, such as the High Efficiency Video Coding (HEVC, also known as H.265) standard, the Versatile Video Coding (VVC, also known as H.266) standard, the H.267 standard, the Audio Video Coding Standard (AVS) standard, and other existing and / or future standards.

[0099] Figure 2 A block diagram illustrating an example of a video encoder according to some embodiments of the present disclosure. The video encoder may be... Figure 1 An example of a video encoder 114 in the system shown.

[0100] The video encoder can be configured to implement any or all of the technologies disclosed herein. Figure 2 In the examples, the video encoder includes multiple functional components. The techniques described in this disclosure can be shared among the various functional components of the video encoder. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0101] Video signals, in terms of how they are acquired, can include those captured by a camera and those generated by a computer. Due to differences in statistical characteristics, the corresponding compression encoding methods may also differ.

[0102] Some video coding technologies, such as HEVC, VVC, and AVS, employ a hybrid coding framework. For example... Figure 2 As shown, the images in the input video signal are encoded sequentially, and the following series of operations and processes are performed:

[0103] 1) Block Partition Structure: Based on a preset size, the input image is divided into several non-overlapping processing units. A similar compression operation is performed on each processing unit. This processing unit can be called a CTU or LCU. Further subdivisions can be made within the CTU or LCU to obtain at least one (or more) basic encoded units, called CUs. Each CU is the most basic element in a coding process. The following describes the various coding methods that can be used for each CU.

[0104] A video encoder may include partitioning units, which can divide the input video image / picture into one or more video blocks. Video encoders and decoders can support various video block sizes.

[0105] 2) Predictive Coding: This includes intra-frame prediction and inter-frame prediction. The original video signal is predicted using a selected reconstructed video signal to obtain the residual video signal. The encoder needs to determine the most suitable predictive coding mode from among many possible modes for the current CU and inform the decoder.

[0106] The video encoder may include a mode selection unit / encoding mode determination unit, which may select one of several encoding / decoding modes (intra-frame encoding / decoding or inter-frame encoding / decoding), for example, based on error results, and use the resulting intra-frame or inter-frame encoded / decoded blocks to generate residual block data to reconstruct the encoded blocks, which can be used as reference pictures / reference images / reference frames. In some examples, the mode selection unit / encoding mode determination unit may select a combination of intra-frame and inter-frame prediction modes, where prediction is based on inter-frame prediction signals and intra-frame prediction signals. In the case of inter-frame prediction, the mode selection unit / encoding mode determination unit may also select a resolution for the block based on the motion vector (e.g., sub-pixel precision or integer pixel precision).

[0107] a. Intra-frame prediction: The predicted signal comes from a region within the same image that has already been encoded and reconstructed (some or all of which can be used as a reference region in the embodiments of this disclosure).

[0108] The basic idea behind intra-frame prediction is to remove spatial redundancy by utilizing the correlation between neighboring pixels. In video coding, neighboring pixels refer to the reconstructed pixels of the coded CUs surrounding the current CU.

[0109] A video encoder may include an intra-prediction unit, which performs intra-prediction on the current video block. When performing intra-prediction on the current video block, the intra-prediction unit can generate prediction data for the current video block based on decoded samples from other video blocks in the same frame. The prediction data for the current video block may include the predicted video block and various syntax elements.

[0110] b. Inter-frame prediction: The predicted signal comes from other images that have already been encoded and are different from the current image (referred to as reference images or reference frames, some or all of which can be used as reference regions in the embodiments of this disclosure).

[0111] To perform inter-frame prediction on the current video block, the motion estimation unit can generate motion information for the current video block by comparing it with one or more reference frames from the decoded image buffer. The motion-compensated prediction unit can then determine the predicted video block for the current video block based on the motion information and decoded samples of images from the decoded image buffer, excluding the image associated with the current video block.

[0112] The motion estimation unit and the motion-compensation prediction unit can perform different operations on the current video block, for example, depending on whether the current video block is in an I-strip, P-strip, or B-strip. As used herein, an "I-strip" can refer to a portion of an image composed of macroblocks, all of which are based on macroblocks within the same image. Furthermore, as used herein, in some aspects, "P-strip" and "B-strip" can refer to portions of an image composed of macroblocks independent of macroblocks within the same image.

[0113] In some examples, the motion estimation unit can perform unidirectional prediction on the current video block, and can search for reference images in list 0 (L0) or list 1 (L1) to find a reference video block for the current video block. The motion estimation unit can then generate a reference index indicating the reference image in list 0 or list 1 containing the reference video block, and a motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. The motion-compensated prediction unit can generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0114] Alternatively, in other examples, the motion estimation unit can perform bidirectional prediction on the current video block. The motion estimation unit can search for reference images in list 0 to find a reference video block for the current video block, and can also search for reference images in list 1 to find another reference video block for the current video block. The motion estimation unit can then generate multiple reference indices and multiple motion vectors, where the multiple reference indices indicate multiple reference images in lists 0 and 1 containing multiple reference video blocks, and the multiple motion vectors indicate multiple spatial displacements between the multiple reference video blocks and the current video block. The motion estimation unit can output the multiple reference indices and multiple motion vectors of the current video block as motion information for the current video block. The motion-compensated prediction unit can generate a predicted video block for the current video block based on the multiple reference video blocks indicated by the motion information of the current video block.

[0115] In some examples, the motion estimation unit can output a complete set of motion information for use in the decoder's decoding process. Alternatively, in some embodiments, the motion estimation unit can reference the motion information of another video block to transmit the motion information of the current video block via signaling. For example, the motion estimation unit can determine that the motion information of the current video block is sufficiently similar to the motion information of adjacent video blocks.

[0116] In one example, the motion estimation unit can indicate a value to the video decoder in the syntax structure associated with the current video block, indicating that the current video block has the same motion information as another video block. In another example, the motion estimation unit can identify another video block and the motion vector difference (MVD) in the syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0117] As discussed above, video encoders can transmit motion vectors via signaling in a predictive manner. Two examples of predictive signaling techniques that can be implemented by video encoders include Advanced Motion Vector Prediction (AMVP) and merged mode signaling.

[0118] In different prediction modes and implementations, the displacement vector may have different names. In this embodiment, it is uniformly described as follows: 1) The displacement vector in inter-frame prediction is called motion vector (MV); 2) The displacement vector in intra-frame block copy is called block vector or block displacement vector; 3) The displacement vector in intra-frame string copy is called string vector (SV).

[0119] The video encoder may include a residual generation unit that generates residual data for the current video block by subtracting (e.g., indicated by a minus sign) multiple predicted video blocks from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample portions of samples in the current video block.

[0120] In other examples, such as in skip mode, residual data for the current video block may not exist, and the residual generation unit may not perform a subtraction operation.

[0121] 3) Transform Coding and Quantization: The residual video signal undergoes transform operations such as DFT (Discrete Fourier Transform) and DCT (Discrete Cosine Transform) to convert the residual video signal into the transform domain, which are called transform coefficients. The residual video signal in the transform domain is then further subjected to lossy quantization, losing some information to make the quantized signal more suitable for compression.

[0122] In some video coding standards, there may be more than one transformation method to choose from. Therefore, the encoder also needs to select one of the transformations for the current CU (current encoding CU) to be encoded and inform the decoder.

[0123] The fineness of quantization is usually determined by the quantization parameters (QP). A larger QP value means that a wider range of transform coefficients will be quantized into the same output, which usually results in greater distortion and a lower bit rate. Conversely, a smaller QP value means that a smaller range of transform coefficients will be quantized into the same output, which usually results in less distortion and a higher bit rate.

[0124] 4) Entropy Coding or Statistical Coding: The quantized transform domain signal will be statistically compressed and encoded according to the frequency of each value, and finally output as a binary (0 or 1) compressed bitstream, or encoded bitstream, or encoded video bitstream, or encoded video data.

[0125] At the same time, other information generated by encoding, such as the selected coding modes and motion vectors, also needs to be entropy encoded to reduce the bit rate.

[0126] Statistical coding is a lossless coding method that can effectively reduce the bit rate required to represent the same signal. Common statistical coding methods include Variable Length Coding (VLC) or Content Adaptive Binary Arithmetic Coding (CABAC).

[0127] 5) Loop Filtering: After the encoded image undergoes inverse quantization, inverse transform, and prediction compensation (the reverse of steps 2) to 4) above, a reconstructed decoded image is obtained. Compared to the original input image, the reconstructed decoded image differs in some information due to the influence of quantization, resulting in distortion. Filtering the reconstructed decoded image, such as deblocking, SAO (Sample Adaptive Offset), or ALF (Adaptive Loop Filter), can effectively reduce the distortion caused by quantization. Since these filtered reconstructed decoded images will serve as a reference for subsequent encoded images to predict future signals, the above filtering operations are also called loop filtering, or filtering operations within the encoding loop.

[0128] Figure 2 A basic flowchart of a video encoder is shown. Figure 2 The k-th CU (labeled as s) k Let's take [x, y] as an example. Here, k is a positive integer greater than or equal to 1 and less than or equal to the number of CUs in the current input image, and s... k [x, y] represents the pixel with coordinates [x, y] in the k-th CU, where x represents the x-coordinate and y represents the y-coordinate. Both x and y are positive integers greater than or equal to 1, and their upper limit depends on the number of pixels in the k-th CU. k The predicted signal is obtained after [x,y] undergoes a preferred processing method such as motion compensation or intra-frame prediction. s k [x,y] and Subtraction yields the residual signal u k [x,y], and then this residual signal u k The [x,y] region is transformed and quantized. The quantized output data has two different uses: one is sent to an entropy encoder for entropy encoding, and the encoded bitstream is stored in a buffer, waiting to be transmitted; the other use is to perform inverse quantization and inverse transform to obtain the signal u'. k [x,y]. The signal u' k [x,y] and The sum is used to obtain a new prediction signal s * k [x,y], and s * k [x, y] are stored in the buffer of the current image. *k [x,y] is obtained by intra-image prediction f(s) * k [x,y]). s * k [x,y] is filtered by a loop to obtain s' k [x,y], and s' k [x, y] are stored in the decoded image buffer for use in generating the reconstructed video. k [x,y] is obtained after motion-compensation prediction, s' r [x+m x ,y+m y ], s' r [x+m x ,y+m y ] represents the reference block, m x and m y These represent the horizontal and vertical components of the motion vector, respectively.

[0129] Based on the above encoding process, it can be seen that at the decoding end, for each CU, after obtaining the compressed bitstream, the decoder first performs entropy decoding to obtain various mode information (including coding mode information, such as intra-frame prediction mode or inter-frame prediction mode) and quantized transform coefficients. Each coefficient undergoes inverse quantization and inverse transform to obtain the residual signal. On the other hand, based on the known coding mode information, the prediction signal corresponding to that CU can be obtained. Adding the two together yields the reconstructed signal. Finally, the reconstructed value of the decoded image needs to undergo loop filtering to generate the final output signal.

[0130] In other examples, the video encoder may include more, fewer, or different functional components. In one example, it may include an Intra-Block Copy (IBC) unit. The IBC unit can perform prediction in IBC mode, where at least one reference picture is the picture containing the current video block.

[0131] Furthermore, although some components (such as motion estimation units and motion-compensation prediction units) can be integrated, for interpretation purposes, these components are... Figure 2 The examples are shown separately.

[0132] Figure 3 This is a block diagram illustrating an example of a video decoder 300 according to some embodiments of the present disclosure. The video decoder 300 may be... Figure 1 An example of video decoder 124 in the system shown.

[0133] The video decoder 300 can be configured to perform any or all of the technologies disclosed herein. Figure 3In the example, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various functional components of the video decoder 300. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0134] exist Figure 3 In the example, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally contrasted with the encoding process described with respect to the video encoder 200.

[0135] Entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded video data blocks). Entropy decoding unit 301 can decode the entropy-encoded video data, and motion compensation unit 302 can determine motion information from the entropy-decoded video data, including motion vectors, motion vector precision, reference picture / image list indices, and other motion information. Motion compensation unit 302 can determine this information, for example, by performing Advanced Motion Vector Prediction (AMVP) and merging mode. Motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indices, and, in the case of a prediction region in a B-strip, an identifier of which reference picture list is associated with each index. As used herein, in some aspects, "merging mode" may refer to deriving motion information from spatially or temporally adjacent blocks.

[0136] The motion compensation unit 302 can generate motion compensation blocks and can perform interpolation based on interpolation filtering. Identifiers for interpolation filters used at sub-pixel precision can be included in the syntax elements.

[0137] The motion compensation unit 302 can use the interpolation filtering used by the video encoder 200 during the encoding of the video block to calculate the subpixel interpolation for the reference block.

[0138] The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 based on the received syntax information, and the motion compensation unit 302 can use the interpolation filter to generate prediction blocks.

[0139] Motion compensation unit 302 may use at least some of the syntax information to determine the block size for encoding the (multiple) frames and / or (multiple) stripes of the coded video sequence, partitioning information describing how each macroblock of the picture in the coded video sequence is divided, a pattern indicating how each partition is encoded, one or more reference frames (and a list of reference frames) for each inter-frame coded block, and other information for decoding the coded video sequence. As used herein, in some aspects, a “strip” can refer to a data structure that can be decoded independently of other stripes of the same picture in terms of entropy encoding / decoding, signal prediction, and residual signal reconstruction. A strip can be the entire picture or a region of a picture.

[0140] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Dequantization unit 303 dequantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 303 applies an inverse transform.

[0141] The reconstruction unit 306 can obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra-frame prediction unit 303. If necessary, deblocking filtering can also be applied to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation / intra-frame prediction and also generates decoded video for presentation on a display device.

[0142] Some exemplary embodiments of this disclosure will be described in detail below. Although some embodiments have been described with reference to general video codecs or other specific video codecs, the disclosed techniques are also applicable to other video codec techniques. Furthermore, although some embodiments describe video codec steps in detail, it should be understood that the corresponding decoding steps for canceling codecs will be implemented by the decoder. Additionally, the term video processing includes video codecs or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another or at different compression bitrates.

[0143] In normal inter-frame prediction or intra-block copy mode, predicted pixels are obtained by searching within a reconstructed reference frame / reference image / reference picture or the reconstructed region of the current frame (the region that has already been encoded and reconstructed), or by searching within its sub-pixel interpolated image (i.e., the interpolated reference image). The interpolated reference image effectively expands the number of candidates for the search, adding several (or more) neighboring sub-pixel positions to each integer pixel location.

[0144] It should be noted that when narrowing the search range of the template using the method provided in the embodiments of this disclosure, subpixel interpolation may or may not be used within the reference area.

[0145] Since the matching search process of the above-mentioned ordinary inter-frame prediction or intra-frame block copy mode not only needs to consider the matching degree of the image (for example, loss functions such as SAD (sum of absolute difference) or SATD (sum of absolute transformed difference) can be used), but also the encoding cost of motion vectors or block vectors, it may not be optimal in the dimension of matching degree alone. That is to say, there is still some room for improvement in prediction accuracy.

[0146] Template Matching Prediction (TMP) is a predictive coding mode that can be used in SCC (Simplified Coded Coding). TMP searches for the most similar template (e.g., an L-shaped template) within a predefined search range (reference region) of the reconstructed portion. The block corresponding to this most similar template is then used as the predicted block / predicted value for the current block. The encoder then signals the use of this mode, and the same prediction operation is performed on the decoder side. TMP can include intra-frame TMP and inter-frame TMP. Intra-frame TMP refers to a reference region located in the current image / frame containing the current block, while inter-frame TMP refers to a reference image / frame containing the reference region within the current block.

[0147] Compared to ordinary inter-frame prediction or intra-frame block copy mode, when template matching prediction is used, no additional encoding of motion vectors or block vectors is required on the encoder side, saving the encoding cost of motion vectors or block vectors and thus improving encoding efficiency.

[0148] Generally, the larger the reference region, the more matching positions there are, and the better the best predicted block can be found by template matching, resulting in higher encoding performance. However, template matching needs to be performed simultaneously at the encoding and decoding ends, and a large reference range leads to higher computational complexity and a larger memory space required to store the reference region. This embodiment of the present disclosure reduces the size of the reference region, i.e., reduces the target search range of the template, thereby reducing the computational complexity at the encoding and decoding ends and reducing the memory space required to store the reference region.

[0149] Figure 4 A flowchart illustrating a template matching search range setting method according to some embodiments of the present disclosure is shown. Figure 4The method provided in the embodiments can be executed by any electronic device with information processing capabilities, for example, it can be executed on the video encoder and / or video decoder side. Figure 4 As shown, the method provided in this disclosure embodiment may include the following steps.

[0150] In S410, the target template features of the current block's template are obtained.

[0151] In this embodiment of the disclosure, if the method is executed by the video encoder, the current block refers to the block currently to be encoded, and can also be called the encoded block / block to be encoded / current block to be encoded, such as the current CU. If the method is executed by the video decoder, the current block refers to the block currently to be decoded, and can also be called the decoded block / block to be decoded / current block to be decoded, such as the current CU to be decoded, or simply the current CU.

[0152] In this embodiment of the disclosure, the template of the current block refers to a set of pixels in the current frame / image where the current block is located, which surrounds or partially surrounds the current block according to a preset shape, and the pixels in the template are adjacent to / near the corresponding pixels in the current block. For example, the template can be set as an L-shaped template (as follows). Figure 7 As shown in the diagram, the template includes a template region adjacent to the left pixel of the current block and a template region adjacent to the top pixel of the current block. This disclosure does not limit the shape of the template for the current block; for example, the template may also include a template region adjacent to the top pixel of the current block, or a template region adjacent to the left pixel of the current block.

[0153] In this embodiment of the disclosure, the target template feature refers to one or more feature values ​​extracted based on the set of pixels / pixel values ​​contained in the template of the current block, and / or, the extracted one or more feature values ​​are transformed, for example, the extracted one or more feature values ​​are converted into hash values. Optionally, the target template feature can also be a fusion process of the extracted one or more feature values ​​and the transformed one or more hash values, such as concatenation / splicing.

[0154] In S420, the target region features of the reference region of the current block are obtained.

[0155] In this embodiment of the disclosure, the reference region of the current block refers to the region in which a matching template most similar to the template of the current block is searched, and the pixels contained within the reference region are reconstructed pixels. For intra-frame template matching, the reference region is located in the reconstructed or encoded region of the current frame / current image. For inter-frame template matching, the reference region is located within a reference frame of the current frame, which has been reconstructed or encoded.

[0156] In this embodiment of the disclosure, the target region feature refers to one or more feature values ​​extracted based on the set of pixels / pixel values ​​contained in the reference region of the current block, and / or, the extracted one or more feature values ​​are transformed, for example, the extracted one or more feature values ​​are converted into hash values. Optionally, the target region feature can also be a fusion process of the extracted one or more feature values ​​and the transformed one or more hash values, such as concatenation / splicing.

[0157] In S430, the target search range of the template is determined based on the target template features and target region features.

[0158] In this embodiment of the disclosure, the target template features and the target region features can be compared. Regions / sub-regions corresponding to the target region features that do not match the target template features are removed from the reference region. The remaining regions / sub-regions are then searched for the most similar matching template. The remaining regions / sub-regions are determined as the target search range of the template.

[0159] The method provided in this disclosure first obtains the target template features of the current block's template and the target region features of the current block's reference region before performing template matching search for the current block. Based on the obtained target template features and target region features, the target search range of the template is determined, which can reduce the search range of template matching, thereby reducing the computational complexity of template matching prediction and reducing the memory space required to store the reference region.

[0160] In an exemplary embodiment, obtaining the target template features of the current block template includes: obtaining the value of a setting position of at least one of the mean, variance, gradient, and sum of the pixel value set in the template; and constructing the target template features based on the value of the setting position of at least one of the mean, variance, gradient, and sum of the pixel value set in the template.

[0161] In an exemplary embodiment, the setting bit is a predetermined number of bits of the feature.

[0162] In some embodiments, the mean of the pixel values ​​of all pixels included in the template can be calculated as a feature value, and this mean can be used as a target template feature. In other embodiments, the variance of the pixel values ​​of all pixels included in the template can be calculated as a feature value, and this variance can be used as the target template feature. In still other embodiments, the gradient of the pixel values ​​of all pixels included in the template can be calculated as a feature value, and this gradient can be used as the target template feature. In yet another embodiment, the gradients of the pixel values ​​of all pixels included in the template can be summed to obtain the gradient sum as a feature value, and this gradient sum can be used as the target template feature. In yet another embodiment, one or more of the above-calculated mean, variance, gradient, gradient sum, etc., of the pixel values ​​of all pixels included in the template can be used as the target template feature. In yet another embodiment, a set of one or more of the above-calculated mean, variance, gradient, gradient sum, etc., of the pixel values ​​of all pixels included in the template can be used as the target template feature.

[0163] In this embodiment of the disclosure, the setting bit can be all or part of the mean, variance, gradient, gradient sum, etc., of the pixel values ​​of all pixels included in the template. The setting bits for the mean, variance, gradient, gradient sum, etc., can be the same or different. For example, the setting bits for the mean and variance can be the high 4 bits, and the setting bits for the gradient sum can be all the bits of the gradient sum.

[0164] For example, the following Figure 7 Taking the L-shaped template of the current block as an example, assuming there are a total of 9 pixels, the average value of these 9 pixels is calculated, and then this average value is represented by 8 bits of binary. For example, if it is 11001010, then the high 4 bits 1100 are selected as the target template feature or a component of the target template feature.

[0165] For example, the variance of the pixel values ​​of these 9 pixels can be used as a feature value, or the high 4 bits of the feature value can be used as a feature value.

[0166] In an exemplary embodiment, the template includes a first region and / or a second region; the gradient includes a vertical gradient of the first region and / or a horizontal gradient of the second region.

[0167] For example, if the template is L-shaped, then the first region is the left template region of the current block, and the second region is the top template region of the current block. Calculate the vertical gradient of the pixels within the left template region. Optionally, the vertical gradient sum can be obtained by summing the vertical gradients of the left template region. Calculate the horizontal gradient of the pixels within the top template region. Optionally, the horizontal gradient sum can be obtained by summing the horizontal gradients of the top template region. That is, the above gradients can include the vertical gradient of the left template region and / or the horizontal gradient of the top template region, and the above gradient sum can include the vertical gradient sum of the left template region and / or the horizontal gradient sum of the top template region.

[0168] In an exemplary embodiment, when the number of pixels in the first region is more than one column, the gradient also includes the horizontal gradient of the first region; when the number of pixels in the second region is more than one row, the gradient also includes the vertical gradient of the second region.

[0169] When the template has only one row and / or one column of pixels, the prediction accuracy is limited. In this embodiment, the left template region may include more than one column of pixels, and / or the upper template region may include more than one row of pixels, thereby improving prediction accuracy. When the left template region has more than one column, the horizontal gradient between different columns of pixels in the left template region is calculated. That is, the gradient may also include the horizontal gradient of the left template region. Optionally, the horizontal gradients between different columns of pixel values ​​in the left template region can be summed, meaning the gradient sum may also include the horizontal gradient sum of the left template region. When the upper template region has more than one row, the vertical gradient between different rows of pixels in the upper template region is calculated. That is, the gradient may also include the vertical gradient of the upper template region. Optionally, the vertical gradients between different rows of pixel values ​​in the upper template region can also be summed, meaning the gradient sum may also include the vertical gradient sum of the upper template region.

[0170] In this embodiment, horizontal and vertical gradient information can be calculated separately, representing the trend of texture change in that direction (horizontal or vertical). The sum of gradients represents the overall trend of the entire template region. Optionally, after obtaining the gradient or gradient sum, the higher-order bits can be used to represent the target template features.

[0171] In an exemplary embodiment, constructing the target template feature based on the value of at least one of the set position of the mean, variance, and gradient of the pixel value set in the template includes: a first part of the target template feature is determined by the value of the set position of the mean of the pixel value set in the template, a second part of the target template feature is determined by the value of the set position of the variance of the pixel value set in the template, and a third part of the target template feature is determined by the value of the set position of the gradient of the pixel value set in the template.

[0172] In some embodiments, after extracting the mean, variance, gradient, gradient sum, and other feature values ​​of the pixel value set in the template, one or more of these features can be used to construct the hash value of the template, and the constructed hash value serves as the feature of the target template. For example, certain bits in the constructed hash value (referred to as the first part of the feature values ​​for distinction) are determined by the mean or the set bit of the mean of the template, certain bits (referred to as the second part of the feature values ​​for distinction) are determined by the variance or the set bit of the variance of the template, and certain bits (referred to as the third part of the feature values ​​for distinction) are determined by the gradient or the gradient sum or the set bit of the gradient or the set bit of the gradient sum of the template.

[0173] In this embodiment of the disclosure, the calculation of the template's hash value can be constructed by comprehensively considering multiple feature information / feature values ​​of the template. For example, assuming the template's hash value is 16 bits, divided into four 4-bit fields, the first field is the high 4 bits of the template's mean, the second field is the high 4 bits of the template's variance, the third field is the high 4 bits of the template's horizontal gradient sum, and the fourth field is the high 4 bits of the template's vertical gradient sum.

[0174] In an exemplary embodiment, obtaining the target region features of the reference region of the current block includes: obtaining a matching template that matches the template within the reference region of the current block; and obtaining the matching template features of the matching template, wherein the target region features include the matching template features. Determining the target search range of the template based on the target template features and the target region features includes: comparing the target template features with the matching template features, and removing matching templates whose matching template features do not match the target template features, thereby determining the target search range.

[0175] For example, such as Figure 7 As shown, within the reference area, moving a matching template with the same shape as the current block's template by positive pixels or subpixels, such as from left to right or from top to bottom, can obtain all matching templates / matching positions / search positions within the reference area that match this template. Figure 7 The search positions 1 to 7 shown are 7 matching templates obtained by taking one pixel at a time. Here, matching means that the shape and size are the same as the template of the current block.

[0176] In this embodiment of the disclosure, the matching template feature refers to one or more feature values ​​extracted based on the set of pixels / pixel values ​​contained in the matching template, and / or, the extracted one or more feature values ​​are transformed, for example, the extracted one or more feature values ​​are converted into hash values. Optionally, the matching template feature can also be a fusion process of the extracted one or more feature values ​​and the transformed one or more hash values, such as concatenation / splicing.

[0177] After obtaining the matching template features of each matching template, each matching template feature is compared with the target template feature. Matching templates whose matching template features do not match the target template features are removed. Among the remaining matching templates, the matching template most similar to the template of the current block is searched. That is, the remaining matching templates are used as the target search range.

[0178] In an exemplary embodiment, obtaining the matching template features of the matching template includes: obtaining the value of a setting position of at least one of the mean, variance, gradient, and sum of the pixel value set in the matching template; and constructing the matching template features based on the value of the setting position of at least one of the mean, variance, gradient, and sum of the pixel value set in the matching template.

[0179] In this embodiment of the disclosure, the target template features, target region features, and matching template features are obtained in the same way. For example, when the target template features are constructed using the set value of at least one of the mean, variance, gradient, and gradient sum of the pixel value set in the template, the matching template features are also constructed using the set value of at least one of the mean, variance, gradient, and gradient sum of the pixel value set in the template.

[0180] In some embodiments, when at least one of the feature values ​​of the mean, variance, gradient, and gradient sum of the pixel value set in the template is used as the feature value of the target template, at least one of the feature values ​​of the mean, variance, gradient, and gradient sum of the pixel value set in the matching template is used as the feature value of the matching template.

[0181] In other embodiments, when a feature value of at least one of the mean, variance, gradient, and gradient sum of the pixel value set in the template is used as the target template feature, a feature value of at least one of the mean, variance, gradient, and gradient sum of the pixel value set in the matching template is used as the matching template feature.

[0182] In some other embodiments, when the hash value converted from the feature value of at least one of the mean, variance, gradient, and gradient of the pixel value set in the template is used as the target template feature, the hash value converted from the feature value of at least one of the feature value of the mean, variance, gradient, and gradient of the pixel value set in the matching template is used as the matching template feature.

[0183] For example, suppose the hash value of the template is 16 bits, divided into four 4-bit fields. The first field is the high 4 bits of the template's mean, the second field is the high 4 bits of the template's variance, the third field is the high 4 bits of the template's horizontal gradient sum, and the fourth field is the high 4 bits of the template's vertical gradient sum. Then, the hash value of the matching template is 16 bits, divided into four 4-bit fields: the first field is the high 4 bits of the matching template's mean, the second field is the high 4 bits of the matching template's variance, the third field is the high 4 bits of the matching template's horizontal gradient sum, and the fourth field is the high 4 bits of the matching template's vertical gradient sum.

[0184] Once the hash value of the template and the hash value of the matching template are obtained, they can be compared. Matching templates whose hash values ​​do not match the hash value of the template are removed, and template matching is not performed. Template matching is then performed on the remaining matching templates (or the remaining reference positions / search positions / matching positions) to search for the target matching template.

[0185] In this embodiment, the comparison of the high-order bits of the feature values ​​of the template and the matching template to determine if they are similar is based on the fact that the low-order bits of the feature values ​​reflect the details of the template and the matching template. Since it is only necessary to determine whether the template and the matching template are approximately the same, removing the low-order bits can reduce the amount of data processing and improve data processing efficiency while maintaining accuracy. The more high-order bits retained for comparison, the fewer remaining matching templates will have the same hash value or feature value as the template.

[0186] In an exemplary embodiment, the method provided in this disclosure further includes: determining a target matching template from matching templates whose matching template features match the target template features; and implementing predictive coding of the current block according to a reference block corresponding to the target matching template.

[0187] In this embodiment, for the remaining matching templates that match the hash value or feature value, template matching is performed to find the best matching position / best matching template as the target matching template, thus completing the predictive encoding of the current block. For example, this can be achieved by comparing the template with the SAD or SATD of each remaining matching template, and selecting the matching template with the smallest SAD or SATD as the target matching template.

[0188] The method provided in this disclosure can be applied to both intra-frame predictive coding and inter-frame predictive coding. When applied to intra-frame predictive coding, the reference block is located within the current frame containing the current block. In this case, the reference region within the current frame can be located to the left or top of the current block. When applied to inter-frame predictive coding, the reference block is located within the reference block of the current block. In this case, the reference region can be the entire reference frame or a portion of the reference frame.

[0189] The method provided in this disclosure can be applied to both the encoding end (e.g., a video encoder) and the decoding end (e.g., a video decoder). That is, both the encoding end and the decoding end execute the same template matching search range setting method to reduce the target search range of the template. In this way, on the one hand, both the encoding end and the decoding end are simplified, reducing computational complexity and reducing the memory space used to store the reference area; on the other hand, the encoding end does not need to encode motion information (including motion vectors) nor does it need to transmit motion information to the decoding end, thereby reducing the encoding cost of motion information and improving encoding efficiency.

[0190] In an exemplary embodiment, obtaining the value of a setting bit of at least one of the mean, variance, gradient, and other features of the pixel value set in the template includes: obtaining the value of a setting bit of at least one of the mean, variance, gradient, and other features of the pixel value set of at least one component signal in the template.

[0191] In this embodiment of the disclosure, the calculation of the template, the feature value and / or hash value of the matching template may consider only one component of the pixel value, or consider two components of the pixel value, or consider three components of the pixel value, or even more components of the pixel value.

[0192] For example, the hash value calculation above can consider only one component (luminance) signal, or it can consider three components (one luminance and two chrominance) at the same time, or it can consider two components (one luminance and one chrominance).

[0193] In an exemplary embodiment, when the set value of at least one of the mean, variance, gradient, and sum of the pixel value sets of multiple component signals in the template is obtained, constructing the target template feature based on the set value of at least one of the mean, variance, gradient, and sum of the pixel value sets in the template includes: constructing template features for each component signal based on the set value of at least one of the mean, variance, gradient, and sum of the pixel value sets of each component signal in the template; and constructing the target template feature based on the template features of each component signal.

[0194] In an exemplary embodiment, constructing the target template feature based on the template features of each component signal includes: summing the template features of each component signal to obtain the target template feature; or: concatenating the template features of each component signal to obtain the target template feature.

[0195] When considering two or more components simultaneously, one approach is to first calculate the feature values ​​and / or hash values ​​of each template and matching template separately based on each component. Then, the feature values ​​and / or hash values ​​of each component of the template are summed to obtain the target template feature, and the feature values ​​and / or hash values ​​of each component of the matching template are summed to obtain the target region feature. Alternatively, one approach is to first calculate the feature values ​​and / or hash values ​​of each template and matching template separately based on each component. Then, the feature values ​​of each component of the template are placed into different fields of the feature values ​​to obtain the target template feature, or the hash values ​​of each component of the template are placed into different fields of the hash values ​​to obtain the target template feature. Similarly, the feature values ​​of each component of the matching template are placed into different fields of the feature values ​​to obtain the matching template feature, or the hash values ​​of each component of the matching template are placed into different fields of the hash values ​​to obtain the matching template feature.

[0196] In an exemplary embodiment, obtaining the target template features of the template of the current block includes: determining the smallest unit; dividing the template into multiple smallest units; obtaining the target features of each smallest unit in the template; and obtaining the target template features based on the target features of each smallest unit in the template.

[0197] In an exemplary embodiment, obtaining the target features of each smallest unit in the template includes: obtaining the set value of at least one of the mean, variance, gradient, and sum of the pixel value set of each smallest unit in the template; and constructing the target features of the corresponding smallest unit based on the set value of at least one of the mean, variance, gradient, and sum of the pixel value set of each smallest unit in the template.

[0198] In an exemplary embodiment, obtaining the target region features of the reference region of the current block includes: dividing the reference region of the current block into multiple smallest units; obtaining the target features of each smallest unit within the reference region; obtaining a matching template that matches the template within the reference region of the current block; and determining the matching template features of the matching template based on the target features of each smallest unit within the reference region, wherein the target region features include the matching template features. Determining the target search range of the template based on the target template features and the target region features includes: comparing the target template features with the matching template features, and removing matching templates whose matching template features do not match the target template features, thereby determining the target search range.

[0199] In this embodiment of the disclosure, a minimum unit range can be defined. The minimum unit can be a power of 2 in size, such as a 2x2 pixel set or a 4x4 pixel set. The following example uses a 2x2 pixel set as the minimum unit, but this disclosure is not limited to this. The template contains several (one or more) minimum units. If the minimum unit is 2x2, then the template is at least an integer multiple of 2x2, such as 2 rows and 2 columns.

[0200] In some embodiments, feature extraction is performed on the reference pixel values ​​(set of pixel values) of each minimum unit within the reference region to obtain at least one of the following: mean, variance, gradient, gradient sum, etc., or a set of at least one of them, as the feature value of each minimum unit. The feature value of the minimum unit can be used as the target feature of the minimum unit. Alternatively, a hash value of the minimum unit can be constructed based on the feature value of the minimum unit, and the hash value of the minimum unit can be used as the target feature of the minimum unit. Based on the feature value and / or hash value of the minimum unit within the matching template, and according to the template shape (e.g., L-shape), feature values ​​and / or hash values ​​of each matching template are constructed. Each field of the feature value and / or hash value of each matching template may include the feature value and / or hash value of each minimum unit included in the matching template.

[0201] In other embodiments, the smallest unit of each matching template can be determined first, and then the feature value and / or hash value of the smallest unit of each matching template can be calculated. The matching template feature is obtained based on the feature value and / or hash value of each smallest unit in the matching template. When calculating the feature value and / or hash value of the smallest unit of the next matching template, if it has already been calculated in a previous matching template, it can be directly reused.

[0202] In this embodiment of the disclosure, feature extraction is performed on the pixel value set of each smallest unit within the template to obtain at least one of the following: mean, variance, gradient, gradient sum, etc., or a set bit of at least one of them, as the feature value of each smallest unit. Alternatively, a hash value of the smallest unit can be constructed based on its feature value. Based on the feature value and / or hash value of each smallest unit within the template, feature values ​​and / or hash values ​​of each template are constructed. The fields of the feature values ​​and / or hash values ​​of each template may include the feature values ​​and / or hash values ​​of each smallest unit included in the template.

[0203] For example, assuming the hash values ​​of the smallest units within a matching template are hash1 to hashn, where n is a positive integer greater than or equal to 1, then the hash value of the matching template can be a multi-field hash value (hash1, hash2, hash3, ..., hashn). Correspondingly, assuming the hash values ​​of the smallest units within the template are hash1' to hash'n, then the hash value of the template is (hash1', hash2', hash3', ..., hash'n).

[0204] In this embodiment, reference positions / matching templates that do not match in feature value and / or hash value can be removed. For the remaining matching templates that match in hash value or feature value, template matching is performed to find the best matching position / best matching template as the target matching template, thus completing the predictive encoding of the current block. For example, this can be achieved by comparing the template with the SAD or SATD of each remaining matching template, and using the matching template with the smallest SAD or SATD as the target matching template.

[0205] This embodiment of the disclosure can construct target template features and target region features with finer granularity by dividing the target into the smallest units, thereby improving the accuracy of the determined target search range and further improving the prediction accuracy.

[0206] In an exemplary embodiment, obtaining the target region features of the reference region of the current block includes: performing downsampling processing on the reference region; extracting features from the downsampled reference region to construct the target region features.

[0207] In this embodiment, a reference region can be downsampled first (e.g., 2:1 downsampling), and then each matching template in the downsampled reference region can be obtained. Feature extraction is performed on each matching template, for example, extracting one or more of the mean, variance, gradient, and gradient sum of the pixel values ​​of each matching template in the downsampled reference region. One or more of the extracted mean, variance, gradient, and gradient sum, or a setting of at least one of them, or a hash value of at least one or more of the setting ... Then, each matching template feature is compared with the target template feature. Matching templates whose features do not match the target template features are removed, and no template matching is performed. The remaining matching templates are used as the target search range to determine the target matching template. The block corresponding to the target matching template is used as the reference block for the current block to achieve predictive encoding of the current block. For example, this can be achieved by comparing the template with the SAD or SATD of each remaining matching template, and using the matching template with the smallest SAD or SATD as the target matching template.

[0208] The embodiments disclosed herein reduce the amount of data calculation when setting the search range by downsampling the reference area, thereby improving the efficiency of setting the search range.

[0209] In an exemplary embodiment, obtaining the target region features of the reference region of the current block includes: dividing the reference region into multiple sub-regions; and obtaining sub-region features of each sub-region based on the set of pixel values ​​in each sub-region, wherein the target region features include the sub-region features of each sub-region. Determining the target search range of the template based on the target template features and the target region features includes: comparing each sub-region feature with the target template feature, and removing sub-regions whose sub-region features do not match the target template feature, thereby determining the target search range of the template.

[0210] In an exemplary embodiment, obtaining the target template feature of the template of the current block includes: using at least one of the mean, variance, and gradient of the set of pixel values ​​in the template as the target template feature. Specifically, obtaining the sub-region feature of each sub-region based on the set of pixel values ​​in each sub-region includes: using at least one of the mean, variance, and gradient of the set of pixel values ​​in the sub-region as the sub-region feature of the corresponding sub-region.

[0211] In this embodiment, the reference region can be divided into several (two or more) sub-regions. From each sub-region, one or more feature values, such as mean, variance, gradient, and gradient sum, are extracted based on the set of pixel values ​​it contains (also called the reconstructed pixel value set), and these are used as sub-region features. Based on the pixel value set of the template of the current block, one or more feature values, such as mean, variance, gradient, and gradient sum, are extracted as target template features. The features of each sub-region are compared with the target template features, and the sub-region with the closest feature values ​​is selected for template matching search, i.e., other sub-regions are removed and no template matching is performed.

[0212] In other embodiments, one or more feature values, such as mean, variance, gradient, gradient sum, etc., can be extracted from each sub-region based on the set of reconstructed pixel values ​​it contains. A hash value based on a set of bits for one or more feature values ​​is then used as a sub-region feature for each sub-region. Similarly, one or more feature values, such as mean, variance, gradient, gradient sum, etc., are extracted from the set of pixel values ​​of the template for the current block. A hash value based on a set of bits for multiple feature values ​​is then used as a target template feature.

[0213] In an exemplary embodiment, each sub-region feature is compared with the target template feature, and sub-regions whose features do not match the target template feature are removed to determine the target search range of the template. This includes selecting sub-regions whose absolute value difference between the target template feature and the sub-region feature meets a predetermined condition as the target search range. For example, the predetermined condition may be the minimum absolute value difference or less than a set threshold. The value of the threshold can be set according to actual needs, and this disclosure does not limit it.

[0214] For example, the average pixel value of each sub-region can be used as the feature value of the sub-region. By comparing it with the average pixel value of the template of the current block, the sub-region (or more) with the smallest absolute value difference is selected as the sub-region for template matching.

[0215] For example, if multiple feature values ​​of the sub-region and the template are considered simultaneously, a cost function can be set to calculate the sum of the differences between the various feature values ​​of the sub-region and the template (e.g., weighted summation). The sub-region with the smallest total difference, or whose total difference is less than a certain threshold, is selected for template matching.

[0216] The method provided in this disclosure can obtain target template features and target region features using any one or more of the methods described above. When multiple methods are used, the order of the methods is not limited. After removing sub-regions and / or matching templates that do not match the target region features obtained by each method with the target template features, template matching is then performed on the remaining sub-regions or matching templates. For example, first, matching template features are obtained through hash values, and matching templates that do not match the matching template features are removed; then, matching template features are obtained through a downsampled reference region, and matching templates that do not match the downsampled matching template features are removed; then, sub-region features are obtained through sub-region division, and mismatched sub-regions are removed; finally, template matching is performed within the finally determined target search range.

[0217] The method provided in this disclosure improves encoding efficiency because the encoder does not need to incur the cost of encoding motion information. Furthermore, the method optimizes the selection of the template matching search range, including intra-frame template matching and inter-frame template matching, effectively reducing the computational complexity of searching within the reference region while maintaining the accuracy of template matching. This method can be applied to products equipped with relevant video codecs or video compression.

[0218] In this embodiment, the template matching search range can be set based on the luminance component. A reference block for the luminance component of the current block is determined using the above method. Based on the reference block, the predicted signal / predicted value / predicted block / predicted pixel / predicted luminance result for the luminance component of the current block is determined. For the chroma component of the current block, it can be assumed that it has a similar motion vector or block vector to the luminance component. Therefore, the chroma motion vector or block vector is simply derived from the luminance motion vector or block vector. For example, for the 420 format, the chroma vector size is half the luminance vector size.

[0219] CCLM (Cross Component Linear Model) uses a luminance-based cross-component prediction of chrominance. The constructed model is y = ax + b, where x represents the pixel value of the downsampled luminance component, and y represents the pixel value of the predicted chrominance component (i.e., the predicted chrominance result). The selected template region is, for example, one line adjacent to the current block (e.g., the current coding block) (the top row and the left column). Figure 5As shown, in S1, the brightness of the current block 511 and its template 512 is downsampled to make the resolution of the brightness component consistent with that of the chromaticity component. In S2, based on the template region 522 in the downsampled brightness component and the template 532 in the original chromaticity component, the values ​​of a and b are obtained by substituting them into the formula y = ax + b, and a linear model is constructed. In S3, the downsampled brightness 521 of the current block is input into the linear model to obtain the predicted chromaticity result 541 of the chromaticity component 531 of the current block.

[0220] Figure 6 Taking intra-frame template matching at the encoding end as an example, the current block is a coded block. The current frame containing the coded block includes reconstructed reference regions and uncoded regions. When performing intra-frame predictive coding on the coded block, an optimal matching position can be selected from the already coded reconstructed regions (i.e., the reconstructed reference regions). The difference between intra-frame template matching and intra-frame block copying mode is that its block displacement vector is obtained by comparing multiple candidate positions (search positions) in the usable reference regions through template matching, and selecting the position with the minimum template matching cost (e.g., minimum SAD or SATD) as the prediction block template, i.e., the target matching template, and the corresponding block is used as the prediction block of the coded block. Figure 6 As shown, the set of neighboring encoded pixels of a coded block (also called the current coded block) forms a fixed template. This template can be matched with the corresponding pixels of the same shape (e.g., L-shape) around a reference block at a certain location within the reference region to obtain a template matching difference. For example, the template difference can be calculated using the template and the sum of the absolute values ​​(SAD) of the differences between each pixel pair in each matching template. The template matching calculation can be performed within a reconstructed reference region agreed upon by both the encoding and decoding methods.

[0221] Figure 7 An example of template matching at various search locations is given. For example... Figure 7 As shown, within the reference area, each movement of the template by one position (which can be an integer pixel or a subpixel) corresponds to a potential matching search point or search position. If the matching search point is at a subpixel position, the pixel value in the template is generated by interpolation from the reconstructed pixel values ​​adjacent to the template position. For example, interpolation can be performed between two adjacent pixels vertically, two adjacent pixels horizontally, or two adjacent pixels diagonally. Alternatively, more neighboring pixels can be included for similar weighted combinations to perform interpolation.

[0222] This disclosure does not limit the method of dividing sub-regions; the encoding and decoding ends can agree in advance to use the same sub-region concept. For example, Figure 8As shown, in intra-frame template matching or inter-frame template matching, the overall reference region can be divided into several (two or more) sub-regions according to a convention. Both the encoder and decoder can perform the aforementioned feature value and / or hash value matching process in all sub-regions, thereby reducing the target search range of the template and simplifying the computational complexity of template matching search for both the encoder and decoder. The encoder and decoder do not need to perform template matching operations in all sub-regions to find the location of the best matching block. Furthermore, the encoder does not need to write the sub-region number (identifier) ​​of the best matching block, or the sub-region size, into the bitstream; the decoder also does not need to determine the sub-region of the best matching block using the decoded identifier and then perform template matching search only within that sub-region.

[0223] Figure 9 A flowchart illustrating a template matching search range setting method according to other embodiments of this disclosure is shown schematically. For example... Figure 9 As shown, the method provided in this embodiment may include S910, which means that the target search range of the template can be determined by performing at least one of the following operations. The following operations can be performed at the decoding end and the encoding end. Any one of S911 to S914 can be performed alone, or several can be combined, and the order of combination can also be arbitrary:

[0224] S911, obtain the hash value of the pixel value set of each matching template in the reference area of ​​the current block, and remove matching templates that do not match the hash value of the pixel value set of the template of the current block.

[0225] Within the reference region, the template is moved one integer pixel or subpixel at a time to obtain all matching positions. For each possible matching position within the reference region, the hash value of the matching template pixel set is calculated. Matching positions with hash values ​​different from the current block's template pixel set are removed, and no template matching is performed. The hash value here can be calculated by extracting certain feature information from a set of pixels.

[0226] (1) Calculate the hash value of the matching template pixel set at each position, based on the pixel value at each position in the matching template.

[0227] Consider using the mean of the set of matching template pixel values ​​as a feature or feature value. Alternatively, you can select specific bits / set values ​​for the mean, such as the high 4 bits.

[0228] Consider the variance information of the matching template pixel value set as a feature or feature value. Alternatively, specific bits / set values ​​of the variance information can be selected, such as the high 4 bits.

[0229] Consider the gradient information of the matching template pixel value set and sum it in the horizontal and vertical directions. For example, calculate the vertical gradient information of the left pixels of the matching template and the horizontal gradient information of the pixels above them. When there is more than one column on the left (more than one row above) of the matching template, calculate the horizontal gradient information of the left pixels (and the vertical gradient information of the pixels above them). The horizontal and vertical gradient information are calculated separately, representing the trend of texture change in each direction. The high-order bits after summation represent the overall trend of the entire template region.

[0230] The calculation of the hash value described above can be constructed by comprehensively considering multiple feature information. For example, certain bits in the hash value are determined by the mean, certain bits by the variance, and certain bits by the gradient.

[0231] The hash value calculation can be performed by considering only one component signal (e.g., brightness) or by considering all three components simultaneously.

[0232] (2) Calculate the hash value of the template of the current block, compare it with each position in the reference area, and remove the positions where the hash value does not match.

[0233] (3) Perform template matching on the remaining reference positions / search positions / matching positions of the hash value matching, find the best matching position, and complete the intra-frame predictive coding / inter-frame predictive coding.

[0234] S912, determine the smallest unit, extract features from the pixel values ​​of each smallest unit in the reference area of ​​the current block, construct the hash value of each smallest unit in the reference area, construct the hash value of each matching template in the reference area based on the hash value of each smallest unit in the reference area, and remove matching templates that do not match the hash value of the pixel value set of the template of the current block.

[0235] Define the range of the smallest unit (e.g., a 2×2 pixel set). Extract features from the reference pixel values ​​of each smallest unit within the reference region to construct a hash value. Based on the template shape, construct a matching template hash value from the hash values ​​of the smallest units. Each field of this hash value is composed of the hash values ​​of the corresponding smallest units or values ​​derived from those hash values. Then, reference positions with mismatched hash values ​​can be removed.

[0236] S913, perform feature extraction on the reference region of the downsampled current block, construct the hash value of the pixel value set of each matching template in the downsampled reference region, and remove matching templates that do not match the hash value of the pixel value set of the template of the current block.

[0237] Feature extraction is performed on the downsampled reference region (e.g., 2:1 downsampling) to construct a hash value. Reference locations with mismatched hash values ​​can then be removed.

[0238] S914, divide the reference region of the current block into multiple sub-regions, obtain the feature value of each sub-region based on the set of pixel values ​​contained in each sub-region, and select the sub-region whose feature value matches the feature value of the template of the current block.

[0239] The reference region is divided into several sub-regions. Each sub-region derives one (or more) feature values ​​based on the set of reconstructed pixel values ​​it contains. These feature values ​​are compared with the feature values ​​derived from the pixel values ​​of the template of the current block. The sub-region with the closest feature values ​​is selected for template matching search.

[0240] Figure 9 Other aspects of the embodiments can be found in the embodiments described above.

[0241] Figure 10 A flowchart illustrating a video decoding method according to an embodiment of the present disclosure is shown schematically. Figure 10 The implementation can be executed by a video decoder or a decoding end. For example... Figure 10 As shown, the method provided in this disclosure embodiment may include the following steps.

[0242] In S1010, the current frame in the encoded video stream is obtained. The current frame includes the current block and its template.

[0243] In S1020, the target template features of the current block's template are obtained.

[0244] In S1030, the target region features of the reference region of the current block are obtained;

[0245] In S1040, the target search range of the template is determined based on the target template features and target region features.

[0246] In S1050, a target matching template that matches the template is searched within the target search range.

[0247] In S1060, the reference block for the current block is determined based on the target matching template.

[0248] In S1070, the current block is decoded based on the reference block.

[0249] The method provided in this disclosure does not require the decoding end to obtain information such as the identifier and size of the sub-region from the bitstream. Instead, it filters the reference region at the decoding end, narrows the search range, improves decoding efficiency, and reduces the computational complexity of template matching.

[0250] Figure 10 Other aspects of the embodiments can be found in the embodiments described above.

[0251] Figure 11A flowchart illustrating a video encoding method according to some embodiments of the present disclosure is shown schematically. Figure 11 The implementation can be performed by a video encoder or encoding terminal. For example... Figure 11 As shown, the method provided in this disclosure embodiment may include the following steps.

[0252] In S1110, the current frame in the video stream to be encoded is obtained, and the current frame includes the current block and its template.

[0253] In S1120, the target template features of the current block's template are obtained.

[0254] In S1130, the target region features of the reference region of the current block are obtained.

[0255] In S1140, the target search range of the template is determined based on the target template features and the target region features.

[0256] In S1150, a target matching template that matches the template is searched within the target search range.

[0257] In S1160, a reference block for the current block is determined based on the target matching template.

[0258] In S1170, the encoding of the current block is implemented according to the reference block.

[0259] The method provided in this disclosure eliminates the need for the encoding end to transmit the identifier and size of the sub-region to the decoding end, and simplifies the template matching search process at the encoding end.

[0260] Figure 11 Other aspects of the embodiments can be found in the embodiments described above.

[0261] In this embodiment of the disclosure, the same process of removing mismatched search positions using hash values ​​or feature values ​​is performed at both the encoding and decoding ends to reduce the search range. The complexity of both the encoding and decoding ends is reduced at the same time, and the encoding end does not need to transmit additional information such as the identifier and size of the sub-region to the decoding end.

[0262] Figure 12 A block diagram illustrating a template matching search range setting device according to some embodiments of the present disclosure is shown. Figure 12 As shown, the template matching search range setting device 1200 provided in this embodiment includes a processing unit 1210. The processing unit 1210 is used to obtain target template features of the template of the current block. The processing unit 1210 is also used to obtain target region features of the reference region of the current block. The processing unit 1210 is further used to determine the target search range of the template based on the target template features and the target region features.

[0263] Figure 12 Other aspects of the embodiments can be found in the embodiments described above.

[0264] Figure 13 A block diagram illustrating a template matching search range setting apparatus according to other embodiments of the present disclosure is shown. Figure 13 As shown, the template matching search range setting device 1300 provided in this embodiment includes a processing unit 1310.

[0265] Processing unit 1310 is configured to perform at least one of the following to determine the target search range of the template:

[0266] Obtain the hash value of the pixel value set of each matching template in the reference area of ​​the current block, and remove the matching templates that do not match the hash value of the pixel value set of the template of the current block;

[0267] Determine the smallest unit, extract features from the pixel values ​​of each smallest unit within the reference region of the current block, construct the hash value of each smallest unit within the reference region, construct the hash value of each matching template within the reference region based on the hash value of each smallest unit within the reference region, and remove matching templates that do not match the hash value of the pixel value set of the template of the current block.

[0268] Feature extraction is performed on the reference region of the current block after downsampling, and the hash value of the pixel value set of each matching template in the downsampling reference region is constructed. Matching templates that do not match the hash value of the pixel value set of the template of the current block are removed.

[0269] The reference region of the current block is divided into multiple sub-regions. The feature value of each sub-region is obtained based on the set of pixel values ​​contained in each sub-region. The sub-region whose feature value matches the feature value of the template of the current block is selected.

[0270] Figure 13 Other aspects of the embodiments can be found in the embodiments described above.

[0271] Figure 14 A block diagram of a video decoder according to some embodiments of the present disclosure is illustrated schematically. Figure 14As shown, the video decoder 1400 provided in this embodiment includes a receiving unit 1410 and a processing unit 1420. The receiving unit 1410 is used to acquire the current frame in the encoded video stream, the current frame including a current block and its template. The processing unit 1420 is used to obtain target template features of the template of the current block. The processing unit 1420 is also used to obtain target region features of a reference region of the current block. The processing unit 1420 is also used to determine a target search range for the template based on the target template features and the target region features. The processing unit 1420 is also used to search for a target matching template that matches the template within the target search range. The processing unit 1420 is also used to determine a reference block for the current block based on the target matching template. The processing unit 1420 is also used to decode the current block according to the reference block.

[0272] Figure 14 Other aspects of the embodiments can be found in the embodiments described above.

[0273] Figure 15 A block diagram of a video encoder according to some embodiments of the present disclosure is illustrated schematically. Figure 15 As shown, the video encoder 1500 provided in this embodiment includes a receiving unit 1510 and a processing unit 1520. The receiving unit 1510 is used to acquire the current frame in the video stream to be encoded, the current frame including a current block and its template. The processing unit 1520 is used to obtain target template features of the template of the current block. The processing unit 1520 is further used to obtain target region features of a reference region of the current block. The processing unit 1520 is further used to determine a target search range for the template based on the target template features and the target region features. The processing unit 1520 is further used to search for a target matching template that matches the template within the target search range. The processing unit 1520 is further used to determine a reference block for the current block based on the target matching template. The processing unit 1520 is further used to encode the current block according to the reference block.

[0274] Figure 15 Other aspects of the embodiments can be found in the embodiments described above.

[0275] This disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in the above embodiments.

[0276] This disclosure provides an electronic device, including: at least one processor; and a storage device configured to store at least one program, which, when executed by the at least one processor, causes the at least one processor to perform the method described in the above embodiments.

[0277] Figure 16 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown.

[0278] It should be noted that, Figure 16 The illustrated electronic device 1600 is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0279] like Figure 16 As shown, the electronic device 1600 includes a central processing unit (CPU) 1601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1602 or programs loaded from storage portion 1608 into random access memory (RAM) 1603. The RAM 1603 also stores various programs and data required for system operation. The CPU 1601, ROM 1602, and RAM 1603 are interconnected via a bus 1604. An input / output (I / O) interface 1605 is also connected to the bus 1604.

[0280] The following components are connected to I / O interface 1605: an input section 1606 including a keyboard, mouse, etc.; an output section 1607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1608 including a hard disk, etc.; and a communication section 1609 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1609 performs communication processing via a network such as the Internet. A drive 1610 is also connected to I / O interface 1605 as needed. Removable media 1611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1610 as needed so that computer programs read from them can be installed into storage section 1608 as needed.

[0281] In particular, according to embodiments of this disclosure, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1609, and / or installed from removable medium 1611. When the computer program is executed by central processing unit (CPU) 1601, it performs various functions defined in the methods and / or apparatus of this application.

[0282] It should be noted that the computer-readable storage medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having at least one wire, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF (Radio Frequency), etc., or any suitable combination thereof.

[0283] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods, apparatus, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing at least one executable instruction for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0284] The units described in the embodiments of this disclosure can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the unit itself.

[0285] On the other hand, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable storage medium carries one or more programs that, when executed by the electronic device, cause the electronic device to perform the methods described in the following embodiments. For example, the electronic device may perform... Figure 6 or Figure 8 The steps shown.

[0286] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this disclosure.

[0287] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0288] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for setting the search range for template matching, characterized in that, include: Obtain the target template features of the current block's template; Obtain the target region features of the reference region of the current block; Based on the target template features and the target region features, the target search range of the template is determined.

2. The method according to claim 1, characterized in that, Obtain the target template features of the current block's template, including: Obtain the value of at least one of the following features from the set of pixel values ​​in the template: mean, variance, gradient, and set position. The target template feature is constructed based on the value of at least one of the set of pixel values ​​in the template, namely the mean, variance, gradient, and so on.

3. The method according to claim 2, characterized in that, The set bit is a predetermined number of bits for the feature.

4. The method according to claim 2, characterized in that, The template includes a first region and / or a second region; the gradient includes a vertical gradient of the first region and / or a horizontal gradient of the second region.

5. The method according to claim 4, characterized in that, When the number of pixels in the first region is more than one column, the gradient also includes the horizontal gradient of the first region; When the second region has more than one row of pixels, the gradient also includes the vertical gradient of the second region.

6. The method according to claim 2, characterized in that, The target template features are constructed based on the values ​​of at least one of the following: mean, variance, gradient, and sum of the pixel values ​​in the template. The first part of the target template feature is determined by the value of the setting position of the mean of the set of pixel values ​​in the template, the second part of the feature is determined by the value of the setting position of the variance of the set of pixel values ​​in the template, and the third part of the feature is determined by the value of the setting position of the gradient sum of the set of pixel values ​​in the template.

7. The method according to claim 2, characterized in that, Obtaining the set value of at least one of the features, namely the mean, variance, gradient, and set position, from the set of pixel values ​​in the template, including: The set of pixel values ​​of at least one component signal in the template is obtained, along with the mean, variance, gradient, and the set value of at least one of the features.

8. The method according to claim 7, characterized in that, When the set value of at least one of the mean, variance, gradient, and other features of the pixel value set of multiple component signals in the template is obtained, the target template features are constructed based on the set value of at least one of the mean, variance, gradient, and other features of the pixel value set in the template, including: Template features for each component signal are constructed based on the set value of at least one of the mean, variance, gradient, and other values ​​of the pixel value set of each component signal in the template. The target template features are constructed based on the template features of each component signal.

9. The method according to claim 8, characterized in that, The target template features are constructed based on the template features of each component signal, including: The target template features are obtained by summing the template features of each component signal; or; The target template feature is obtained by concatenating the template features of each component signal.

10. The method according to any one of claims 1 to 9, characterized in that, Obtaining the target region features of the reference region of the current block includes: Within the reference area of ​​the current block, a matching template that matches the template is obtained; Obtain the matching template features of the matching template, wherein the target region features include the matching template features; The process of determining the target search range of the template based on the target template features and the target region features includes: The target template features are compared with the matching template features, and matching templates that do not match the target template features are removed to determine the target search range.

11. The method according to claim 10, characterized in that, Obtaining the matching template features of the matching template includes: Obtain the value of at least one of the following features from the set of pixel values ​​in the matching template: mean, variance, gradient, and set position. The matching template features are constructed based on the set value of at least one of the mean, variance, gradient, and other values ​​of the pixel value set in the matching template.

12. The method according to claim 10, characterized in that, Also includes: The target matching template is determined from the matching templates whose matching template features match the target template features; The predictive encoding of the current block is performed based on the reference block corresponding to the target matching template.

13. The method according to claim 1, characterized in that, Obtain the target template features of the current block's template, including: Determine the smallest unit; The template is divided into multiple smallest units; The target features of each smallest unit in the template are obtained respectively; The target template features are obtained based on the target features of each smallest unit in the template.

14. The method according to claim 13, characterized in that, Obtain the target features of each smallest unit in the template, including: Obtain the value of at least one of the following features from the set of pixel values ​​of each smallest unit in the template: mean, variance, gradient, and set position. The target feature of the corresponding smallest unit is constructed based on the set value of at least one of the mean, variance, gradient, and sum of the pixel value set of each smallest unit in the template.

15. The method according to claim 13 or 14, characterized in that, Obtaining the target region features of the reference region of the current block includes: The reference region of the current block is divided into multiple minimum units; The target features of each smallest unit within the reference region are obtained respectively; Within the reference area of ​​the current block, a matching template that matches the template is obtained; Based on the target features of each smallest unit within the reference region, the matching template features of the matching template are determined, wherein the target region features include the matching template features; The process of determining the target search range of the template based on the target template features and the target region features includes: The target template features are compared with the matching template features, and matching templates that do not match the target template features are removed to determine the target search range.

16. The method according to claim 1, characterized in that, Obtaining the target region features of the reference region of the current block includes: The reference region is downsampled. Feature extraction is performed on the downsampled reference region to construct the target region features.

17. The method according to claim 1, characterized in that, Obtaining the target region features of the reference region of the current block includes: The reference region is divided into multiple sub-regions; Based on the set of pixel values ​​in each sub-region, the sub-region features of each sub-region are obtained, and the target region features include the sub-region features of each sub-region. The process of determining the target search range of the template based on the target template features and the target region features includes: Each sub-region feature is compared with the target template feature, and sub-regions whose features do not match the target template feature are removed to determine the target search range of the template.

18. A template matching search range setting device, characterized in that, include: The processing unit is used to obtain the target template features of the current block's template. The processing unit is further configured to obtain the target region features of the reference region of the current block; The processing unit is further configured to determine the target search range of the template based on the target template features and the target region features.

19. An electronic device, characterized in that, include: At least one processor; A storage device configured to store at least one program, which, when executed by the at least one processor, causes the at least one processor to implement the method as described in any one of claims 1 to 17.

20. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on a computer, it causes the computer to perform the method of any one of claims 1 to 17.

Images