Video encoding method and apparatus, video decoding method and apparatus, computer-readable medium, and electronic device

Abstract

Provided are a video encoding method and apparatus, a video decoding method and apparatus, a computer-readable medium, and an electronic device. The video decoding method comprises: acquiring a video bitstream, wherein the video bitstream comprises a current block; determining a template for the current block, wherein the template comprises a decoded first region adjacent to the current block; acquiring a decoded second region adjacent to the current block; on the basis of pixel values in the second region, selecting from a preset interpolation filter group an interpolation filter for template matching; on the basis of the selected interpolation filter, determining a matching block matching the template; and on the basis of a first vector from the template to the matching block, determining a prediction block for decoding the current block. In the technical solution of the present application, the interpolation filter for a template matching process can be adaptively selected on the basis of the pixel values in the region adjacent to the current block, which helps to improve the precision of template matching, thereby improving the encoding and decoding efficiency.

Description

Video encoding and decoding methods, apparatuses, computer-readable media and electronic devices

[0001] This application claims priority to Chinese Patent Application No. 2025102421380, filed on February 27, 2025, entitled "Video Coding / Decoding Method, Apparatus, Computer-Readable Medium and Electronic Device", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the fields of computer and communication technology, and more specifically, to a video encoding / decoding method, apparatus, computer-readable medium, and electronic device. Background Technology

[0003] In video encoding and decoding, template matching is a process that searches for the optimal template matching position within a certain reference area and then uses the corresponding reference block for predictive coding. Whether it's intra-frame or inter-frame template matching, the reference position pointed to by the motion vector (for inter-frame template matching) or displacement vector (for intra-frame template matching) during template matching calculations is not necessarily an integer pixel position. Therefore, pixel interpolation is needed to generate the corresponding sub-pixel positions before performing template matching. In related technologies, the choice of interpolation filter is usually fixed. This fixed selection limits the accuracy of template matching, which in turn affects the video encoding and decoding performance. Summary of the Invention

[0004] Embodiments of this application provide a video encoding / decoding method, apparatus, computer-readable medium, and electronic device, which can adaptively select an interpolation filter for the template matching process based on the pixel values ​​in the neighboring regions of the current block, thereby improving the accuracy of template matching and thus enhancing encoding / decoding efficiency.

[0005] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part by practice of this application.

[0006] In a first aspect, embodiments of this application provide a video decoding method, including:

[0007] Acquire the video stream, which includes the current block;

[0008] A template is determined for the current block, the template including a first region that has been decoded adjacent to the current block;

[0009] Obtain the decoded second region adjacent to the current block;

[0010] Based on the pixel values ​​in the second region, an interpolation filter for template matching is selected from a preset interpolation filter group;

[0011] Based on the selected interpolation filter, determine the matching block that matches the template;

[0012] Based on the first vector from the template to the matching block, a prediction block for decoding the current block is determined.

[0013] Secondly, embodiments of this application provide a video encoding method, including:

[0014] Get the current block in the video;

[0015] A template is determined for the current block, the template including a first encoded region adjacent to the current block;

[0016] Obtain the encoded second region adjacent to the current block;

[0017] Based on the pixel values ​​in the second region, an interpolation filter for template matching is selected from a preset interpolation filter group;

[0018] Based on the selected interpolation filter, determine the matching block that matches the template;

[0019] Based on the first vector from the template to the matching block, a prediction block for encoding the current block is determined.

[0020] Thirdly, embodiments of this application provide a video decoding apparatus, including:

[0021] The acquisition unit is configured to acquire a video stream, the video stream including a current block; determine a template for the current block, the template including a first region that has been decoded adjacent to the current block; and acquire a second region that has been decoded adjacent to the current block.

[0022] The determining unit is configured to select an interpolation filter for template matching from a preset interpolation filter bank based on the pixel values ​​in the second region.

[0023] The processing unit is configured to determine a matching block that matches the template based on a selected interpolation filter; and to determine a prediction block for decoding the current block based on a first vector from the template to the matching block.

[0024] Fourthly, embodiments of this application provide a video encoding apparatus, including:

[0025] The acquisition unit is configured to acquire the current block in the video; determine a template for the current block, the template including a first encoded region adjacent to the current block; and acquire a second encoded region adjacent to the current block.

[0026] The determining unit is configured to select an interpolation filter for template matching from a preset interpolation filter bank based on the pixel values ​​in the second region.

[0027] The encoding unit is configured to determine a matching block that matches the template based on a selected interpolation filter; and to determine a prediction block for encoding the current block based on a first vector from the template to the matching block.

[0028] Fifthly, embodiments of this application provide a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the video decoding method or video encoding method as described in the above embodiments.

[0029] Sixthly, embodiments of this application provide an electronic device, including: one or more processors; and a storage device for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the video decoding method or video encoding method as described in the above embodiments.

[0030] In a seventh aspect, embodiments of this application provide a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of an electronic device reads from and executes the computer program from the computer-readable storage medium, causing the electronic device to perform the video decoding or video encoding methods provided in the various alternative embodiments described above.

[0031] Eighthly, embodiments of this application provide a method for storing a video stream, wherein the video stream is decoded according to the video decoding method described in the above embodiments, or the video stream is generated according to the video encoding method described in the above embodiments. Attached Figure Description

[0032] Figure 1 illustrates a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied;

[0033] Figure 2 shows a schematic diagram of the placement of the video encoding device and the video decoding device in the streaming system;

[0034] Figure 3 shows a basic flowchart of a video encoder;

[0035] Figure 4 shows a schematic diagram of an inter-frame prediction process;

[0036] Figure 5 shows a schematic diagram of an inter-frame prediction process;

[0037] Figure 6 illustrates a schematic diagram of an intra-frame template matching process;

[0038] Figure 7 illustrates a schematic diagram of an intra-frame template matching search process;

[0039] Figure 8 illustrates a schematic diagram of an inter-frame template matching process;

[0040] Figure 9 shows a schematic diagram of pixel positions appearing during template matching;

[0041] Figure 10 shows a flowchart of a video decoding method according to an embodiment of this application;

[0042] Figure 11 shows a flowchart of a video encoding method according to an embodiment of this application;

[0043] Figure 12 shows a block diagram of a video decoding apparatus according to an embodiment of this application;

[0044] Figure 13 shows a block diagram of a video encoding apparatus according to an embodiment of this application;

[0045] Figure 14 shows a schematic diagram of the structure of a computer system suitable for implementing the computer device of the present application. Detailed Implementation

[0046] Exemplary embodiments will now be described in a more comprehensive manner with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to these examples; rather, these embodiments are provided so that this application will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0047] Furthermore, the features, structures, or characteristics described in this application can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to provide a full understanding of the embodiments of this application. However, those skilled in the art will recognize that when implementing the technical solutions of this application, not all the detailed features in the embodiments may be used, one or more specific details may be omitted, or other methods, elements, devices, steps, etc., may be employed.

[0048] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0049] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0050] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0051] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0052] Figure 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied.

[0053] As shown in Figure 1, system architecture 100 includes multiple terminal devices that can communicate with each other via, for example, a network 150. For instance, system architecture 100 may include a first terminal device 110 and a second terminal device 120 interconnected via network 150. In the embodiment of Figure 1, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0054] For example, the first terminal device 110 can encode video data (e.g., a video image stream captured by the terminal device 110) to transmit it to the second terminal device 120 via the network 150. The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device 120 can receive the encoded video data from the network 150, decode the encoded video data to recover the video data, and display video images based on the recovered video data.

[0055] In one embodiment of this application, system architecture 100 may include a third terminal device 130 and a fourth terminal device 140 that perform bidirectional transmission of encoded video data, such as during a video conference. For bidirectional data transmission, each of the third terminal device 130 and the fourth terminal device 140 may encode video data (e.g., a video image stream captured by the terminal device) for transmission over network 150 to the other terminal device. Each of the third terminal device 130 and the fourth terminal device 140 may also receive encoded video data transmitted by the other terminal device, decode the encoded video data to recover the video data, and display the video images on an accessible display device based on the recovered video data.

[0056] In the embodiment shown in FIG1, the first terminal device 110, the second terminal device 120, the third terminal device 130 and the fourth terminal device 140 may be servers or terminals, but the principles disclosed in this application are not limited to these.

[0057] Servers can be standalone physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals can be smartphones, tablets, laptops, desktop computers, smart speakers, smart voice interaction devices, smartwatches, smart home appliances, in-vehicle terminals, aircraft, etc., but are not limited to these.

[0058] The network 150 shown in Figure 1 represents any number of networks, including, for example, wired and / or wireless communication networks, that transmit encoded video data between the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140. The communication network 150 may exchange data in circuit-switched and / or packet-switched channels. This network may include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of network 150 may be irrelevant to the operation of the disclosure herein.

[0059] In one embodiment of this application, Figure 2 illustrates the placement of the video encoding device and the video decoding device in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital television (TV), storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0060] The streaming system may include an acquisition subsystem 213, which may include a video source 201 such as a digital camera, which creates an uncompressed video image stream 202. In an embodiment, the video image stream 202 includes samples captured by a digital camera. The video image stream 202 is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data 204 (or encoded video bitstream 204). The video image stream 202 may be processed by an electronic device 220, which includes a video encoding device 203 coupled to the video source 201. The video encoding device 203 may include hardware, software, or a combination of hardware and software to implement or enforce aspects of the disclosed subject matter as described in more detail below. The encoded video data 204 (or encoded video bitstream 204) is depicted as a thin line to emphasize the lower data volume of the encoded video data 204 (or encoded video bitstream 204), which may be stored on a streaming server 205 for future use. One or more streaming client subsystems, such as client subsystems 206 and 208 in FIG. 2, can access streaming server 205 to retrieve copies 207 and 209 of encoded video data 204. Client subsystem 206 may include, for example, a video decoding device 210 in electronic device 230. Video decoding device 210 decodes the incoming copy 207 of the encoded video data and produces an output video picture stream 211 that can be displayed on display 212 (e.g., a screen) or another presentation device. In some streaming systems, the encoded video data 204, video data 207, and video data 209 (e.g., video stream) may be encoded according to certain video encoding / compression standards.

[0061] It should be noted that electronic devices 220 and 230 may include other components not shown in the figures. For example, electronic device 220 may include a video decoding device, and electronic device 230 may also include a video encoding device.

[0062] In one embodiment of this application, taking High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC) from international video coding standards, as well as the Chinese national video coding standard AVS, as examples, after an input video frame image, the video frame image is divided into several non-overlapping processing units according to a block size. Each processing unit performs a similar compression operation. This processing unit is called a Coding Tree Unit (CTU) or Largest Coding Unit (LCU). The CTU can be further subdivided into more refined units to obtain one or more basic Coding Units (CUs). The CU is the most basic element in a coding process.

[0063] In another embodiment, this processing unit can also be called a tile, which is a rectangular area of ​​a multimedia data frame that can be independently decoded and encoded. In the Alliance for Open Media Video 1 (AV1) standard, the tile can be further subdivided into one or more superblocks (SBs). The SB is the starting point for block partitioning and can be further divided into multiple subblocks. The superblocks are then further subdivided into one or more blocks. Each block is the most basic element in a coding process. Optionally, an SB can contain several blocks (Bs).

[0064] The above method of dividing video frame images can be called a block partition structure. The following introduces some concepts in the encoding process:

[0065] Predictive coding includes intra-frame prediction and inter-frame prediction. The original video signal is predicted from a selected reconstructed video signal to obtain a residual video signal. The encoder needs to decide which predictive coding mode to choose for the current coding unit (or coding block) and inform the decoder. Intra-frame prediction refers to the predicted signal coming from a region within the same image that has already been encoded and reconstructed; inter-frame prediction refers to the predicted signal coming from another encoded image (called a reference image) that is different from the current image.

[0066] Transform and Quantization: After the residual video signal undergoes transformation operations such as Discrete Fourier Transform (DFT) and Discrete Cosine Transform (DCT), the signal is transformed into the transform domain, and these are called transform coefficients. The transform coefficients are then subjected to lossy quantization, losing some information to make the quantized signal more suitable for compression. In some video coding standards, there may be more than one transform method to choose from. Therefore, the encoder needs to select one of the transform methods for the current coding unit (or coding block) and inform the decoder. The fineness of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a wider range of values ​​will be quantized into the same output, which usually leads to greater distortion and a lower bit rate. Conversely, a smaller QP value means that coefficients with a smaller range of values ​​will be quantized into the same output, which usually leads to less distortion and a higher bit rate.

[0067] Entropy coding, or statistical coding, involves statistically compressing the quantized transform-domain signal based on the frequency of each value, ultimately outputting a binary (0 or 1) compressed bitstream. Simultaneously, other information generated during encoding, such as the selected coding mode and motion vector data, also requires entropy coding to reduce the bit rate. Statistical coding is a lossless coding method that effectively reduces the bit rate required to represent the same signal. Common statistical coding methods include Variable Length Coding (VLC) and Content Adaptive Binary Arithmetic Coding (CABAC).

[0068] Context-Based Binary Arithmetic Coding (CABAC) primarily involves three steps: binarization, context modeling, and binary arithmetic coding. After binarizing the input syntax elements, the binary data can be encoded using either a regular coding mode or a bypass coding mode. The bypass coding mode eliminates the need to assign a specific probability model to each binary bit; the input binary bit bin value is directly encoded using a simple bypass encoder, thus accelerating the overall encoding and decoding speed. Generally, different syntax elements are not completely independent, and even identical syntax elements possess a certain degree of memory. Therefore, according to conditional entropy theory, using other encoded syntax elements for conditional coding can further improve coding performance compared to independent coding or memoryless coding. This encoded symbol information used as conditions is called the context. In the regular coding mode, the binary bits of the syntax elements sequentially enter the context modeler. The encoder assigns an appropriate probability model to each input binary bit based on the values ​​of previously encoded syntax elements or binary bits; this process is called context modeling. The context model corresponding to a grammatical element can be located using the context index increment (ctxIdxInc) and the context index start (ctxIdxStart). After the bin value and the assigned probability model are fed into the binary arithmetic encoder for encoding, the context model needs to be updated based on the bin value, which is the adaptive process in encoding.

[0069] Loop Filtering: The transformed and quantized signal undergoes inverse quantization, inverse transform, and prediction compensation to obtain a reconstructed image. Due to the effects of quantization, the reconstructed image differs from the original image in some aspects, resulting in distortion. Therefore, filtering operations can be performed on the reconstructed image, such as deblocking filters (DB), sample adaptive offset (SAO), or adaptive loop filters (ALF), to effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will serve as a reference for subsequent coded images to predict future image signals, the aforementioned filtering operations are also called loop filtering, i.e., filtering operations within the coding loop.

[0070] In one embodiment of this application, Figure 3 shows a basic flowchart of a video encoder, which is illustrated using intra-frame prediction as an example. The original image signal s... k[x,y] and the predicted image signal Perform the difference operation to obtain the residual signal u. k [x, y], residual signal u k After transformation and quantization, [x, y] is transformed to obtain quantization coefficients. These coefficients are then used to obtain the encoded bitstream through entropy encoding, and to obtain the reconstructed residual signal u′ through inverse quantization and inverse transform. k Predict the image signal [x, y]. With the reconstructed residual signal u′ k [x, y] superimposed to generate image signals Image signal On one hand, the signal is input to the intra-frame mode decision module and the intra-frame prediction module for intra-frame prediction processing; on the other hand, the reconstructed image signal s′ is output through loop filtering. k Reconstructing the image signal s′ from [x, y] k [x, y] can be used as a reference image for the next frame for motion estimation and motion compensation prediction. Then, based on the motion compensation prediction result s′ r [x+m x y+m y ] and intra-frame prediction results Obtain the predicted image signal for the next frame. And continue repeating the above process until the coding is complete.

[0071] Based on the above encoding process, at the decoding end, for each encoding unit (or encoding block), after acquiring the compressed bitstream (i.e., bitstream), entropy decoding is performed to obtain various mode information and quantization coefficients. Then, the quantization coefficients undergo inverse quantization and inverse transform processing to obtain the residual signal. On the other hand, based on the known encoding mode information, the prediction signal corresponding to the encoding unit (or encoding block) can be obtained. Then, the residual signal and the prediction signal are added together to obtain the reconstructed signal. The reconstructed signal then undergoes loop filtering and other operations to generate the final output signal.

[0072] In the field of coding technology, intra-frame prediction is a commonly used predictive coding technique. Intra-frame prediction is based on the spatial correlation of pixels in a video image, deriving the predicted value of the current coding block from adjacent coded regions. Inter-frame prediction utilizes the temporal correlation of video, using pixels from neighboring coded images to predict pixels in the current image, thereby effectively removing temporal redundancy and saving bits of coding residual data. As shown in Figure 4, P represents the current frame, Pr represents the reference frame, B represents the current block, and Br represents the reference block of B. The coordinates of B′ in the reference frame are the same as the coordinates of B in the current frame, and the coordinates of Br are (x... r y rThe coordinates of B′ are (x, y). The displacement between the current block and its reference block is called the motion vector (MV), where MV = (x, y). r -x, y r -y). In other words, inter-frame prediction refers to the process of searching for a reference block in neighboring encoded images (i.e., reference frames) based on the current block to be encoded in the current frame, with the aim of removing temporal redundancy in the video signal. As shown in Figure 5, the current block to be encoded in the current frame is searched for within a certain range in the reference frame (i.e., the search area formed by the search box (i.e., the reference area)) according to block matching criteria to obtain the best matching block. Optionally, commonly used block matching criteria in video coding include: minimum mean square error (MSE), sum of absolute differences (SAD), etc.

[0073] Intra-frame template matching is a type of intra-frame prediction technique that aims to find the region most similar to the current block within the coded region of the current frame. The block vector (BV) in intra-frame template matching is obtained through template matching, which involves comparing multiple candidate reference positions in the available reference region and selecting the position with the lowest template matching cost as the matching position. The candidate reference block at this matching position is the matching block of the template. As shown in Figure 6, the set of neighboring coded pixels of the current block forms a fixed template region (i.e., the template of the current block, also called the first region). This template region is matched with the region of the same shape corresponding to the candidate reference block of the current block at a certain position within the coded reference region (i.e., the reference block template region, also called the candidate reference block of the template), resulting in a template matching difference, which serves as the template matching cost. For example, the template difference can be calculated using the sum of the absolute values ​​of the differences between each pair of pixels in the template (SAD). The candidate reference block of the template with the lowest matching cost can be determined as the matching block of the template. The vector from the template to the matching block can be used to determine the predicted block of the current block. In other words, the vector from the current block to the predicted block is the same as the vector from the template to the matching block.

[0074] In intra-frame template matching, template matching calculations are performed within a reconstructed reference region agreed upon by both the codec and decoder. As shown in Figure 7, within the reference region, each movement of the template by one position (either an integer pixel or a fraction of a pixel) corresponds to a potential matching search position (i.e., a candidate reference position, or a candidate matching position). If the matching search position is at a fraction of a pixel, the pixel value of the matching search position is generated by interpolation from the reconstructed pixel values ​​of the adjacent regions corresponding to the matching search position.

[0075] Similar to intra-frame template matching, inter-frame template matching also searches for the optimal template matching position within a certain reference region using template matching methods, and then uses the corresponding prediction block for predictive coding. The difference is that the prediction block for inter-frame template matching comes from an already encoded reference image. Specifically, as shown in Figure 8, a template region (i.e., the template of the current block, also called the first region) is formed based on the neighboring encoded regions of the current block. This template is matched with the region of the same shape surrounding a reference block of the current block in the reference frame (i.e., the reference block template region, also called the candidate reference block of the template) to obtain a template matching difference. This difference serves as the cost of template matching. The goal of inter-frame template matching is to find the candidate motion vector with the minimum cost as the optimal motion vector for the current block. For example, the cost of template matching can be represented as the minimum mean square error (MSE) or the sum of absolute differences (SAD). For example, the candidate reference block of the template with the minimum matching cost can be determined as the matching block of the template. The vector from the template to the matching block can be used to determine the prediction block of the current block.

[0076] Whether it's intra-frame template matching or inter-frame template matching, the reference position pointed to by the motion vector (corresponding to inter-frame template matching) or displacement vector (corresponding to intra-frame template matching) during template matching calculation is not necessarily an integer pixel position. For example, referring to Figure 9, if the sub-pixel position P is between integer pixels B and C, then pixel interpolation is needed to generate the corresponding reference block template region pixels for the sub-pixel position before performing template matching calculation. Typically, two-tap (using the two nearest integer pixels at the sub-pixel position, one on each side) or four-tap (using the four nearest integer pixels at the sub-pixel position, two on each side) interpolation filters are considered.

[0077] Generally, the general form of an n-tap interpolation filter can be represented by the following formula 1:

[0078] Where, x i f represents the i-th encoded pixel used for interpolation filtering; p,i y represents the coefficients of the interpolation filter; p represents the pixel position; y represents the predicted value obtained by the interpolation filter; n represents the number of encoded pixels used.

[0079] For example, taking a 4-tap filter with a precision of 32 bits for interpolation filtering, the predicted value of a sub-pixel needs to be obtained by interpolating the four nearest pixels (as shown in Figure 9, the predicted value of point P is obtained by interpolating four integer pixels A, B, C, and D). The precision of the sub-pixel is 1 / 32, meaning that the integer pixels between B and C are divided into 32 equal parts, and each sub-pixel position is refined to 1 / 32 of its position. To avoid floating-point operations, the filter coefficients are amplified and rounded, and the predicted value is generated by interpolation filtering according to Formula 2.

[0080] The coefficients of the interpolation filter corresponding to pixel position p are shown in Table 1:

[0081] Table 1

[0082] In related technologies, the selection of interpolation filters is usually fixed. This fixed selection method limits the accuracy of template matching, which in turn affects the video encoding and decoding performance. Based on the above technical problems, this application proposes a new technical solution that adaptively selects the interpolation filter for template matching based on the pixel values ​​in the neighboring regions of the current block. This dynamic selection mechanism enables the more accurate generation of reference blocks for templates at pixel-level locations, thereby significantly improving the accuracy of template matching and enhancing encoding and decoding efficiency.

[0083] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0084] Figure 10 shows a flowchart of a video decoding method according to an embodiment of this application. This video decoding method can be executed by a device with computing processing capabilities, such as a terminal device or a server. Referring to Figure 10, the video decoding method includes at least steps S1010 to S1060, which are described in detail below:

[0085] In S1010, the video stream is acquired, which includes the current block.

[0086] In S1020, a template is determined for the current block, the template including a first region that has been decoded adjacent to the current block.

[0087] In S1030, a decoded second region adjacent to the current block is obtained. The first region may be the same as or different from the second region. In one embodiment, the second region includes the first region. In other words, the first region is equal to the second region, or a part of the second region.

[0088] In S1040, an interpolation filter for template matching is selected from a preset interpolation filter group based on the pixel values ​​in the second region.

[0089] In S1050, a matching block that matches the template is determined based on the selected interpolation filter.

[0090] In some embodiments, the matching block is the reference block with the lowest matching cost to the template selected from at least one candidate reference block.

[0091] In one embodiment, step S1050 includes: obtaining at least one candidate reference position for the template; calculating the quantile pixel value of the at least one candidate reference position using the selected interpolation filter to obtain at least one candidate reference block; and selecting the candidate reference block with the minimum matching cost from the at least one candidate reference block as the matching block. The matching cost between the template of the current block and the candidate reference block is, for example, the difference between the template of the current block and the candidate reference block calculated using a cost function. The cost function is a minimum mean square error function or an absolute error sum function, etc. In other words, the matching cost is, for example, the minimum mean square error or the absolute error sum.

[0092] In some embodiments, obtaining at least one candidate reference position for the template includes one of the following:

[0093] Obtain at least one candidate motion vector for the current block; based on the at least one candidate motion vector, determine at least one candidate reference position for the template from the reference frame;

[0094] Obtain at least one candidate displacement vector of the current block, and determine at least one candidate reference position of the template from the current frame based on the candidate displacement vector;

[0095] Determine the reference region corresponding to the template; select at least one candidate reference position from the reference region. The reference region may be, for example, a preset region in a reference frame or a decoded region of the current frame.

[0096] In S1060, a prediction block for decoding the current block is determined based on a first vector from the template to the matching block.

[0097] In some embodiments, the position of the predicted block can be obtained by calculating the sum of the position of the current block and the first vector.

[0098] In some embodiments, the reconstructed pixel value of the current block can be obtained by calculating the sum of the residuals of the predicted block and the current block obtained from the video bitstream.

[0099] In this embodiment, an interpolation filter for template matching can be selected based on the pixel values ​​in the neighboring regions of the current block, and then the selected interpolation filter can be used for template matching. This allows for adaptive selection of the interpolation filter. This dynamic selection mechanism enables more accurate generation of pixel values ​​at sub-pixel positions during the template matching process, thereby significantly improving the accuracy of template matching and enhancing encoding and decoding efficiency.

[0100] In some embodiments, the current block uses an inter-frame template matching mode, and the first vector is a first motion vector. In some embodiments, the current block uses an intra-frame template matching mode, and the first vector is a first block vector (also referred to as a first displacement vector).

[0101] In some embodiments, the present application can obtain the reference block template region corresponding to the current block using the template matching mode.

[0102] In some optional embodiments, the video contains a sequence of video image frames, which includes a series of images. Each image can be further divided into slices, and each slice can be further divided into a series of LCUs (or CTUs). Each LCU contains several CUs. Video image frames are encoded in blocks. In some newer video coding standards, such as H.264, there are macroblocks (MBs), which can be further divided into multiple prediction blocks for predictive coding. In the HEVC standard, basic concepts such as coding units (CUs), prediction units (PUs), and transform units (TUs) are used to functionally divide various block units, and a novel tree-based structure is used for description. For example, a CU can be divided into smaller CUs according to a quadtree, and these smaller CUs can be further divided to form a quadtree structure. In the embodiments of this application, the current block, reference block, etc., can be a CU, or a block smaller than a CU, such as a smaller block obtained by dividing a CU.

[0103] In some alternative embodiments, the current block can use either inter-frame template matching mode or intra-frame template matching mode.

[0104] Optionally, if the current block uses inter-frame template matching mode, then when determining the reference block template region (i.e., the matching block of the template) corresponding to the current block, at least one candidate motion vector corresponding to the current block can be obtained, and then at least one reference block template region can be determined in the reference frame based on the candidate motion vector. Optionally, the candidate motion vector corresponding to the current block can be selected from a list of motion vector candidates, or it can be directly decoded from the video bitstream.

[0105] Optionally, if the current block uses intra-frame template matching mode, then when determining the reference block template region corresponding to the current block, the candidate displacement vector corresponding to the current block can be obtained, and then the reference block template region can be determined in the reference region (i.e., the decoded region in the current frame where the current block is located) based on the candidate displacement vector. Optionally, the candidate displacement vector corresponding to the current block can be obtained from the video bitstream.

[0106] Optionally, when determining the reference block template region corresponding to the current block, a reference region corresponding to the current block can also be determined (if it is intra-frame coding, then the reference region corresponding to the current block is the decoded region in the current frame where the current block is located; if it is inter-frame coding, then the reference region corresponding to the current block is a preset region in the reference frame). Then, template matching search is performed within the reference region to obtain the corresponding reference block template region. The preset region is, for example, a rectangular region of a preset size centered on the co-occurring block of the current block.

[0107] In some embodiments, the present application embodiments may determine the filter type for interpolation filtering of the reference block template region based on the pixel values ​​in the neighboring region (i.e., the second region) of the current block.

[0108] In some optional embodiments, the neighboring region (i.e., the second region) of the current block is the region spatially adjacent to the current block. For example, it could be m rows of pixels above the current block and n columns of pixels to the left of the current block, where m and n can be the same or different. In one example, the neighboring region of the current block could be 2 rows of pixels above the current block and 2 columns of pixels to the left of the current block.

[0109] Optionally, the adjacent region of the current block and the template of the current block (i.e., the first region) can be the same region or different regions. For example, both the adjacent region of the current block and the template of the current block can be the two rows of pixels above the current block and the two columns of pixels to the left of the current block. Alternatively, the adjacent region of the current block can be the two rows of pixels above the current block and the two columns of pixels to the left of the current block; while the template of the current block can be the one row of pixels above the current block and the one column of pixels to the left of the current block.

[0110] In some optional embodiments, when determining the filter type for interpolation filtering of the reference block template region based on the pixel values ​​in the neighboring regions of the current block, statistical processing can be performed on the pixel values ​​in the neighboring regions of the current block to obtain statistical results of the pixel values ​​corresponding to the neighboring regions of the current block. Then, based on the correspondence between the statistical results of the pixel values ​​and the filter type, the filter type corresponding to the statistical results of the pixel values ​​is selected as the filter type for interpolation filtering of the reference block template region. In this embodiment, since the statistical results of the pixel values ​​corresponding to the neighboring regions of the current block can reflect the texture complexity of the current block (such as gradient magnitude, edge characteristics, etc.), the interpolation filter selected based on the statistical results can more accurately match the actual texture requirements of the current block, thereby improving the accuracy of the interpolation filtering process.

[0111] In some embodiments, selecting an interpolation filter for template matching from a preset interpolation filter group based on the pixel values ​​in the second region includes: performing statistics on the pixel values ​​in the second region to obtain statistical results of the pixel values ​​in the second region; wherein the statistical results are used to characterize the image features of the second region; and selecting an interpolation filter corresponding to the statistical results from the interpolation filter group based on a preset correspondence between the statistical results and the interpolation filters.

[0112] In some embodiments, the pixel values ​​within the second region are statistically analyzed to obtain statistical results of the pixel values ​​in the second region, including:

[0113] Based on the pixel values ​​within the second region, calculate the absolute gradient value at each pixel location within the second region; and determine the average of the absolute gradient values ​​at each pixel location within the second region as the statistical result.

[0114] In some embodiments, the pixel values ​​in the second region are statistically analyzed to obtain the statistical results of the pixel values ​​in the second region, including: calculating the absolute value of the difference between the pixel values ​​of adjacent pixels in the second region based on the pixel values ​​in the second region, so as to obtain the absolute value of the difference corresponding to each pixel in the second region; and determining the average value of the absolute values ​​of the difference corresponding to each pixel in the second region as the statistical result.

[0115] In some embodiments, selecting an interpolation filter corresponding to the statistical results from the interpolation filter bank based on a preset correspondence between statistical results and interpolation filters includes:

[0116] If the statistical result of the pixel value is greater than or equal to the set threshold, then the first interpolation filter is selected;

[0117] If the statistical result of the pixel value is less than the set threshold, then a second interpolation filter is selected, wherein the number of taps of the first interpolation filter is different from the number of taps of the second interpolation filter.

[0118] In some embodiments, the number of taps in the first interpolation filter is less than the number of taps in the second interpolation filter.

[0119] In some optional embodiments, when performing statistical processing on the pixel values ​​in the neighboring region (i.e., the second region) of the current block, the absolute gradient value at each pixel position in the neighboring region can be calculated based on the pixel values ​​in the neighboring region of the current block. Then, the average value of the absolute gradient values ​​corresponding to the neighboring region can be calculated based on the absolute gradient values ​​at each pixel position in the neighboring region, and the average value of the absolute gradient values ​​can be used as the statistical result of the pixel values.

[0120] Optionally, the absolute gradient value at each pixel location within the second region of the current block can be the absolute gradient value in the horizontal direction at each pixel location, the absolute gradient value in the vertical direction at each pixel location, or a combined value calculated based on the horizontal and vertical gradients at each pixel location. Here, the horizontal gradient Gx represents the change in pixel value in the horizontal direction; the vertical gradient Gy represents the change in pixel value in the vertical direction; the absolute value of the horizontal gradient can be represented as |Gx|, the absolute value of the vertical gradient can be represented as |Gy|, and the combined value calculated based on the horizontal and vertical gradients can be represented as |Gx|+|Gy|.

[0121] In some alternative embodiments, the Sobel operator can be used to compute the horizontal and vertical gradients, wherein the convolution kernel of the Sobel operator for the horizontal gradient can be represented as a kernel. x The convolution kernel of the Sobel operator in the vertical direction can be represented as a kernel. y By using a convolution kernel to perform a convolution operation, the gradient of each pixel within the second region of the current block can be calculated. Specifically, the kernel... x and Kernel y It can be represented as:

[0122] Alternatively, other operators can be used to calculate the horizontal and vertical gradients of each pixel within the second region of the current block.

[0123] In some optional embodiments, when performing statistical processing on the pixel values ​​in the second region of the current block, the absolute value of the pixel value difference between adjacent pixels in the second region can be calculated based on the pixel values ​​in the second region to obtain the absolute value of the difference corresponding to each pixel in the second region. Then, based on the absolute value of the difference corresponding to each pixel in the second region, the average value of the absolute value of the difference corresponding to the second region can be calculated, and the average value of the absolute value of the difference can be used as the statistical result of the pixel value.

[0124] Optionally, when calculating the pixel value difference between adjacent pixels in the second region, for row pixels in the second region, the pixel value of the left pixel can be subtracted from the pixel value of the right pixel in the adjacent pixels, and the result can be used as the pixel value difference corresponding to the left pixel. Similarly, for column pixels in the second region, the pixel value of the upper pixel can be subtracted from the pixel value of the lower pixel in the adjacent pixels, and the result can be used as the pixel value difference corresponding to the upper pixel. Similarly, the pixel value difference corresponding to the column pixel can be calculated.

[0125] In some optional embodiments, after calculating the statistical results of the pixel values ​​corresponding to the second region of the current block, the corresponding filter type can be selected based on the interval in which the statistical results of the pixel values ​​fall. For example, if the statistical results of the pixel values ​​fall within the first interval, then the filter type corresponding to the first region is selected; if the statistical results of the pixel values ​​fall within the second interval, then the filter type corresponding to the second region is selected; ...; if the statistical results of the pixel values ​​fall within the nth interval, then the filter type corresponding to the nth region is selected. Optionally, n can be an integer greater than or equal to 2.

[0126] Optionally, in a specific example of this application, if the statistical result of the pixel value is greater than or equal to a set threshold, a first filter type can be selected; if the statistical result of the pixel value is less than the set threshold, a second filter type can be selected. The number of taps for the first filter type is different from the number of taps for the second filter type. That is, in this embodiment, two intervals can be set: an interval greater than or equal to the set threshold and an interval less than the set threshold. Then, different filter types are selected based on the interval in which the statistical result of the pixel value falls.

[0127] In some optional embodiments, if the statistical result of the pixel values ​​can be the average of the absolute gradient values ​​or the average of the absolute difference values ​​as described in the preceding embodiments, then the number of taps for the first filter type can be less than the number of taps for the second filter type. In this example, when the statistical result of the pixel values ​​(such as the average of the absolute gradient values, the average of the absolute difference values, etc.) is large, the image details are more significant, and excessive interpolation may introduce blur or distortion. Therefore, a filter with a small number of taps can be selected to avoid over-smoothing and retain more original details. Conversely, when the statistical result of the pixel values ​​(such as the average of the absolute gradient values, the average of the absolute difference values, etc.) is small, the image details are fewer and relatively smoother. A filter with a large number of taps can be used to improve prediction accuracy. Therefore, the number of taps for the first filter type can be less than the number of taps for the second filter type. Optionally, the first filter type can be a filter with 2 taps, and the second filter type can be a filter with 4 taps.

[0128] In some optional embodiments, the threshold value can be set according to the magnitude of the quantization parameter. It should be noted that the quantization parameter QP is a key parameter in video coding used to control compression quality and bitrate; it determines the precision of quantization of image data during encoding. A larger QP value results in stronger image compression, greater loss of image details, and reduced image texture complexity; a smaller QP value results in weaker image compression, retains more image details, and has higher image texture complexity. Therefore, changes in the quantization parameter lead to changes in image texture complexity, necessitating dynamic adjustment of the threshold value to adapt to different encoding conditions.

[0129] Optionally, when the QP value is large, due to the significant loss of texture details after high image compression, the overall image tends to be smooth. In this case, even if the absolute value of the gradient or the absolute value of the pixel difference is large, it may only be due to noise or a small number of residual edges, rather than a truly complex texture. Therefore, the threshold value can be appropriately increased so that more areas are judged to use filters with fewer taps (such as 2-tap filters), i.e., simple interpolation filtering can meet the requirements.

[0130] When the QP value is small, the image retains more details and has higher texture complexity. At this time, the absolute value of the gradient or the absolute value of the pixel difference can more accurately reflect the texture features of the image. Therefore, the threshold can be appropriately reduced so that more areas can use filters with more taps (such as 4-tap filters). This allows for the capture of image details through more complex interpolation filtering.

[0131] Therefore, in an optional embodiment of this application, the value of the threshold can be positively correlated with the value of the quantization parameter. This mechanism of dynamically adjusting the threshold can flexibly select an appropriate operation strategy according to the actual content of the image and the encoding conditions, thereby achieving the best balance between encoding efficiency and prediction accuracy.

[0132] In some embodiments, the present application can use a filter corresponding to a determined filter type to perform interpolation filtering on the reference block template region, and perform template matching based on the result of the interpolation filtering. It should be noted that the interpolation filtering can be performed on the reference block template region only when it is at a fractional pixel position. That is, by using the selected interpolation filter, the fractional pixel value of the candidate reference position is calculated to determine the candidate reference block. Then, the candidate block with the lowest rate-distortion cost is selected from the candidate reference blocks and used as the matching block for template matching. In other words, the optimal template matching position (i.e., the matching block matching the template) is obtained solely through matching search. The specific process can be referred to the technical solution of the foregoing embodiments and will not be repeated here.

[0133] Figure 10 illustrates the technical solution of this application embodiment from the perspective of video decoding. The technical solution of this application embodiment will be further described below from the perspective of video encoding, in conjunction with Figure 11. The encoding method in Figure 11 corresponds to the decoding method in Figure 10.

[0134] Figure 11 shows a flowchart of a video encoding method according to an embodiment of this application. This video encoding method can be executed by a device with computing processing capabilities, such as a terminal device or a server. Referring to Figure 11, the video encoding method includes at least steps S1110 to S1160, which are described in detail below:

[0135] In S1110, retrieve the current block in the video;

[0136] In S1120, a template is determined for the current block, the template including an encoded first region adjacent to the current block;

[0137] In S1130, the encoded second region adjacent to the current block is obtained;

[0138] In S1140, an interpolation filter for template matching is selected from a preset interpolation filter group based on the pixel values ​​in the second region.

[0139] In S1150, a matching block that matches the template is determined based on the selected interpolation filter;

[0140] In S1160, a prediction block for encoding the current block is determined based on a first vector from the template to the matching block.

[0141] In this embodiment, an interpolation filter for template matching can be selected based on the pixel values ​​in the neighboring regions of the current block, and then the selected interpolation filter can be used for template matching. This allows for adaptive selection of the interpolation filter. This dynamic selection mechanism enables more accurate generation of pixel values ​​at sub-pixel positions during the template matching process, thereby significantly improving the accuracy of template matching and enhancing encoding and decoding efficiency.

[0142] In some embodiments, selecting an interpolation filter for template matching from a preset interpolation filter set based on pixel values ​​within the second region includes:

[0143] The pixel values ​​within the second region are statistically analyzed to obtain statistical results of the pixel values ​​in the second region; wherein the statistical results are used to characterize the image features of the second region.

[0144] Based on the preset correspondence between statistical results and interpolation filters, an interpolation filter corresponding to the statistical results is selected from the interpolation filter group.

[0145] In some embodiments, the pixel values ​​within the second region are statistically analyzed to obtain statistical results of the pixel values ​​in the second region, including:

[0146] Based on the pixel values ​​within the second region, calculate the absolute value of the gradient at each pixel location within the second region;

[0147] The average of the absolute values ​​of the gradients at each pixel location within the second region is determined as the statistical result.

[0148] In some embodiments, the pixel values ​​within the second region are statistically analyzed to obtain statistical results of the pixel values ​​in the second region, including:

[0149] Based on the pixel values ​​in the second region, calculate the absolute value of the pixel value difference between adjacent pixels in the second region to obtain the absolute value of the difference corresponding to each pixel in the second region;

[0150] The average of the absolute values ​​of the differences between each pixel in the second region is determined as the statistical result.

[0151] In some embodiments, selecting an interpolation filter corresponding to the statistical results from the interpolation filter bank based on a preset correspondence between statistical results and interpolation filters includes:

[0152] If the pixel value statistics result is greater than or equal to the set threshold, then the first interpolation filter is selected;

[0153] If the pixel value statistics result is less than the set threshold, then a second interpolation filter is selected, wherein the number of taps of the first interpolation filter is different from the number of taps of the second interpolation filter.

[0154] In some embodiments, the number of taps in the first interpolation filter is less than the number of taps in the second interpolation filter.

[0155] In some embodiments, the size of the set threshold is associated with the size of the quantization parameter.

[0156] In some embodiments, the value of the set threshold is positively correlated with the value of the quantization parameter.

[0157] In some embodiments, determining a matching block that matches the template based on a selected interpolation filter includes:

[0158] Obtain at least one candidate reference block for the template;

[0159] Using the selected interpolation filter, calculate the quantile pixel values ​​of the at least one candidate reference block;

[0160] Based on the quantile pixel values ​​of the at least one candidate reference block, the candidate reference block with the lowest matching cost is selected from the at least one candidate reference block and used as the matching block.

[0161] In some embodiments, obtaining at least one candidate reference block for the template includes one of the following:

[0162] Obtain at least one candidate motion vector for the current block; based on the at least one candidate motion vector, determine at least one candidate reference block for the template from the reference frame;

[0163] Obtain at least one candidate displacement vector of the current block, and determine at least one candidate reference block of the template from the current frame based on the candidate displacement vector;

[0164] Determine the reference region corresponding to the template; select at least one candidate reference block from the reference region.

[0165] In some embodiments, the second region may be the same as or different from the first region.

[0166] In some optional embodiments, the encoder may also add at least one flag bit to the video bitstream, the value of which indicates the selected interpolation filter. This at least one flag bit may include one or more of the following: flag bits contained in the sequence header, flag bits contained in the image header, flag bits contained in the strip header, flag bits contained in the coding tree unit (CTU) header, and flag bits contained in the coding block.

[0167] Specifically, for example, the encoder can use a flag bit for a coded block in a CTU to indicate the filter type used by the coded block in the CTU during template matching.

[0168] It should be noted that the processing at the video encoding end is similar to that at the video decoding end. For details, please refer to the aforementioned processing at the decoding end, which will not be repeated here.

[0169] The technical solution of this application mainly considers that in the template matching process, the selection of interpolation filter is usually fixed. This fixed selection method limits the accuracy of template matching, which in turn affects the encoding and decoding performance of video. Therefore, it proposes to adaptively select the filter type for interpolation filtering of the reference block template region based on the pixel values ​​in the neighboring region of the current block, thereby significantly improving the accuracy of template matching and improving encoding and decoding efficiency.

[0170] In a specific application scenario of this application, a set of interpolation filters can be defined for the template matching process, such as a 2-tap filter and a 4-tap filter. Simultaneously, the motion vector (or displacement vector) for template matching of the current block can be determined by decoding the video bitrate, or a specific location for template matching search can be determined, pointing to the reference image and reference block used for prediction.

[0171] Optionally, the motion vector can be unidirectional (i.e., obtaining motion information from only one reference frame), bidirectional (i.e., obtaining motion information from two reference frames simultaneously), or multidirectional (i.e., obtaining motion information from multiple reference frames simultaneously).

[0172] Optionally, the motion vector applies to the entire current block, rather than being derived independently for each sub-block within the current block.

[0173] In some alternative embodiments, the set of pixels in the second region of the current block on which the type of interpolation filter is determined may be the same as or different from the set of pixels in the defined template region. For example, the set of pixels in the determined second region of the current block may include the top two rows of pixels and the left two columns of pixels in the current block, while the set of pixels in the template region of the current block may include the top one row of pixels and the left one column of pixels in the current block.

[0174] In some optional embodiments, the type of interpolation filter is determined by comparing and analyzing the pixel set of the second region of the current block. For example, gradients can be calculated for the pixels in the second region of the current block, and the average absolute value of the gradient at each pixel position can be used to determine whether a 4-tap filter should be used. Optionally, if the average value exceeds a certain threshold, a 2-tap filter is used; otherwise, a 4-tap filter is used. Optionally, this threshold can take into account the quantization parameters used in the encoding, and different thresholds can be set for different quantization parameters.

[0175] For example, the difference between pixels in the second region of the current block can be calculated row by row / column by column. Based on the average absolute value of the pixel differences at each pixel location, it can be determined whether to use a 4-tap filter. Optionally, if the average value exceeds a certain threshold, a 2-tap filter is used; otherwise, a 4-tap filter is used. Optionally, this threshold can take into account the quantization parameters used in the encoding, setting different thresholds for different quantization parameters.

[0176] In some alternative embodiments, the type of interpolation filter selected can be identified for a coding region of a specified size. For example, a flag bit can be used for each coding block in a CTU to indicate whether a 4-tap or 2-tap filter is used for interpolation filtering during template matching.

[0177] In some optional embodiments, once the type of interpolation filter is determined, template matching can be performed using the selected interpolation filter to obtain the difference result of the template matching. This difference result serves as the cost of template matching, thereby finding a motion vector or displacement vector with the minimum cost as the optimal motion vector or optimal displacement vector for the current block.

[0178] It should be noted that the technical solutions of the above embodiments of this application can improve prediction accuracy and coding efficiency by adaptively selecting the interpolation filter used for template matching. The technical solutions of the various embodiments shown above can be used individually or in combination. Furthermore, the technical solutions of the embodiments of this application can be applied to video codecs or video compression and related products.

[0179] The following describes an apparatus embodiment of this application, which can be used to perform the methods described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments described above.

[0180] Figure 12 shows a block diagram of a video decoding apparatus according to an embodiment of the present application. The video decoding apparatus can be installed in a device with computing processing capabilities, such as a terminal device or a server.

[0181] Referring to FIG12, a video decoding apparatus 1200 according to an embodiment of the present application includes: an acquisition unit 1202, a determination unit 1204, and a processing unit 1206.

[0182] The acquisition unit 1202 is configured to acquire a video stream, the video stream including a current block; determine a template for the current block, the template including a first region that has been decoded adjacent to the current block; and acquire a second region that has been decoded adjacent to the current block.

[0183] The determining unit 1204 is configured to select an interpolation filter for template matching from a preset interpolation filter bank based on the pixel values ​​in the second region.

[0184] Processing unit 1206 is configured to determine a matching block that matches the template based on a selected interpolation filter; and to determine a prediction block for decoding the current block based on a first vector from the template to the matching block.

[0185] In this embodiment, an interpolation filter for template matching can be selected based on the pixel values ​​in the neighboring regions of the current block, and then the selected interpolation filter can be used for template matching. This allows for adaptive selection of the interpolation filter. This dynamic selection mechanism enables more accurate generation of pixel values ​​at sub-pixel positions during the template matching process, thereby significantly improving the accuracy of template matching and enhancing encoding and decoding efficiency.

[0186] Figure 13 shows a block diagram of a video encoding apparatus according to an embodiment of the present application. The video encoding apparatus can be installed in a device with computing processing capabilities, such as a terminal device or a server.

[0187] Referring to FIG13, a video encoding apparatus 1300 according to an embodiment of the present application includes: an acquisition unit 1302, a determination unit 1304 and an encoding unit 1306.

[0188] The acquisition unit 1302 is configured to acquire the current block in the video; determine a template for the current block, the template including a first region that has been decoded adjacent to the current block; and acquire a second region that has been encoded adjacent to the current block.

[0189] The determining unit 1304 is configured to select an interpolation filter for template matching from a preset interpolation filter bank based on the pixel values ​​in the second region.

[0190] The encoding unit 1306 is configured to determine a matching block that matches the template based on a selected interpolation filter; and to determine a prediction block for encoding the current block based on a first vector from the template to the matching block.

[0191] Figure 14 shows a schematic diagram of the structure of a computer system suitable for implementing the computer device of the present application. The electronic device may be a video encoding device or a video decoding device in the foregoing embodiments.

[0192] It should be noted that the computer system 1400 of the computer device shown in Figure 14 is only an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0193] As shown in Figure 14, the computer system 1400 may include a Central Processing Unit (CPU) 1401, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1402 or programs loaded from storage portion 1408 into Random Access Memory (RAM) 1403, such as performing the methods described in the above embodiments. The RAM 1403 also stores various programs and data required for system operation. The CPU 1401, ROM 1402, and RAM 1403 are interconnected via a bus 1404. An Input / Output (I / O) interface 1405 is also connected to the bus 1404.

[0194] The following components can be connected to I / O interface 1405: an input section 1406 including a keyboard, mouse, etc.; an output section 1407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1408 including a hard disk, etc.; and a communication section 1409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to I / O interface 1405 as needed. Removable media 1411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1410 as needed so that computer programs read from them can be installed into storage section 1408 as needed.

[0195] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1409, and / or installed from removable medium 1411. When the computer program is executed by central processing unit (CPU) 1401, it performs various functions defined in the system of this application.

[0196] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a computer program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0197] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and a computer program.

[0198] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0199] In another aspect, this application also provides a computer-readable medium, which may be included in the computer device described in the above embodiments; or it may exist independently and not assembled into the computer device. The computer-readable medium carries one or more computer programs, which, when executed by the computer device, cause the computer device to perform the methods described in the above embodiments.

[0200] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0201] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, and includes several instructions to cause a computer device to execute the method according to the embodiments of this application.

[0202] For example, an electronic device can be a video decoding device, in which case the video decoding device can execute the video decoding method shown in Figure 10; or, for instance, an electronic device can be a video encoding device, in which case the video encoding device can execute the video encoding method shown in Figure 11.

[0203] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0204] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A video decoding method, executed in an electronic device, the method comprising: Acquire the video stream, which includes the current block; A template is determined for the current block, the template including a first region that has been decoded adjacent to the current block; Obtain the decoded second region adjacent to the current block; Based on the pixel values ​​in the second region, an interpolation filter for template matching is selected from a preset interpolation filter group; Based on the selected interpolation filter, determine the matching block that matches the template; Based on the first vector from the template to the matching block, a prediction block for decoding the current block is determined.

2. The video decoding method according to claim 1, characterized in that, Based on the pixel values ​​within the second region, an interpolation filter for template matching is selected from a preset interpolation filter bank, including: The pixel values ​​within the second region are statistically analyzed to obtain statistical results of the pixel values ​​in the second region; wherein the statistical results are used to characterize the image features of the second region. Based on the preset correspondence between statistical results and interpolation filters, an interpolation filter corresponding to the statistical results is selected from the interpolation filter group.

3. The video decoding method according to claim 2, characterized in that, The pixel values ​​within the second region are statistically analyzed to obtain the statistical results of the pixel values ​​in the second region, including: Based on the pixel values ​​within the second region, calculate the absolute value of the gradient at each pixel location within the second region; The average of the absolute values ​​of the gradients at each pixel location within the second region is determined as the statistical result.

4. The video decoding method according to claim 2 or 3, characterized in that, The pixel values ​​within the second region are statistically analyzed to obtain the statistical results of the pixel values ​​in the second region, including: Based on the pixel values ​​in the second region, calculate the absolute value of the pixel value difference between adjacent pixels in the second region to obtain the absolute value of the difference corresponding to each pixel in the second region; The average of the absolute values ​​of the differences between each pixel in the second region is determined as the statistical result.

5. The video decoding method according to any one of claims 2-4, characterized in that, Based on the preset correspondence between statistical results and interpolation filters, an interpolation filter corresponding to the statistical results is selected from the interpolation filter bank, including: If the pixel value statistics result is greater than or equal to the set threshold, then the first interpolation filter is selected; If the pixel value statistics result is less than the set threshold, then a second interpolation filter is selected, wherein the number of taps of the first interpolation filter is different from the number of taps of the second interpolation filter.

6. The video decoding method according to claim 5, characterized in that, The number of taps in the first interpolation filter is less than the number of taps in the second interpolation filter.

7. The video decoding method according to claim 5 or 6, characterized in that, The value of the set threshold is related to the value of the quantization parameter.

8. The video decoding method according to any one of claims 5-7, characterized in that, The value of the set threshold is positively correlated with the value of the quantization parameter.

9. The video decoding method according to any one of claims 1-8, characterized in that, Based on the selected interpolation filter, determining the matching block that matches the template includes: Obtain at least one candidate reference position for the template; Using the selected interpolation filter, the quantile pixel values ​​of the at least one candidate reference position are calculated to obtain at least one candidate reference block; Based on the at least one candidate reference block, select the candidate reference block with the lowest matching cost from the at least one candidate reference block and use it as the matching block.

10. The video decoding method according to claim 9, characterized in that, Obtain at least one candidate reference position for the template, including one of the following: Obtain at least one candidate motion vector for the current block; based on the at least one candidate motion vector, determine at least one candidate reference position for the template from the reference frame; Obtain at least one candidate displacement vector of the current block, and determine at least one candidate reference position of the template from the current frame based on the candidate displacement vector; Determine the reference area corresponding to the template; Select at least one candidate reference location from the reference region.

11. The video decoding method according to any one of claims 1 to 10, characterized in that, The second region may be the same as or different from the first region.

12. A video encoding method, characterized in that, include: Get the current block in the video; A template is determined for the current block, the template including a first encoded region adjacent to the current block; Obtain the encoded second region adjacent to the current block; Based on the pixel values ​​in the second region, an interpolation filter for template matching is selected from a preset interpolation filter group; Based on the selected interpolation filter, determine the matching block that matches the template; Based on a first vector from the template to the reference block, a prediction block for encoding the current block is determined.

13. The video encoding method according to claim 12, characterized in that, Also includes: Add at least one flag bit to the video bitstream, the value of which is used to indicate the selected interpolation filter, and the at least one flag bit includes one or more of the following flag bits: flag bits contained in the sequence header, flag bits contained in the image header, flag bits contained in the strip header, flag bits contained in the coding tree unit (CTU) header, and flag bits contained in the coding block.

14. A video decoding device, characterized in that, include: An acquisition unit is configured to acquire a video stream, the video stream including a current block; and to determine a template for the current block, the template including a first region that has been decoded adjacent to the current block. Obtain the decoded second region adjacent to the current block; The determining unit is configured to select an interpolation filter for template matching from a preset interpolation filter bank based on the pixel values ​​in the second region. The processing unit is configured to determine a matching block that matches the template based on a selected interpolation filter; Based on the first vector from the template to the matching block, a prediction block for decoding the current block is determined.

15. A video encoding device, characterized in that, include: The acquisition unit is configured to acquire the current block in the video; A template is determined for the current block, the template including a first encoded region adjacent to the current block; Obtain the encoded second region adjacent to the current block; The determining unit is configured to select an interpolation filter for template matching from a preset interpolation filter bank based on the pixel values ​​in the second region. The encoding unit is configured to determine a matching block that matches the template based on a selected interpolation filter; Based on the first vector from the template to the matching block, a prediction block for encoding the current block is determined.

16. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 13.

17. A computer device, characterized in that, include: One or more processors; A memory for storing one or more computer programs that, when executed by one or more processors, cause the computer device to perform the method of any one of claims 1 to 13.

18. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, wherein a processor of a computer device reads from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method of any one of claims 1 to 13.

19. A method for storing video streams, characterized in that, The video stream is decoded using the video decoding method according to any one of claims 1 to 11, or the video stream is generated using the video encoding method according to claim 12 or 13.

Images