Video coding method and apparatus, video decoding method and apparatus, and computer-readable medium and electronic device

Abstract

Provided are a video coding method and apparatus, a video decoding method and apparatus, and a computer-readable medium and an electronic device. The video decoding method comprises: receiving a video bitstream; on the basis of the video bitstream, determining at least one candidate motion vector of a current block; constructing a template of the current block; on the basis of the at least one candidate motion vector, determining at least one reference block of the current block in a reference frame; on the basis of the position of each reference block among the at least one reference block, determining in the reference frame a template prediction area corresponding to each reference block; on the basis of a pixel value in the template prediction area corresponding to each reference block, determining a pixel value in a first interpolation extension area; and on the basis of the pixel value of the first interpolation extension area, determining, by means of interpolation filtering processing, sub-pixel interpolation of the corresponding template prediction area. By means of the embodiments of the present application, it is not necessary to decode a pixel value in an interpolation extension area from a bitstream, thereby reducing bandwidth overheads of hardware implementation, and thus facilitating an improvement in the efficiency of coding and decoding.

Description

Video coding method and device, computer readable medium and electronic device

[0001] This application claims priority to the Chinese patent application No. 202410743791.0, filed on June 7, 2024, and entitled "Video coding method and device, computer readable medium and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of computer and communication technology, in particular, to a video coding method and device, computer readable medium and electronic device. BACKGROUND

[0003] Inter-frame template matching (TM) is an important inter-frame prediction technology in the field of video coding, aiming to further refine the motion vector of the current block based on template matching technology, so as to obtain a more accurate prediction result. If the template prediction area in the reference frame is sub-pixel accuracy, interpolation filtering processing needs to be performed, and when performing interpolation filtering processing, pixels in the interpolation expansion area need to be read, which will introduce a large bandwidth overhead in hardware implementation. The high hardware overhead limits the practical application of the coding technology. SUMMARY

[0004] Embodiments of the present application provide a video coding method and device, computer readable medium and electronic device, which can use the pixel values in the template prediction area to determine the pixel values in the first interpolation expansion area used for interpolation filtering processing in inter-frame template matching. The decoding end does not need to decode the pixel values in the first interpolation expansion area from the code stream, which reduces the bandwidth overhead in hardware implementation, thereby facilitating the improvement of the coding efficiency of the video.

[0005] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0006] In a first aspect, embodiments of the present application provide a video decoding method, comprising:

[0007] receiving a video code stream;

[0008] determining at least one candidate motion vector for inter-frame template matching of a current block based on the video code stream;

[0009] constructing a template of the current block, the template comprising reconstructed pixel points around the current block;

[0010] determining at least one reference block of the current block in a reference frame according to the at least one candidate motion vector;

[0011] based on a position of each of the at least one reference block, determine a template prediction region corresponding to each reference block in the reference frame, each template prediction region having a same position relative to the corresponding reference block as the template has relative to the current block;

[0012] based on pixel values within each template prediction region, determine pixel values within a first interpolation expansion region for performing interpolation filtering on the corresponding template prediction region;

[0013] based on the pixel values of the first interpolation expansion region, determine sub-pixel interpolation of the corresponding template prediction region by interpolation filtering as a prediction value of the template corresponding to the corresponding template prediction region.

[0014] In a second aspect, an embodiment of the present application provides a video encoding method, comprising:

[0015] based on the video data, determine at least one candidate motion vector for inter-frame template matching of the current block;

[0016] construct a template of the current block, the template comprising pixel points around the current block;

[0017] based on the at least one candidate motion vector, determine at least one reference block of the current block in a reference frame;

[0018] based on a position of each of the at least one reference block, determine a template prediction region corresponding to each reference block in the reference frame, each template prediction region having a same position relative to the corresponding reference block as the template has relative to the current block;

[0019] based on pixel values within each template prediction region, determine pixel values within a first interpolation expansion region for performing interpolation filtering on the corresponding template prediction region;

[0020] based on the pixel values of the first interpolation expansion region, determine sub-pixel interpolation of the corresponding template prediction region by interpolation filtering as a prediction value of the template corresponding to the corresponding template prediction region.

[0021] In a third aspect, an embodiment of the present application provides a video decoding apparatus, comprising:

[0022] a decoding unit, configured to receive a video bitstream; based on the video bitstream, determine at least one candidate motion vector for inter-frame template matching of a current block; construct a template of the current block, the template comprising reconstructed pixel points around the current block;

[0023] The reference block determination unit is configured to determine at least one reference block of the current block in a reference frame according to the at least one candidate motion vector.

[0024] The pixel value determination unit is configured to determine a template prediction region corresponding to each reference block in the reference frame based on the position of each reference block in the at least one reference block, each template prediction region having the same position relative to the corresponding reference block as the template has relative to the current block; and determine pixel values in a first interpolation expansion region for performing interpolation filtering processing on the corresponding template prediction region according to pixel values in the template prediction region corresponding to each reference block.

[0025] The processing unit is configured to determine sub-pixel interpolation of the corresponding template prediction region by interpolation filtering processing based on the pixel values in the first interpolation expansion region, as a prediction value of the template corresponding to the corresponding template prediction region.

[0026] In a fourth aspect, an embodiment of the present application provides a video encoding apparatus, comprising:

[0027] The vector determination unit is configured to obtain video data, the video data comprising a current block; determine at least one candidate motion vector for inter-frame template matching of the current block based on the video data; and construct a template of the current block, the template comprising pixel points around the current block.

[0028] The reference block determination unit is configured to determine at least one reference block of the current block in a reference frame according to the at least one candidate motion vector.

[0029] The pixel value determination unit is configured to determine at least one reference block of the current block in a reference frame according to the at least one candidate motion vector; determine a template prediction region corresponding to each reference block in the reference frame based on the position of each reference block in the at least one reference block, each template prediction region having the same position relative to the corresponding reference block as the template has relative to the current block; and determine pixel values in a first interpolation expansion region for performing interpolation filtering processing on the corresponding template prediction region according to pixel values in the template prediction region corresponding to each reference block.

[0030] The processing unit is configured to determine sub-pixel interpolation of the corresponding template prediction region by interpolation filtering processing based on the pixel values in the first interpolation expansion region, as a prediction value of the template corresponding to the corresponding template prediction region.

[0031] In a fifth aspect, an embodiment of the present application provides a computer readable medium having a computer program stored thereon, the computer program being executed by a processor to implement the video decoding method or the video encoding method as described in the above embodiments.

[0032] In a sixth aspect, an electronic device is provided, which includes one or more processors, and a storage device configured to store one or more computer programs. When the one or more computer programs are executed by the one or more processors, the electronic device is caused to implement the video decoding method or the video encoding method as described in the above embodiments.

[0033] In a seventh aspect, a computer program product is provided, which includes a computer program stored in a computer readable storage medium. A processor of an electronic device reads and executes the computer program from the computer readable storage medium, so that the electronic device performs the video decoding method or the video encoding method provided in the various optional embodiments described above. BRIEF DESCRIPTION OF DRAWINGS

[0034] FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied;

[0035] FIG. 2 shows a schematic diagram of the placement of a video encoding device and a video decoding device in a streaming system;

[0036] FIG. 3 shows a basic flowchart of a video encoder;

[0037] FIG. 4 shows a schematic diagram of an angular prediction direction in an intra prediction mode;

[0038] FIG. 5 shows a schematic diagram of intra prediction;

[0039] FIG. 6 shows a schematic diagram of an inter prediction process;

[0040] FIG. 7 shows a schematic diagram of an inter prediction process;

[0041] FIG. 8 shows a schematic diagram of a diamond search;

[0042] FIG. 9 shows a schematic diagram of a pixel position distribution at sub-pixel accuracy;

[0043] FIG. 10 shows a schematic diagram of an interpolation range of an interpolation filter;

[0044] FIG. 11 shows a schematic diagram of a process of inter template matching;

[0045] FIG. 12 shows an example diagram of a coarse search and a fine search;

[0046] FIG. 13 shows a schematic diagram of a search region and an interpolation expansion region;

[0047] FIG. 14 shows a flowchart of a video decoding method according to an embodiment of the present application;

[0048] FIG. 15 shows a diagram of a first interpolation expansion region according to an embodiment of the present application;

[0049] FIG. 16 shows a diagram of pixel value setting in the first interpolation expansion region according to an embodiment of the present application;

[0050] FIG. 17 shows a diagram of pixel value setting in the first interpolation expansion region according to an embodiment of the present application;

[0051] FIG. 18 shows a diagram of pixel value setting in the first interpolation expansion region according to an embodiment of the present application;

[0052] FIG. 19 shows a diagram of a search region according to an embodiment of the present application;

[0053] FIG. 20 shows a diagram of a second interpolation expansion region according to an embodiment of the present application;

[0054] FIG. 21 shows a flowchart of a video encoding method according to an embodiment of the present application;

[0055] FIG. 22 shows a block diagram of a video decoding apparatus according to an embodiment of the present application;

[0056] FIG. 23 shows a block diagram of a video encoding apparatus according to an embodiment of the present application;

[0057] FIG. 24 shows a structural diagram of a computer system of an electronic device suitable for implementing embodiments of the present application. DETAILED DESCRIPTION

[0058] Example implementations are now described with reference to the drawings; however, these descriptions are not intended to limit the scope of the application, but are merely intended to provide example examples for the purposes of illustration. The details described herein can be varied substantially, and the examples can be practiced in various forms and implementations, which are not limited to the examples described herein.

[0059] Moreover, features, structures or characteristics described in connection with the present application can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are recited in order to provide a thorough understanding of the embodiments of the present application. One skilled in the relevant art, however, will recognize that the embodiments of the present application can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth.

[0060] In embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0061] The block diagrams illustrated in the accompanying drawings are merely functional entities, and do not necessarily have to correspond to physically independent entities. That is, the functional entities can be implemented in the form of software, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0062] The flowcharts illustrated in the accompanying drawings are merely exemplary illustrations, and do not necessarily include all contents and operations / steps, nor do they have to be executed in the order described. For example, some operations / steps can be further decomposed, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to actual conditions.

[0063] It should be noted that "multiple" referred to herein means two or more. The association relationship of "and / or" describes the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0064] FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of embodiments of the present application can be applied.

[0065] As shown in FIG. 1, the system architecture 100 includes a plurality of terminal devices that can communicate with each other via, for example, a network 150. For example, the system architecture 100 can include a first terminal device 110 and a second terminal device 120 interconnected via the network 150. In the embodiment of FIG. 1, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0066] For example, the first terminal device 110 can code video data (for example, a stream of video pictures that are captured by the terminal device 110) for transmission to the second terminal device 120 via the network 150, the coded video data being transmitted in the form of one or more coded video bitstreams, and the second terminal device 120 can receive the coded video data from the network 150, decode the coded video data to recover the video data, and display video pictures based on the recovered video data.

[0067] In an embodiment of the present application, the system architecture 100 can include a third terminal device 130 and a fourth terminal device 140 that perform bidirectional transmission of encoded video data that can occur, for example, during a video conference. For bidirectional transmission of data, each terminal device of the third and fourth terminal devices 130, 140 can code video data (e.g., a stream of video pictures that are captured by the terminal device) for transmission to the other terminal device of the third and fourth terminal devices 130, 140 by way of the network 150. Each of the third and fourth terminal devices 130, 140 also can receive the encoded video data transmitted by the other terminal device of the third and fourth terminal devices 130, 140, and can decode the encoded video data to recover the video data and can display video pictures according to the recovered video data on an accessible display device.

[0068] In the embodiment shown in FIG. 1, the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140 can be servers or terminals, although the principles of the present application can not be limited thereto.

[0069] The server can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platform. The terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart voice interaction device, a smart watch, a smart home appliance, a vehicle-mounted terminal, an aircraft, and the like, but is not limited thereto.

[0070] The network 150 shown in FIG. 1 represents any number of networks that convey encoded video data among the first, second, third, and fourth terminal devices 110, 120, 130, 140, including for example wire- and / or wireless communication networks. The communication network 150 can exchange data in circuit-switched and / or packet-switched channels. The network can include telecommunication networks, local area and / or wide area networks, and / or the Internet. For the purposes of the present application, the architecture and topology of the network 150 can be immaterial to the operation of the disclosed application unless otherwise explained herein below.

[0071] In an embodiment of the application, FIG. 2 shows the placement of video encoding and video decoding devices in a streaming environment. The disclosed subject matter can be equally applicable to other video enabled applications including, for example, video conferencing, digital television (TV), storing of compressed video on digital media including CD, DVD, memory stick and the like, and so on.

[0072] The streaming system can include a capture subsystem 213 that can include a video source 201, for example a digital camera, that creates an uncompressed video picture stream 202. In an embodiment, the video picture stream 202 includes samples as they are captured by the digital camera. In contrast to encoded video data 204 (or encoded video bitstreams 204), the video picture stream 202 is depicted with a bold line to emphasize the high data volume of the video picture stream 202 that can be processed by the electronic device 220 that includes a video encoding device 203 coupled to the video source 201. The video encoding device 203 can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in greater detail below. In contrast to the video picture stream 202, the encoded video data 204 (or encoded video bitstream 204) is depicted with a thin line to emphasize the relatively lower data volume of the encoded video data 204 (or encoded video bitstream 204) that can be stored on a streaming server 205 for future use. One or more streaming client subsystems, such as client subsystem 206 and client subsystem 208 in FIG. 2, can access the streaming server 205 to retrieve copies 207 and 209 of the encoded video data 204. The client subsystem 206 can include a video decoding device 210, for example, in the electronic device 230. The video decoding device 210 decodes the incoming copy 207 of encoded video data and creates an outgoing video picture stream 211 that can be rendered on a display 212, such as a display screen, or other rendering device. In some streaming systems, the encoded video data 204, video data 207, and video data 209 (e.g., video bitstreams) can be encoded according to certain video encoding / compression standards.

[0073] It is noted that the electronic devices 220 and 230 can include other components not shown in the figures. For example, the electronic device 220 can include a video decoding device and the electronic device 230 can also include a video encoding device.

[0074] In an embodiment of the present application, taking the High Efficiency Video Coding (HEVC) in the international video coding standard, Versatile Video Coding (VVC), and the Chinese national video coding standard AVS as examples, when an input video frame image is input, the video frame image is divided into a plurality of non-overlapping processing units according to a block size, and each processing unit will perform similar compression operations. This processing unit is called a Coding Tree Unit (CTU), or referred to as a Largest Coding Unit (LCU). The CTU can be further divided into one or more basic Coding Units (CUs) for more detailed division. The CU is the most basic element in the coding link.

[0075] In another embodiment, the processing unit can also be referred to as a coding tile (i.e., a tile), which is a rectangular area of a multimedia data frame that can be independently decoded and encoded. In the Alliance for Open Media Video 1 (AV1) standard formulated by the Open Media Alliance, the coding tile can be further divided into one or more Superblocks (SBs) for more detailed division. The SB is the starting point of block division and can be further divided into a plurality of subblocks. Then, the SB is further divided into one or more blocks (B). Each block is the most basic element in the coding link. Alternatively, one SB can contain a plurality of B.

[0076] The above-mentioned division method of the video frame image can be referred to as a block partition structure. Some concepts in the coding process are introduced as follows:

[0077] Predictive coding (Predictive Coding): Predictive coding includes intra prediction and inter prediction, etc. After the original video signal is predicted by the selected reconstructed video signal, a residual video signal is obtained. The encoding end needs to determine which prediction coding mode is selected for the current coding unit (or coding block) and inform the decoding end. Intra prediction refers to that the predicted signal comes from the already coded and reconstructed region in the same image; inter prediction refers to that the predicted signal comes from the already coded other image (referred to as reference image) different from the current image.

[0078] Transform & Quantization: After the residual video signal is transformed by Discrete Fourier Transform (DFT), Discrete Cosine Transform (DCT) or other transform operations, the signal is converted into the transform domain, which is called transform coefficient. The transform coefficient is further subjected to a lossy quantization operation, which loses some information, so that the quantized signal is conducive to compressed expression. In some video coding standards, more than one transform method can be selected, so the encoding end also needs to select one of them for the current coding unit (or coding block) and inform the decoding end. The degree of quantization precision is usually determined by the quantization parameter (QP). If the QP value is large, the coefficients with larger value range will be quantized to the same output, so it usually brings larger distortion and lower code rate; on the contrary, if the QP value is small, the coefficients with smaller value range will be quantized to the same output, so it usually brings smaller distortion and higher code rate.

[0079] Entropy Coding or Statistical Coding: The quantized transform domain signal will be statistically compressed and coded according to the frequency of each value, and finally the binary (0 or 1) compressed code stream will be output. At the same time, other information such as the selected coding mode, motion vector data, etc. also needs to be entropy coded to reduce the code rate. Statistical coding is a lossless coding method that can effectively reduce the code rate required to express the same signal. Common statistical coding methods include variable length coding (VLC) or context-based binary arithmetic coding (CABAC).

[0080] The CABAC process mainly includes three steps: binarization, context modeling and binary arithmetic coding. After the input syntax elements are binarized, the binary data can be encoded by regular coding mode and bypass coding mode. The bypass coding mode does not need to assign a specific probability model to each binary bit, and the input binary bit bin value is directly encoded by a simple bypass encoder to speed up the entire encoding and decoding speed. Generally, different syntax elements are not completely independent, and the same syntax element also has certain memory. Therefore, according to the conditional entropy theory, the use of other coded syntax elements for conditional coding can further improve the coding performance compared with independent coding or non-memory coding. These coded symbol information used as conditions are called contexts. In the regular coding mode, the binary bits of the syntax elements enter the context modeler in order, and the encoder assigns an appropriate probability model to each input binary bit according to the value of the previously coded syntax element or binary bit. This process is called context modeling. The context model corresponding to the syntax element can be located through the context index increment (ctxIdxInc) and the context index start (ctxIdxStart). After the bin value and the assigned probability model are sent to the binary arithmetic encoder for coding, the context model needs to be updated according to the bin value, that is, the adaptive process in coding.

[0081] Loop filtering: The signal after transformation and quantization is obtained by inverse quantization, inverse transformation and prediction compensation to obtain the reconstructed image. Compared with the original image, the reconstructed image has some differences due to the influence of quantization, that is, the reconstructed image will produce distortion. Therefore, the reconstructed image can be filtered, for example, deblocking filter (DB), sample adaptive offset (SAO) or adaptive loop filter (ALF) filter, which can effectively reduce the distortion degree caused by quantization. Since these filtered reconstructed images will be used as references for subsequent encoded images to predict future image signals, the above filtering operation is also called loop filtering, that is, the filtering operation in the coding loop.

[0082] In an embodiment of the present application, FIG. 3 shows a basic flowchart of a video encoder, in which the intra prediction is taken as an example for illustration. In the flowchart, the original image signal s k[x, y] and the prediction image signal Difference operation is performed to obtain the residual signal u k [x, y], the residual signal u k After the transform and quantization processing, the quantized coefficients are obtained, which are encoded into bitstream by entropy coding on one hand, and are reconstructed into the reconstructed residual signal u' by inverse quantization and inverse transform processing on the other hand k [x, y], the prediction image signal and the reconstructed residual signal u' k [x, y] is superimposed to generate the image signal the image signal On one hand, it is input into the intra mode decision module and the intra prediction module for intra prediction processing, and on the other hand, it is output as the reconstructed image signal s' by loop filtering k [x, y], the reconstructed image signal s' k [x, y] can be used as the reference image for motion estimation and motion compensation prediction of the next frame. Then, based on the result s' of the motion compensation prediction r [x + m x , y + m y ] and the intra prediction result to obtain the prediction image signal of the next frame and the above process is repeated until the encoding is completed.

[0083] Based on the above encoding process, at the decoding end, for each coding unit (or coding block), after the compressed code stream (i.e. bitstream) is obtained, entropy decoding is performed to obtain various mode information and quantized coefficients. Then, the quantized coefficients are processed by inverse quantization and inverse transform to obtain the residual signal. On the other hand, according to the known encoding mode information, the prediction signal corresponding to the coding unit (or coding block) can be obtained, and then the residual signal is added to the prediction signal to obtain the reconstructed signal. The reconstructed signal is further processed by loop filtering and other operations to generate the final output signal. In this series of encoding processes, the encoding framework mainly makes decisions based on rate-distortion optimization (RDO) to select the optimal encoding parameters.

[0084] In the field of coding technology, intra prediction is a commonly used prediction coding technology. Intra prediction is based on the correlation of pixels in a video image in the spatial domain, and derives the prediction value of the current coding block from the adjacent coded area. The AVS3 second stage adopts the extended intra angle prediction mode (EIPM). In the last generation of AVS2, there are 33 intra prediction modes in total, including 30 angle prediction modes and 3 special prediction modes (Plane prediction mode, DC prediction mode and Bilinear prediction mode), 2 most probable modes (MPM) are used for coding, and the remaining modes are coded using 5-bit fixed-length coding. To support more precise angle prediction, the angle prediction mode in AVS3 is expanded to 62, as shown in FIG. 4, and the newly added angle prediction mode is numbered from 34 to 65.

[0085] When using the angle prediction mode, the pixel points in the current prediction block will take the reference pixel value at the corresponding position in the reference pixel row or column according to the direction corresponding to the angle of the prediction mode as the prediction value. As shown in FIG. 5, for the pixel point P in the prediction block, first, according to the prediction angle in the figure, the position of the reference pixel in the already coded pixel row will be determined, and then the reference pixel value is taken as the prediction value of the pixel point P. It should be noted that not all pixel positions point to reference pixel positions with integer pixel accuracy, such as the reference pixel position of pixel point P in FIG. 5, which is a certain sub-pixel position between pixels B and C, so the prediction pixel value of this position needs to be obtained by interpolation using the surrounding pixels. In order to improve the efficiency of intra prediction, on-chip memory is usually used to store the reference pixels for intra prediction.

[0086] As shown in FIG. 6, inter prediction utilizes the correlation in the video temporal domain, uses the pixels of the neighboring coded image to predict the pixels of the current image, so as to effectively remove the video temporal redundancy, and effectively save the bits of the coded residual data. Wherein, P represents the current frame, Pr represents the reference frame, B represents the current coding block, and Br represents the reference block of B. The coordinates of B' in the reference frame are the same as the coordinate position of B in the current frame, the coordinates of Br are (xr, yr), and the coordinates of B' are (x, y). The displacement between the current coding block and the reference block thereof is referred to as a motion vector (MV), wherein MV=(xr-x, yr-y). In other words, inter prediction refers to the process of searching the reference block from the neighboring coded image (i.e. the reference frame) according to the current block to be coded in the current frame, so as to remove the temporal redundancy of the video signal. As shown in FIG. 7, the current block to be coded in the current frame is searched in a certain range (i.e. in the search area formed by the search frame) in the reference frame according to the block matching criterion, and the best matching block is obtained. Optionally, the commonly used block matching criterion in video coding includes the minimum mean square error (MSE), sum of absolute difference (SAD), and the like.

[0087] The motion estimation search process can often use a plurality of different search shapes or search algorithms, wherein the full search algorithm is the most direct search algorithm, that is, all possible pixel positions in the search frame are traversed, and the MV corresponding to the best matching block obtained is the global optimal MV. However, the full search has a high complexity, and a fast search algorithm such as a two-dimensional logarithmic search algorithm, a three-step search algorithm, a TZSearch search algorithm, and the like is generally used. For example, as shown in FIG. 8, the candidate MVP is taken as the search starting point (i.e. the point located at the center shown in FIG. 8), and the search is performed in the diamond search area, wherein the search step length is increased according to the integer power of 2, and the point with the minimum rate-distortion cost is selected as the optimal search result of the current search stage.

[0088] Since inter prediction needs to obtain pixel prediction value in reference frame by using motion vector, in actual situation, the movement of objects between adjacent image frames is not necessarily based on integer pixel as the basic unit, therefore, it is necessary to improve the accuracy of motion estimation to sub-pixel level, that is, to interpolate the reference image to improve the accuracy of motion compensation, and then to improve the coding efficiency. FIG. 9 shows a schematic diagram of pixel position distribution of sub-pixel accuracy, wherein the black square represents the integer pixel position (the resolution of all integer pixels in a frame is equal to the video resolution), and the black circle represents the 1 / 4 sub-pixel position. In AVS3, the common inter prediction mode supports five kinds of motion vector accuracy: 1 / 4, 1 / 2, 1, 2 and 4. When the MV accuracy is 1 / 4 or 1 / 2, the 8-tap interpolation filter can be used to interpolate the luminance prediction value, and the 4-tap interpolation filter can be used to interpolate the chrominance prediction value. When interpolating the boundary sub-pixel of the current prediction block, it is often necessary to read additional pixels outside the current prediction block for 8-tap or 4-tap interpolation calculation, that is, the total number of pixels required by the interpolation process is often greater than the total number of pixels of the current prediction block. Specifically, FIG. 10 shows the pixel range required by the 8-tap interpolation filter for horizontal interpolation and vertical interpolation under 1 / 4 pixel accuracy (wherein the black square in FIG. 10 represents the integer pixel position, the black circle represents the 1 / 4 sub-pixel position, and the black square represents the integer pixel position outside the prediction block).

[0089] Inter template matching is an important inter prediction technology in the field of video coding, which aims to further refine the motion vector of the current block based on template matching technology, so as to obtain more accurate prediction results. Specifically, first, M prediction motion vectors (Motion Vector Prediction, MVP) are obtained from the Advanced Motion Vector Prediction (AMVP), Skip or Direct mode in the inter prediction process as candidate MVPs of the current block to be predicted (i.e. the coding block shown in FIG. 11), and then a template region (the width of the template region is N, N is a positive integer) is constructed based on the surrounding reconstructed pixels of the current block to be predicted, and then template matching search is performed on each available direction of the candidate MVP to determine the best matching block.

[0090] The inter-frame template matching usually adopts the way of coarse search first and then fine search. In the process of coarse search, firstly, the initial MVP is updated by using a certain shape search method with the corresponding candidate MVP as the search starting point in the reference frame corresponding to the candidate MVP, and the cost between the template prediction region (as shown in FIG. 11) in the reference frame corresponding to the updated MVP and the template region of the current block to be predicted (i.e. the encoding block shown in FIG. 11) is calculated in each search. If the template prediction region in the reference frame is a sub-pixel accuracy region, interpolation is needed to generate the template prediction region before the cost calculation. In each search process, the point with the minimum template cost is taken as the center point of the next search. If the center point is optimal, the current search is terminated, and the optimal coarse search MV is obtained. Then, fine search is performed based on a certain shape to obtain the best MV. As shown in FIG. 12, it is an example of the shape of coarse search and fine search. The main difference between the coarse search and the fine search is the search step.

[0091] After the best MV is obtained by searching, the inter-frame template matching directly generates the matching block (i.e. the optimal reference block) corresponding to the current block by using the interpolation filtering method introduced in the above embodiment. Therefore, in order to successfully generate the matching block of the current block, in addition to the reference block corresponding to the current block (as shown in FIG. 11, the current block is the encoding block, and the reference block is the prediction block), the inter-frame template matching also needs to obtain the pixels outside a certain range of the reference block (i.e. the search region pixels) and the additional pixels that may be needed in the final interpolation process (i.e. the pixels in the interpolation expansion region). As shown in FIG. 13, in an optional embodiment, the search region is a set size region surrounding the reference block, and the interpolation expansion region is a set size region outside the search region.

[0092] From the perspective of the decoding end, in order to generate the prediction block of the current block, the ordinary inter-frame prediction technology only needs to read the reference block with the corresponding pixel size in the reference frame according to the motion vector, while the inter-frame template matching technology also needs to read the search region pixels and the interpolation expansion region pixels; and when the interpolation filtering processing of the template prediction region is needed, the interpolation expansion region pixels corresponding to the template prediction region also need to be read. It can be seen that the inter-frame template matching technology introduces a large bandwidth overhead in hardware implementation, and the excessive hardware overhead limits the practical application of the encoding technology.

[0093] Based on this, the technical scheme of the embodiment of the present application proposes a new video coding technology applied to inter-frame template matching, which can use the existing pixel values in the reference frame to determine the pixel values in the interpolation expansion region (including the interpolation expansion region in the inter-frame template matching process and the interpolation expansion region in the motion compensation), and then make the decoding end not need to decode the pixel values in the interpolation expansion region from the code stream when decoding, thereby avoiding reading more pixels in the actual coding process, reducing the bandwidth overhead on the hardware implementation, and thus being beneficial to improving the coding efficiency of the video.

[0094] The implementation details of the technical scheme of the embodiment of the present application are described in detail as follows:

[0095] FIG. 14 shows a flowchart of a video decoding method according to one embodiment of the present application, which can be executed by an electronic device with a computing processing function, such as a terminal device or a server. Referring to FIG. 14, the video decoding method at least includes S1410 to S1470, which are described in detail as follows:

[0096] In S1420, a video code stream is received.

[0097] In S1420, at least one candidate motion vector for inter-frame template matching of a current block is determined based on the video code stream.

[0098] In some embodiments, the video includes a sequence of video image frames, and the sequence of video image frames includes a series of images, each image can be further divided into slices (Slices), and each slice can be divided into a series of LCUs (or CTUs), and each LCU contains a plurality of CUs. The video image frames are encoded in units of blocks during encoding, and in some new video coding standards, such as the H.264 standard, there are macroblocks (macroblock, MB), which can be further divided into a plurality of prediction blocks (prediction) that can be used for prediction encoding. In the HEVC standard, the basic concepts of coding unit (CU), prediction unit (PU), and transform unit (TU) are adopted, and a variety of block units are divided in terms of function, and a new tree-based structure is used for description. For example, a CU can be divided into smaller CUs according to a quadtree, and the smaller CUs can be further divided, thereby forming a quadtree structure. The current block, the reference block, and the matching block in the embodiment of the present application can be a CU, or a smaller block than the CU, such as a smaller block obtained by dividing the CU.

[0099] In some embodiments, the candidate motion vector for inter template matching of the current block can be one or multiple. The candidate motion vector can be directly decoded from the video bitstream, or can be obtained from one or more motion vectors in the AMVP, Skip or Direct mode in the inter prediction process as the candidate motion vector of the current block.

[0100] In S1430, a template of the current block is constructed, the template including the reconstructed pixels around the current block. For example, the template of the current block in FIG. 11.

[0101] In S1440, at least one reference block of the current block is determined in the reference frame according to the at least one candidate motion vector.

[0102] In some embodiments, the candidate motion vector can be used to locate the position corresponding to the current block in the reference frame, which is the reference block corresponding to the current block.

[0103] In S1450, a template prediction region corresponding to each of the at least one reference block is determined in the reference frame based on the position of the corresponding reference block, each template prediction region having the same position relative to the corresponding reference block as the template relative to the current block. In addition, each template prediction region has the same shape and size as the template.

[0104] In S1460, the pixel values in a first interpolation expansion region for interpolation filtering of the corresponding template prediction region are determined according to the pixel values in the template prediction region corresponding to each reference block.

[0105] In some optional embodiments, the template prediction region can include the pixel region above the reference block and the pixel region to the left of the reference block. Specifically, as shown in FIG. 11, the template prediction region is the pixel region above the reference block (i.e., the prediction block shown in FIG. 11) and the pixel region to the left of the reference block, i.e., the shaded region formed to the left and above the prediction block shown in FIG. 11.

[0106] The first interpolation expansion region includes at least one of: a first number of pixel regions above the template prediction region; a second number of pixel regions below the template prediction region; a third number of pixel regions left of the template prediction region; a fourth number of pixel regions right of the template prediction region; a pixel region above-left of the template prediction region; a pixel region above-right of the template prediction region; a pixel region below-left of the template prediction region; and a pixel region below-right of the template prediction region. Optionally, the first number, the second number, the third number and the fourth number can be equal, unequal, or partially equal, such as being selected from 1, 2, 3, 4, 5, 6, etc.

[0107] Optionally, as shown in FIG. 15, the first interpolation expansion region can have multiple setting modes (the setting modes in FIG. 15 are only examples), i.e., the first interpolation expansion region can be at least one of above, below, left and right of the template prediction region. Optionally, the first interpolation expansion region can also include pixel regions of one or more of the top-left corner, the top-right corner, the bottom-left corner and the bottom-right corner in modes A to O shown in FIG. 15. As shown in FIG. 15, taking mode M as an example, if the pixel regions of the top-left corner and the bottom-left corner are included, the first interpolation expansion region can be as shown in mode M' in FIG. 15; and as shown in mode O in FIG. 15, if the pixel regions of the top-left corner, the top-right corner, the bottom-left corner and the bottom-right corner are included, the first interpolation expansion region can be as shown in mode P in FIG. 15.

[0108] In some optional embodiments, when determining the pixel values in the first interpolation expansion region, the pixel values in the template prediction region can be copied as the pixel values in the first interpolation expansion region, such as copying a number of pixel values to fill the pixel values in the set positions in the first interpolation expansion region. Specifically, as shown in FIG. 16, the pixel value at position 1601 in the template prediction region can be copied, and the pixel value at position 1602 in the template prediction region can be copied, as the pixel values at the corresponding positions in the first interpolation expansion region. And the pixel value at the top-left corner of the first interpolation expansion region can be obtained by copying the pixel value at a specified position (such as the pixel value at the top-left corner of the template prediction region, the pixel value at the adjacent position in the first interpolation expansion region).

[0109] In some optional embodiments, when determining the pixel values in the first interpolation expansion region, the pixel values in the template prediction region can be copied based on a symmetry axis to obtain the pixel values in the first interpolation expansion region. Specifically, as shown in FIG. 17, the copying is performed in the square dotted line box 1701 to obtain the pixel values at the corresponding positions in the first interpolation expansion region; and the copying is performed in the square dotted line box 1702 to obtain the pixel values at the corresponding positions in the first interpolation expansion region.

[0110] In some optional embodiments, when determining the pixel value in the first interpolation expansion region, the pixel value in the template prediction region can be copied based on the symmetry point to obtain the pixel value in the first interpolation expansion region. As shown in FIG. 17, the pixel value at the upper left corner of the first interpolation expansion region is copied based on the symmetry point 1703.

[0111] In some optional embodiments, when determining the pixel value in the first interpolation expansion region, the pixel value in the template prediction region can be weighted calculated to obtain the pixel value in the first interpolation expansion region. Specifically, as shown in FIG. 18, P1' in the first interpolation expansion region = (2*P4+P1+P7+2)>>2; P2' in the first interpolation expansion region = (2*P5+P2+P8+2)>>2; P3' in the first interpolation expansion region = (2*P6+P3+P9+2)>>2.

[0112] In some optional embodiments, when determining the pixel value in the first interpolation expansion region, the pixel value in the template prediction region can be processed based on the intra prediction to obtain the pixel value in the first interpolation expansion region. For example, the pixel value at the corresponding position in the first interpolation expansion region can be filled based on the calculation formula of the angle mode, the direct current (DC) mode, the planar (Planar) mode, etc.

[0113] It should be noted that the above embodiments of determining the pixel value in the first interpolation expansion region can be used in combination, or used alone. Meanwhile, the pixel at the upper left corner of the first interpolation expansion region can be obtained by copying or symmetry of the pixel above or left of the first interpolation expansion region, or by one of the weighted calculation, the intra prediction, etc.; the pixel at the upper right corner of the first interpolation expansion region can be obtained by copying or symmetry of the pixel above or right of the first interpolation expansion region, or by one of the weighted calculation, the intra prediction, etc.; the pixel at the lower left corner of the first interpolation expansion region can be obtained by copying or symmetry of the pixel below or left of the first interpolation expansion region, or by one of the weighted calculation, the intra prediction, etc.; the pixel at the lower right corner of the first interpolation expansion region can be obtained by copying or symmetry of the pixel below or right of the first interpolation expansion region, or by one of the weighted calculation, the intra prediction, etc.

[0114] In S1470, based on the pixel value in the first interpolation expansion region, the sub-pixel interpolation of the corresponding template prediction region is determined by the interpolation filtering processing, as the prediction value of the template corresponding to the corresponding template prediction region.

[0115] In some optional embodiments, after the template prediction region is interpolated and filtered to obtain the prediction value of the template, inter-frame template matching can be performed based on the prediction value of the template to obtain the best motion vector corresponding to the at least one candidate motion vector. Specifically, the cost between the template prediction region and the template region of the current block can be calculated, and then the point with the minimum template cost is selected as the center point of the next search. If the center point is optimal, the current search is terminated. After the optimal coarse search MV is obtained, fine search is performed based on a certain shape to obtain the best motion vector.

[0116] In some optional embodiments, after the best motion vector is obtained, the matching block corresponding to the current block can be determined according to the best motion vector, and then the matching block can be interpolated and filtered. Alternatively, when the matching block is interpolated and filtered, a related interpolation filtering method can be used, such as reading the pixels in the search region and the pixels in the second interpolation expansion region (i.e., the interpolation expansion region used for interpolation filtering in the motion compensation process) used for interpolation filtering of the matching block. Specifically, the interpolation filtering can be performed by using a bilinear interpolation method, a nearest neighbor interpolation method, a bicubic interpolation method, a 6-tap interpolation filter, or a 12-tap interpolation filter.

[0117] Alternatively, when the matching block is interpolated and filtered, the pixel values in the second interpolation expansion region used for interpolation filtering in the motion compensation process can also be determined according to the pixel values in the set region in the reference frame, and then the matching block is interpolated and filtered based on the second interpolation expansion region. The technical solution of this embodiment makes it unnecessary to decode the pixel values in the second interpolation expansion region from the code stream, reduces the bandwidth overhead in hardware implementation, and is beneficial to improving the coding and decoding efficiency of the video.

[0118] In some optional embodiments, the setting region for determining the pixel values in the second interpolation expansion region can include at least one of the following: a first pixel position included in the search region in the reference frame, a second pixel position in the reference block corresponding to the matching block. In other words, the setting region includes at least one of a sub-region in the search region and a sub-region of the reference block corresponding to the matching block. The reference block corresponding to the matching block refers to the reference block determined by the initial candidate motion vector corresponding to the best motion vector (which is one of the candidate motion vectors generated in step S1430). The local search with each candidate motion vector as the initial candidate motion vector can determine the best motion vector corresponding to each candidate motion vector. On this basis, the best motion vector for determining the matching block can be selected from the best motion vectors corresponding to the plurality of candidate motion vectors, i.e., the best motion vectors corresponding to the plurality of candidate motion vectors. Optionally, the first pixel position can be the position of one pixel or the positions of a plurality of pixels; the second pixel position can be the position of one pixel or the positions of a plurality of pixels. When determining the pixel values in the second interpolation expansion region according to the pixel values in the setting region, the scheme for determining the pixel values in the first interpolation expansion region can be referred to, i.e., the pixel values in the second interpolation expansion region can be obtained by one or more of the following manners: copying the pixel values in the setting region, copying the pixel values in the setting region based on the symmetry axis, copying the pixel values in the setting region based on the symmetry point, performing weighted calculation on the pixel values in the setting region, performing intra-frame prediction processing based on the pixel values in the setting region, etc.

[0119] In some embodiments, the search region is a predetermined region in the reference frame containing the collocated block of the current block. The search region contains at least one reference block corresponding to at least one candidate motion vector.

[0120] In some optional embodiments, each reference block corresponds to one search region. The search region of each reference block includes at least one of the following regions: a pixel region of a certain number of rows (such as 1, 2, 3, 4, 5, 6, etc.) above the reference block; a pixel region of a certain number of rows below the reference block; a pixel region of a certain number of columns to the left of the reference block; a pixel region of a certain number of columns to the right of the reference block; a pixel region above and to the left of the reference block; a pixel region above and to the right of the reference block; a pixel region below and to the left of the reference block; a pixel region below and to the right of the reference block.

[0121] Optionally, as shown in FIG. 19, the search area can have various configurations (ways A-P in FIG. 19 are merely examples). Optionally, the search area can also include the pixel regions of the top-left corner, top-right corner, bottom-left corner, and bottom-right corner in ways A to O shown in FIG. 19. For example, if the pixel region of the top-left corner and bottom-left corner is included, as in way M in FIG. 19, the search area can be as shown in way M' in FIG. 19. For another example, if the pixel region of the top-left corner, top-right corner, bottom-left corner, and bottom-right corner is included, as in way O in FIG. 19, the search area can be as shown in way P in FIG. 19. In some optional embodiments, the pixel values in the search area can be obtained from the reference frame, i.e., directly read from the reference frame, without using additional methods for filling.

[0122] In some optional embodiments, the range of the second interpolation expansion area can be determined according to at least one of the following information: the position of the matching block, the number of taps of the interpolation filter, and the range of the search area. The following are described respectively:

[0123] In an embodiment of the present application, when determining the range of the second interpolation expansion area, if the pixels needed for the interpolation processing of the matching block exceed the range of the search area, the search area can be further expanded to obtain the second interpolation expansion area. If the pixels needed for the interpolation processing of the matching block are within the range of the search area, the interpolation filtering processing can be implemented without expanding the search area.

[0124] In some optional embodiments, the second interpolation expansion area obtained by expanding the search area can be the pixel region of the fifth number of rows above the search area and the pixel region of the sixth number of rows below the search area, when the absolute difference between the horizontal coordinate of the top-left vertex of the reference block and the horizontal coordinate of the top-left vertex of the matching block is less than or equal to a first threshold value, and the absolute difference between the vertical coordinate of the top-left vertex of the reference block and the vertical coordinate of the top-left vertex of the matching block is greater than a second threshold value. Optionally, the fifth number and the sixth number can be equal or not equal, such as can be selected from 1, 2, 3, 4, 5, 6, etc. The first threshold value can be the difference between half of the number of taps used in the horizontal direction by the interpolation filter used in the motion compensation process and a set constant value; the second threshold value can be the difference between half of the number of taps used in the vertical direction by the interpolation filter used in the motion compensation process and a set constant value, which can be 1 or other values, etc. The number of taps used in the horizontal direction by the interpolation filter used in the motion compensation process and the number of taps used in the vertical direction can be equal or not equal.

[0125] Optionally, if the number of taps is 4, the fifth number can be 1, and the sixth number can be 2; if the number of taps is 6, the fifth number can be 2, and the sixth number can be 3; if the number of taps is 8, the fifth number can be 3, and the sixth number can be 4; if the number of taps is 12, the fifth number can be 5, and the sixth number can be 6.

[0126] In some optional embodiments, if the absolute difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is less than or equal to a first threshold, the absolute difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than a second threshold, and the difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than 0, then the pixel region of the fifth number of rows above the search region is taken as the region range of the second interpolation expansion region. Optionally, the fifth number can be selected from 1, 2, 3, 4, 5, 6, etc. The first threshold can be the difference between half of the number of taps used by the interpolation filter used in the motion compensation process in the horizontal direction and a set constant value; the second threshold can be the difference between half of the number of taps used by the interpolation filter used in the motion compensation process in the vertical direction and a set constant value, which can be 1 or other values, etc. The number of taps used by the interpolation filter used in the motion compensation process in the horizontal direction can be equal to or different from the number of taps used in the vertical direction.

[0127] Optionally, if the number of taps is 4, the fifth number can be 1; if the number of taps is 6, the fifth number can be 2; if the number of taps is 8, the fifth number can be 3; if the number of taps is 12, the fifth number can be 5.

[0128] In some optional embodiments, if the absolute difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is less than or equal to a first threshold, the absolute difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than a second threshold, and the difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is less than or equal to 0, then the pixel region of the sixth number of rows below the search region is taken as the region range of the second interpolation expansion region. Optionally, the sixth number can be selected from 1, 2, 3, 4, 5, 6, etc. The first threshold can be the difference between half of the number of taps used by the interpolation filter used in the motion compensation process in the horizontal direction and a set constant value; the second threshold can be the difference between half of the number of taps used by the interpolation filter used in the motion compensation process in the vertical direction and a set constant value, which can be 1 or other values, etc. The number of taps used by the interpolation filter used in the motion compensation process in the horizontal direction can be equal to or different from the number of taps used in the vertical direction.

[0129] Optionally, the sixth number can be 2 if the tap number is 4, 3 if the tap number is 6, 4 if the tap number is 8, and 6 if the tap number is 12.

[0130] In some optional embodiments, the second interpolation expansion region can be obtained by expanding the search region, and the region range of the second interpolation expansion region can be a pixel region of a seventh number of columns located on the left of the search region and a pixel region of an eighth number of columns located on the right of the search region when an absolute difference between the top-left vertex vertical coordinate of the reference block and the top-left vertex vertical coordinate of the matching block is less than or equal to a third threshold value and an absolute difference between the top-left vertex horizontal coordinate of the reference block and the top-left vertex horizontal coordinate of the matching block is greater than a fourth threshold value. Optionally, the seventh number and the eighth number can be equal or not equal, such as being selected from 1, 2, 3, 4, 5, 6, etc. Optionally, the third threshold value can be a difference between half of a tap number used by an interpolation filter in a vertical direction in a motion compensation process and a set constant value, and the fourth threshold value can be a difference between half of a tap number used by the interpolation filter in a horizontal direction in the motion compensation process and the set constant value, which can be 1 or other values, etc. The tap number used by the interpolation filter in the horizontal direction in the motion compensation process can be equal to or not equal to the tap number used by the interpolation filter in the vertical direction in the motion compensation process.

[0131] Optionally, the seventh number can be 1 and the eighth number can be 2 if the tap number is 4, 2 and the eighth number can be 3 if the tap number is 6, 3 and the eighth number can be 4 if the tap number is 8, and 5 and the eighth number can be 6 if the tap number is 12.

[0132] In some optional embodiments, the region range of the second interpolation expansion region can be a pixel region of a seventh number of columns located on the left of the search region when an absolute difference between the top-left vertex vertical coordinate of the reference block and the top-left vertex vertical coordinate of the matching block is less than or equal to a third threshold value, an absolute difference between the top-left vertex horizontal coordinate of the reference block and the top-left vertex horizontal coordinate of the matching block is greater than a fourth threshold value, and a difference between the top-left vertex horizontal coordinate of the reference block and the top-left vertex horizontal coordinate of the matching block is greater than 0. Optionally, the seventh number can be selected from 1, 2, 3, 4, 5, 6, etc. Optionally, the third threshold value can be a difference between half of a tap number used by an interpolation filter in a vertical direction in a motion compensation process and a set constant value, and the fourth threshold value can be a difference between half of a tap number used by the interpolation filter in a horizontal direction in the motion compensation process and the set constant value, which can be 1 or other values, etc. The tap number used by the interpolation filter in the horizontal direction in the motion compensation process can be equal to or not equal to the tap number used by the interpolation filter in the vertical direction in the motion compensation process.

[0133] Optionally, the seventh number can be 1 if the tap number is 4; the seventh number can be 2 if the tap number is 6; the seventh number can be 3 if the tap number is 8; the seventh number can be 5 if the tap number is 12.

[0134] In some optional embodiments, if the absolute difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is less than or equal to a third threshold value, the absolute difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is greater than a fourth threshold value, and the difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is less than or equal to 0, then the pixel region of the eighth number of columns located to the right of the search region is taken as the region range of the second interpolation expansion region. Optionally, the eighth number can be selected from 1, 2, 3, 4, 5, 6, etc. Optionally, the third threshold value can be the difference between half of the tap number used by the interpolation filter used in the motion compensation process in the vertical direction and a set constant value; the fourth threshold value can be the difference between half of the tap number used by the interpolation filter used in the motion compensation process in the horizontal direction and a set constant value, which can be 1 or other values, etc. The tap number used by the interpolation filter used in the motion compensation process in the horizontal direction can be equal to or different from the tap number used in the vertical direction.

[0135] Optionally, the eighth number can be 2 if the tap number is 4; the eighth number can be 3 if the tap number is 6; the eighth number can be 4 if the tap number is 8; the eighth number can be 6 if the tap number is 12.

[0136] In some optional embodiments, the second interpolation expansion region can be obtained by expanding the search region, and the region range of the second interpolation expansion region can be a fifth number of rows of pixel regions above the search region, a sixth number of rows of pixel regions below the search region, a seventh number of columns of pixel regions left to the search region, and an eighth number of columns of pixel regions right to the search region, when an absolute difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is greater than a fifth threshold value, and an absolute difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than a sixth threshold value. Optionally, the fifth number, the sixth number, the seventh number and the eighth number can be equal, can not be equal, or can be partially equal, such as can be selected from 1, 2, 3, 4, 5, 6, etc. Optionally, the fifth threshold value can be a difference between a half of a tap number used by an interpolation filter used in the motion compensation process in the horizontal direction and a set constant value; the sixth threshold value can be a difference between a half of a tap number used by the interpolation filter used in the motion compensation process in the vertical direction and a set constant value, which can be 1 or other values, etc. The tap number used by the interpolation filter used in the motion compensation process in the horizontal direction and the tap number used in the vertical direction can be equal or can not be equal.

[0137] In some optional embodiments, if the absolute difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is greater than the fifth threshold value, the absolute difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than the sixth threshold value, and the difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than 0, the fifth number of rows of pixel regions above the search region can be taken as the region range of the second interpolation expansion region. Optionally, the fifth number can be selected from 1, 2, 3, 4, 5, 6, etc. Optionally, the fifth threshold value can be a difference between a half of a tap number used by an interpolation filter used in the motion compensation process in the horizontal direction and a set constant value; the sixth threshold value can be a difference between a half of a tap number used by the interpolation filter used in the motion compensation process in the vertical direction and a set constant value, which can be 1 or other values, etc. The tap number used by the interpolation filter used in the motion compensation process in the horizontal direction and the tap number used in the vertical direction can be equal or can not be equal.

[0138] In some optional embodiments, if the absolute difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is greater than a fifth threshold, the absolute difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than a sixth threshold, and the difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is less than or equal to 0, then the pixel region of a sixth number of rows below the search region is taken as the region range of the second interpolation expansion region. Optionally, the sixth number can be selected from 1, 2, 3, 4, 5, 6, etc. Optionally, the fifth threshold can be the difference between half of the number of taps used in the horizontal direction by an interpolation filter used in the motion compensation process and a set constant value; the sixth threshold can be the difference between half of the number of taps used in the vertical direction by the interpolation filter used in the motion compensation process and a set constant value, which can be 1 or other values, etc. The number of taps used in the horizontal direction by the interpolation filter used in the motion compensation process can be equal to or different from the number of taps used in the vertical direction.

[0139] In some optional embodiments, if the absolute difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is greater than a fifth threshold, the absolute difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than a sixth threshold, and the difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than 0, then the pixel region of a seventh number of columns to the left of the search region is taken as the region range of the second interpolation expansion region. Optionally, the seventh number can be selected from 1, 2, 3, 4, 5, 6, etc. Optionally, the fifth threshold can be the difference between half of the number of taps used in the horizontal direction by an interpolation filter used in the motion compensation process and a set constant value; the sixth threshold can be the difference between half of the number of taps used in the vertical direction by the interpolation filter used in the motion compensation process and a set constant value, which can be 1 or other values, etc. The number of taps used in the horizontal direction by the interpolation filter used in the motion compensation process can be equal to or different from the number of taps used in the vertical direction.

[0140] In some optional embodiments, if the absolute difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is greater than a fifth threshold value, the absolute difference between the top-left corner vertical coordinate of the reference block and the top-left corner vertical coordinate of the matching block is greater than a sixth threshold value, and the difference between the top-left corner horizontal coordinate of the reference block and the top-left corner horizontal coordinate of the matching block is less than or equal to 0, the pixel region of an eighth number of columns located to the right of the search region is taken as the range of the second interpolation expansion region. Optionally, the eighth number can be selected from 1, 2, 3, 4, 5, 6, etc. Optionally, the fifth threshold value can be the difference between half of the number of taps used in the horizontal direction by an interpolation filter used in the motion compensation process and a set constant value; the sixth threshold value can be the difference between half of the number of taps used in the vertical direction by the interpolation filter used in the motion compensation process and a set constant value, which can be 1 or other values, etc. The number of taps used in the horizontal direction by the interpolation filter used in the motion compensation process can be equal to or different from the number of taps used in the vertical direction.

[0141] It should be noted that the top-left corner in the above embodiments is only an example, and the right-top corner, the left-bottom corner, the right-bottom corner, the center of the block, or any coordinate point at the same position in the reference block and the matching block can also be used.

[0142] Based on the above embodiments, it is assumed that the coordinates of the current block (such as the coordinates of the top-left corner of the current block) are (x, y), the width of the current block is w, the height of the current block is h, the initial MV of the inter-frame template matching is , and the best MV is Since the positions of the reference block and the best matching block are different, the second interpolation expansion region can be determined in the following manner:

[0143] When , the second interpolation expansion region can be obtained by expanding I pixels upward (i.e., expanding I rows of pixels upward) and expanding J pixels downward (i.e., expanding J rows of pixels downward) based on the search region. Optionally, under the above condition, and when , only I pixels can be expanded upward based on the search region; when , only J pixels can be expanded downward based on the search region.

[0144] When , the second interpolation expansion region can be obtained by expanding I pixels to the left (i.e., expanding I columns of pixels to the left) and expanding J pixels to the right (i.e., expanding J columns of pixels to the right) based on the search region. Optionally, under the above condition, and when , only I pixels can be expanded to the left based on the search region; when , only J pixels can be expanded to the right based on the search region.

[0145] When I1, J1 pixels are respectively extended upwards and downwards (i.e. I1 rows of pixels are extended upwards, and J1 rows of pixels are extended downwards), and I2, J2 pixels are respectively extended leftwards and rightwards (i.e. I2 columns of pixels are extended leftwards, and J2 columns of pixels are extended rightwards) based on the search region to obtain a second interpolation expansion region. Alternatively, under the above conditions, and I2 pixels are extended leftwards only based on the search region; under the above conditions, and J2 pixels are extended rightwards only based on the search region; under the above conditions, and I1 pixels are extended upwards only based on the search region; under the above conditions, and J1 pixels are extended downwards only based on the search region.

[0146] When the second interpolation expansion region does not exceed the boundary of the search region, i.e. no pixels need to be extended.

[0147] In the above examples, I, J, I1, J1, I2, J2 are all positive integers. T1 represents the number of taps of an interpolation filter used in a motion compensation process in the horizontal direction; and T2 represents the number of taps of the interpolation filter used in the motion compensation process in the vertical direction.

[0148] In an embodiment of the present application, if the second interpolation expansion region is determined based on the region range of the search region, then the region range of the second interpolation expansion region can be determined based on at least one of the following regions: a ninth number of rows of pixel regions above the search region; a tenth number of rows of pixel regions below the search region; an eleventh number of columns of pixel regions left of the search region; a twelfth number of columns of pixel regions right of the search region; a pixel region above and left of the search region; a pixel region above and right of the search region; a pixel region below and left of the search region; and a pixel region below and right of the search region.

[0149] Optionally, as shown in FIG. 20, the second interpolation expansion region can have various settings (the settings in FIG. 20 are described by taking the search region surrounding the reference block as an example, and the settings in FIG. 20 are merely examples), i.e., the second interpolation expansion region can be located at least one of the top, bottom, left and right of the search region. Optionally, the second interpolation expansion region can also contain the pixel region of one or more of the top-left corner, top-right corner, bottom-left corner and bottom-right corner in the modes A to O shown in FIG. 20. As shown in FIG. 20, taking mode M as an example, if the pixel region containing the top-left corner and bottom-left corner, the second interpolation expansion region can be as shown in mode M' in FIG. 20; and as shown in mode O in FIG. 20, if the pixel region containing the top-left corner, top-right corner, bottom-left corner and bottom-right corner, the second interpolation expansion region can be as shown in mode P in FIG. 20.

[0150] In an embodiment of the present application, if the second interpolation expansion region is determined according to the tap number of the interpolation filter used in the motion compensation process, when the tap number of the interpolation filter used in the motion compensation process is 4, the pixel region of 1 row located above the search region, the pixel region of 2 rows located below the search region, the pixel region of 1 column located left of the search region, and the pixel region of 2 columns located right of the search region can be taken as the range of the second interpolation expansion region. The specific values in this embodiment are merely examples.

[0151] In an embodiment of the present application, if the second interpolation expansion region is determined according to the tap number of the interpolation filter used in the motion compensation process, when the tap number of the interpolation filter used in the motion compensation process is 6, the pixel region of 2 rows located above the search region, the pixel region of 3 rows located below the search region, the pixel region of 2 columns located left of the search region, and the pixel region of 3 columns located right of the search region can be taken as the range of the second interpolation expansion region. The specific values in this embodiment are merely examples.

[0152] In an embodiment of the present application, if the second interpolation expansion region is determined according to the tap number of the interpolation filter used in the motion compensation process, when the tap number of the interpolation filter used in the motion compensation process is 8, the pixel region of 3 rows located above the search region, the pixel region of 4 rows located below the search region, the pixel region of 3 columns located left of the search region, and the pixel region of 4 columns located right of the search region can be taken as the range of the second interpolation expansion region. The specific values in this embodiment are merely examples.

[0153] In one embodiment of the present application, if the second interpolation expansion region is determined according to the tap number of the interpolation filter used in the motion compensation process, then when the tap number of the interpolation filter used in the motion compensation process is 12, the pixel region of 5 rows above the search region, the pixel region of 6 rows below the search region, the pixel region of 5 columns left of the search region, and the pixel region of 6 columns right of the search region can be taken as the region range of the second interpolation expansion region. The specific values in this embodiment are only examples.

[0154] In one embodiment of the present application, if the second interpolation expansion region is determined according to the tap number of the interpolation filter used in the motion compensation process, then when the tap number of the interpolation filter used in the motion compensation process is 12, the pixel region of 5 columns left of the search region, and the pixel region of 6 columns right of the search region can be taken as the region range of the second interpolation expansion region. The specific values in this embodiment are only examples.

[0155] In one embodiment of the present application, if the second interpolation expansion region is determined according to the tap number of the interpolation filter used in the motion compensation process, then when the tap number of the interpolation filter used in the motion compensation process is 12, the pixel region of 5 rows above the search region, and the pixel region of 6 rows below the search region can be taken as the region range of the second interpolation expansion region. The specific values in this embodiment are only examples.

[0156] In some optional embodiments, if the coordinate position of the reference block exceeds the image boundary of the reference frame, the coordinate position of the reference block can be adjusted to be on the image boundary; wherein the image boundary of the reference frame can be the boundary of the reference frame image, or can also be the boundary of the reference frame image after extending a set size region (such as a set row and / or a set column) outward.

[0157] In some optional embodiments, if the coordinate position of the matching block exceeds the image boundary of the reference frame, the coordinate position of the matching block can be adjusted to be on the image boundary; wherein the image boundary of the reference frame can be the boundary of the reference frame image, or can also be the boundary of the reference frame image after extending a set size region (such as a set row and / or a set column) outward.

[0158] In some optional embodiments, if the search region exceeds the image boundary of the reference frame, the boundary of the search region can be adjusted to be on the image boundary; wherein the image boundary of the reference frame can be the boundary of the reference frame image, or can also be the boundary of the reference frame image after extending a set size region (such as a set row and / or a set column) outward.

[0159] Specifically, assuming that the coordinates of the current block (such as the coordinates of the top-left corner of the current block) are (x, y), the width of the current block is w, the height of the current block is h, the initial MV and the best MV of the inter-frame template matching are and respectively, then the top-left coordinates, the top-right coordinates, the bottom-left coordinates and the bottom-right coordinates of the reference block of the current block are and the top-left coordinates, the top-right coordinates, the bottom-left coordinates and the bottom-right coordinates of the best matching block corresponding to the best MV are the top-left coordinates, the top-right coordinates, the bottom-left coordinates and the bottom-right coordinates of the best matching block corresponding to the best MV are Therefore, the coordinate values cannot exceed the image boundary, and it can also be understood that the reference block and the best matching block cannot exceed the image boundary. Alternatively, the image boundary can be the boundary of the current coding image or the boundary of the current coding image extended outward by N1 pixel units, where N1 is a positive integer (for example, the boundary of the current coding image extended outward by one CTU length).

[0160] If the coordinates of the reference block and the best matching block do not exceed the image boundary, the template matching and the pixel interpolation are normally performed; otherwise, the coordinates exceeding the image boundary are adjusted to the current image boundary position. In addition, the coordinates of the best matching block can be on the boundary of the search region, and therefore the boundary of the search region cannot exceed the image boundary.

[0161] In some optional embodiments, the number of taps of the first interpolation filter for performing the interpolation filtering on the template prediction region and the number of taps of the second interpolation filter for performing the interpolation filtering on the matching block are selected from the following numbers of taps: 1 tap, 2 taps, 4 taps, 6 taps, 8 taps, 12 taps; or the first interpolation filter and the second interpolation filter can also use the nearest neighbor interpolation, the bilinear interpolation and the bicubic interpolation to perform the interpolation. Alternatively, the nearest neighbor interpolation, the bilinear interpolation and the bicubic interpolation can be regarded as the interpolation filters with 1 tap, 2 taps and 4 taps respectively.

[0162] It should be noted that the number of taps of the first interpolation filter and the number of taps of the second interpolation filter are the same or different. For example, the first interpolation filter can be a 12-tap interpolation filter, and the second interpolation filter can also be a 12-tap interpolation filter; or the first interpolation filter can be a 6-tap interpolation filter, and the second interpolation filter can be a 12-tap interpolation filter.

[0163] In some optional embodiments, when the technical solutions of the embodiments of the present application are applied, the decoding end can determine the tap number of the interpolation filter used in the inter-frame template matching and the tap number of the interpolation filter used in the motion compensation according to the manner negotiated with the encoding end. For example, considering that the inter-frame template matching process itself does not require a too-high-precision interpolation filter, while the motion compensation process requires the quality of the matching block obtained finally to be as good as possible, the codec can default to use a 6-tap interpolation filter based on filling the interpolation expansion region with the existing pixels (i.e., the scheme of filling the first interpolation expansion region with the pixels in the template prediction region proposed in the embodiments of the present application) in the inter-frame template matching process, and use a 12-tap interpolation filter based on filling the interpolation expansion region with the existing pixels (i.e., the scheme of filling the second interpolation expansion region with the pixels in the search region and / or the reference block proposed in the embodiments of the present application) in the motion compensation process.

[0164] In some optional embodiments, when the technical solutions of the embodiments of the present application are applied, the decoding end can decode at least one flag bit from the video code stream, and then determine the tap number of the interpolation filter used in the inter-frame template matching and the tap number of the interpolation filter used in the motion compensation according to the value of the at least one flag bit. Optionally, the at least one flag bit includes one or more of the following flag bits: a flag bit included in a sequence header, a flag bit included in a picture header, a flag bit included in a slice header, a flag bit included in a CTU header, and a flag bit included in a coding block.

[0165] Specifically, the encoding end can adaptively select the interpolation filtering method optimal for each process by RDO, and indicate which specific manner is used by one or more of the block level, the CTU level, the slice level, the frame level, and the sequence level flag bit, so that the decoding end can directly parse the flag bit and obtain the corresponding processing manner. Specifically, for example, the encoding end can use 2 flag bits, the value of one flag bit is used to indicate the tap number of the interpolation filter used in the inter-frame template matching, and the value of the other flag bit is used to indicate the tap number of the interpolation filter used in the motion compensation.

[0166] In some embodiments of the present application, based on the foregoing scheme, the tap number used by the interpolation filter in the horizontal direction in the inter-frame template matching process can be the same as or different from the tap number used by the interpolation filter in the vertical direction; the tap number used by the interpolation filter in the horizontal direction in the motion compensation process can be the same as or different from the tap number used by the interpolation filter in the vertical direction.

[0167] In one specific application scenario of the present application, the inter-frame template matching process and the motion compensation process can use interpolation filters with different tap numbers, such as 1-tap, 2-tap, 4-tap, 6-tap, 8-tap and 12-tap. Optionally, several specific examples of the interpolation filter types that can be used in the inter-frame template matching process and the motion compensation process are shown in Table 1 below:

[0168] Table 1

[0169] In some optional embodiments, the 12-tap interpolation filter coefficients and the 6-tap interpolation filter coefficients at 1 / 4-pixel accuracy in the inter-frame template matching process can be as shown in Table 2 below:

[0170] Table 2

[0171] The schemes in the above embodiments of the present application can be used alone or in combination. The technical solutions of the embodiments of the present application can be used in the search process and / or the motion compensation process of technologies based on template matching, such as intra-frame and inter-frame, and can be applied to products of video codecs.

[0172] FIG. 21 shows a flowchart of a video encoding method according to one embodiment of the present application, which can be executed by a device with computing processing function, such as a terminal device or a server. Referring to FIG. 21, the video encoding method includes at least S2110 to S2170, which are described in detail as follows:

[0173] S2110, obtaining video data, the video data including a current block;

[0174] S2120, determining at least one candidate motion vector for inter-frame template matching of the current block based on the video data;

[0175] S2130, constructing a template of the current block, the template including pixel points around the current block;

[0176] S2140, determining at least one reference block of the current block in a reference frame according to the at least one candidate motion vector;

[0177] S2150, determining a template prediction region corresponding to each reference block in the reference frame based on the position of each reference block, each template prediction region having the same position relative to the corresponding reference block as the template has relative to the current block;

[0178] S2160, determining pixel values in a first interpolation expansion region for interpolation filter processing of the corresponding template prediction region according to the pixel values in the template prediction region corresponding to each reference block.

[0179] S2170, determining sub-pixel interpolation of the corresponding template prediction region based on the pixel values of the first interpolation expansion region by interpolation filtering processing, as a prediction value of a template corresponding to the corresponding template prediction region.

[0180] It should be noted that the processing process at the encoding end is similar to the processing process at the decoding end, and details can be referred to the aforementioned processing process at the decoding end, and will not be repeated here.

[0181] The device embodiment of the present application will be described below, which can be used to execute the method described in the above embodiments of the present application. For details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application described above.

[0182] FIG. 22 shows a block diagram of a video decoding device according to an embodiment of the present application, which can be arranged in a device with computing processing function, such as a terminal device or a server.

[0183] Referring to FIG. 22, the video decoding device 2200 according to an embodiment of the present application includes a decoding unit 2202, a reference block determination unit 2204, a pixel value determination unit 2206, and a processing unit 2208.

[0184] The decoding unit 2202 is configured to receive a video bitstream, determine at least one candidate motion vector for inter-frame template matching of a current block based on the video bitstream, and construct a template of the current block, the template including reconstructed pixel points around the current block. The reference block determination unit 2204 is configured to determine at least one reference block of the current block in a reference frame according to the at least one candidate motion vector. The pixel value determination unit 2206 is configured to determine a template prediction region corresponding to each reference block in the reference frame based on a position of each reference block, each template prediction region having a same position relative to the corresponding reference block as the template has relative to the current block, determine pixel values in a first interpolation expansion region for interpolation filtering processing of the corresponding template prediction region based on pixel values in the template prediction region corresponding to each reference block. The processing unit 2208 is configured to determine sub-pixel interpolation of the corresponding template prediction region based on the pixel values of the first interpolation expansion region by interpolation filtering processing, as a prediction value of a template corresponding to the corresponding template prediction region.

[0185] In some embodiments of the present application, based on the foregoing scheme, the video decoding apparatus 2200 further comprises a searching unit configured to determine a best motion vector corresponding to the at least one candidate motion vector by inter-frame template matching based on the sub-pixel interpolation of the template prediction region corresponding to each reference block; and determine a matching block of the current block according to the best motion vector; and a filtering unit configured to perform interpolation filtering processing on the matching block.

[0186] In some embodiments of the present application, based on the foregoing scheme, the filtering unit is configured to determine pixel values in a second interpolation expansion region used for performing interpolation filtering processing in the motion compensation process according to pixel values in a set region in the reference frame; and perform interpolation filtering processing on the matching block based on the second interpolation expansion region.

[0187] In some embodiments of the present application, based on the foregoing scheme, the set region comprises at least one of the following regions: a first pixel position included in a search region in the reference frame, and a second pixel position in a reference block corresponding to the matching block.

[0188] In some embodiments of the present application, based on the foregoing scheme, a tap number of a first interpolation filter used for performing interpolation filtering processing on the template prediction region, and a tap number of a second interpolation filter used for performing interpolation filtering processing on the matching block are selected from the following tap numbers: 1 tap, 2 taps, 4 taps, 6 taps, 8 taps, 12 taps.

[0189] Wherein the tap number of the first interpolation filter and the tap number of the second interpolation filter are the same or different.

[0190] In some embodiments of the present application, based on the foregoing scheme, the first interpolation filter is a 12-tap interpolation filter, and the second interpolation filter is a 12-tap interpolation filter; or

[0191] The first interpolation filter is a 6-tap interpolation filter, and the second interpolation filter is a 12-tap interpolation filter.

[0192] In some embodiments of the present application, based on the foregoing scheme, the template prediction region comprises a pixel region above the reference block and a pixel region left of the reference block; wherein the first interpolation expansion region comprises at least one of the following regions:

[0193] A first number of rows of pixel regions above the template prediction region;

[0194] A second number of rows of pixel regions below the template prediction region;

[0195] a third number of column pixel regions located left of the template prediction region;

[0196] a fourth number of column pixel regions located right of the template prediction region;

[0197] a pixel region located top-left of the template prediction region;

[0198] a pixel region located top-right of the template prediction region;

[0199] a pixel region located bottom-left of the template prediction region;

[0200] a pixel region located bottom-right of the template prediction region.

[0201] In some embodiments of the present application, based on the foregoing scheme, the pixel value determination unit 2206 is configured to determine the pixel values in the first interpolation expansion region used for interpolation filtering in the inter-template matching process according to at least one of the following manners:

[0202] copying the pixel values in the template prediction region as the pixel values in the first interpolation expansion region;

[0203] symmetrically copying the pixel values in the template prediction region based on a symmetry axis to obtain the pixel values in the first interpolation expansion region;

[0204] symmetrically copying the pixel values in the template prediction region based on a symmetry point to obtain the pixel values in the first interpolation expansion region;

[0205] performing weighted calculation on the pixel values in the template prediction region to obtain the pixel values in the first interpolation expansion region;

[0206] performing intra prediction based on the pixel values in the template prediction region to obtain the pixel values in the first interpolation expansion region.

[0207] In some embodiments of the present application, based on the foregoing scheme, the processing unit 2208 is further configured to determine the tap number of the interpolation filter used in the inter-template matching process and the tap number of the interpolation filter used in the motion compensation process according to a manner negotiated with the encoding end; or

[0208] decode at least one flag bit from the video code stream, and determine the tap number of the interpolation filter used in the inter-template matching process and the tap number of the interpolation filter used in the motion compensation process according to the value of the at least one flag bit.

[0209] In some embodiments of the present application, based on the foregoing scheme, the at least one flag comprises one or more of the following flags: a flag included in a sequence header, a flag included in a picture header, a flag included in a slice header, a flag included in a CTU (coding tree unit) header, and a flag included in a coding block.

[0210] In some embodiments of the present application, based on the foregoing scheme, the interpolation filter used in the inter-frame template matching process uses the same number of taps in the horizontal direction and the vertical direction or different number of taps in the horizontal direction and the vertical direction; the interpolation filter used in the motion compensation process uses the same number of taps in the horizontal direction and the vertical direction or different number of taps in the horizontal direction and the vertical direction.

[0211] FIG. 23 shows a block diagram of a video encoding apparatus according to an embodiment of the present application, which can be arranged in a device with computing processing function, such as a terminal device or a server.

[0212] Referring to FIG. 23, a video encoding apparatus 2300 according to an embodiment of the present application includes a vector determination unit 2302, a reference block determination unit 2304, a pixel value determination unit 2306, and a processing unit 2308.

[0213] The vector determination unit 2302 is configured to obtain video data, the video data including a current block; determine at least one candidate motion vector for inter-frame template matching of the current block based on the video data; and construct a template of the current block, the template including pixel points around the current block. The reference block determination unit 2304 is configured to determine at least one reference block of the current block in a reference frame according to the at least one candidate motion vector. The pixel value determination unit 2306 is configured to determine a template prediction region corresponding to each of the at least one reference block in the reference frame based on a position of the each reference block, each template prediction region having a same position relative to the template as the position of the each reference block; and determine pixel values in a first interpolation expansion region for interpolation filtering of the each template prediction region based on pixel values in the each template prediction region. The processing unit 2308 is configured to determine sub-pixel interpolation of the each template prediction region based on the pixel values in the first interpolation expansion region by interpolation filtering, as a prediction value of a template corresponding to the each template prediction region.

[0214] FIG. 24 shows a structural schematic diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application, which can be the video encoding apparatus or the video decoding apparatus in the foregoing embodiments.

[0215] It should be noted that the computer system 2400 of the electronic device shown in FIG. 24 is merely one example. It should not be understood to limit the scope of the application embodiments in any way.

[0216] As shown in FIG. 24, the computer system 2400 can include a central processing unit (CPU) 2401 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 2402 or loaded into a random access memory (RAM) 2403 from a storage section 2408, such as performing the methods described in the above embodiments. Various programs and data required for the operation of the system are also stored in the RAM 2403. The CPU 2401, the ROM 2402, and the RAM 2403 are connected to each other through a bus 2404. An input / output (I / O) interface 2405 is also connected to the bus 2404.

[0217] The following components can be connected to the I / O interface 2405: an input section 2406 including a keyboard, a mouse, etc.; an output section 2407 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage section 2408 including a hard disk, etc.; and a communication section 2409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 2409 performs communication processing via a network such as the Internet. A drive 2410 is also connected to the I / O interface 2405 as necessary. A removable recording medium 2411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 2410 as necessary, so that a computer program read therefrom is installed into the storage section 2408 as necessary.

[0218] In particular, according to the embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments of the present application include a computer program product comprising a computer program for performing the methods shown in the flowcharts carried on a computer readable medium. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 2409, and / or installed from the removable recording medium 2411. When the computer program is executed by the central processing unit (CPU) 2401, various functions defined in the systems of the present application are performed.

[0219] It should be noted that the computer-readable medium in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a computer program that can be used by or in conjunction with an instruction execution system, device or apparatus. In this application, the computer-readable signal medium can include a data signal carrying computer-readable computer programs in a baseband or as a part of a carrier wave. Such a propagated data signal can take on various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium that can transmit, propagate or transport a program for use by or in connection with an instruction execution system, device or apparatus. The computer program contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination thereof.

[0220] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In the flowcharts or block diagrams, each block can represent a module, a program segment or a part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than that shown in the drawings. For example, two blocks that are shown in succession can actually be executed substantially in parallel, and sometimes in reverse order, depending on the involved functions. It should also be noted that each block in the block diagrams or flowcharts, and the combination of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer programs.

[0221] The units described in the embodiments of the present application can be implemented by software, or can be implemented by hardware, and the units described can also be arranged in a processor. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0222] As another aspect, the present application also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device. The computer readable medium carries one or more computer programs, which, when executed by the electronic device, enable the electronic device to implement the method described in the above embodiments.

[0223] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into several modules or units.

[0224] From the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes several instructions to enable an electronic device to perform the method according to the embodiments of the present application.

[0225] For example, the electronic device can be a video decoding apparatus, which can perform the video decoding method shown in FIG. 14; for another example, the electronic device can be a video encoding apparatus, which can perform the video encoding method shown in FIG. 21.

[0226] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the application following the general principles thereof and including such departures from the present disclosure as come within known use or custom in the art.

[0227] It is to be understood that the application is not limited to the precise construction already described above and shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should only be limited by the claims appended hereto.

Claims

1. A method of video decoding, performed in an electronic device, the method comprising: receiving a video bitstream; determining at least one candidate motion vector for inter-template matching of a current block based on the video bitstream; constructing a template of the current block, the template comprising reconstructed pixel points around the current block; determining at least one reference block of the current block in a reference frame according to the at least one candidate motion vector; determining, for each of the at least one reference block, a template prediction region corresponding to the reference block in the reference frame based on a position of the reference block, each template prediction region having a same position relative to the position of the corresponding reference block as the template has relative to the current block; determining, according to pixel values within each template prediction region, pixel values within a first interpolation extension region for interpolation filtering of the corresponding template prediction region; determining, based on the pixel values of the first interpolation extension region, a sub-pixel interpolation of the corresponding template prediction region by interpolation filtering as a prediction value of a template corresponding to the corresponding template prediction region. 2.The method of claim 1, further comprising: determining, based on the sub-pixel interpolation of each template prediction region, a best motion vector corresponding to the at least one candidate motion vector by inter-template matching; determining a matching block of the current block according to the best motion vector; performing interpolation filtering on the matching block.

3. The video decoding method of claim 2, wherein, The interpolation filtering on the matching block comprises: determining, according to pixel values within a set region in the reference frame, pixel values within a second interpolation extension region for interpolation filtering in a motion compensation process; performing interpolation filtering on the matching block based on the second interpolation extension region.

4. The video decoding method of claim 3, wherein, The set region comprises at least one of: a first pixel position included in a search region in the reference frame, a second pixel position in a reference block corresponding to the matching block.

5. The video decoding method of any of claims 2-4, wherein, A tap number of a first interpolation filter for the interpolation filtering on the template prediction region and a tap number of a second interpolation filter for the interpolation filtering on the matching block are selected from 1-tap, 2-tap, 4-tap, 6-tap, 8-tap, 12-tap respectively; wherein the tap number of the first interpolation filter and the tap number of the second interpolation filter are the same or different. 6.The method of claim 5, wherein: the first interpolation filter is a 12-tap interpolation filter and the second interpolation filter is a 12-tap interpolation filter; or the first interpolation filter is a 6-tap interpolation filter and the second interpolation filter is a 12-tap interpolation filter.

7. The video decoding method of any of claims 1-6, wherein, The template prediction region comprises a pixel region above the corresponding reference block and a pixel region left of the corresponding reference block; wherein the first interpolation extension region comprises at least one of: a first number of rows of pixel regions above the template prediction region; a second number of rows of pixel regions below the template prediction region; a third number of columns of pixel regions left of the template prediction region; a fourth number of pixel regions located to the right of the template prediction region; a pixel region located at the top-left of the template prediction region; a pixel region located at the top-right of the template prediction region; a pixel region located at the bottom-left of the template prediction region; a pixel region located at the bottom-right of the template prediction region.

8. The video decoding method of any of claims 1-7, wherein, The pixel values in the first interpolation expansion region used for the interpolation filtering of the corresponding template prediction region are determined according to the pixel values in the template prediction region corresponding to each reference block, including at least one of the following manners: copying the pixel values in the template prediction region as the pixel values in the first interpolation expansion region; copying the pixel values in the template prediction region based on a symmetry axis to obtain the pixel values in the first interpolation expansion region; copying the pixel values in the template prediction region based on a symmetry point to obtain the pixel values in the first interpolation expansion region; performing a weighted calculation on the pixel values in the template prediction region to obtain the pixel values in the first interpolation expansion region; performing an intra prediction on the pixel values in the template prediction region to obtain the pixel values in the first interpolation expansion region.

9. The video decoding method of any of claims 1-8, wherein, The method further comprises: determining the number of taps of the interpolation filter used in the inter-template matching and the number of taps of the interpolation filter used in the motion compensation according to a manner negotiated with an encoding end; or decoding at least one flag bit from the video code stream, and determining the number of taps of the interpolation filter used in the inter-template matching and the number of taps of the interpolation filter used in the motion compensation according to the value of the at least one flag bit.

10. The video decoding method of claim 9, wherein, The at least one flag bit includes one or more of the following flag bits: a flag bit included in a sequence header, a flag bit included in a picture header, a flag bit included in a slice header, a flag bit included in a coding tree unit (CTU) header, and a flag bit included in a coding block.

11. The video decoding method of claim 9, wherein, The number of taps used by the interpolation filter in the horizontal direction is the same as or different from the number of taps used by the interpolation filter in the vertical direction in the inter-template matching. The number of taps used by the interpolation filter in the horizontal direction is the same as or different from the number of taps used by the interpolation filter in the vertical direction in the motion compensation.

12. A video encoding method, executed in an electronic device, the method comprising: obtaining video data, the video data comprising a current block; determining at least one candidate motion vector for inter-template matching of the current block based on the video data; constructing a template of the current block, the template comprising pixel points around the current block; determining at least one reference block of the current block in a reference frame according to the at least one candidate motion vector; determining a template prediction region corresponding to each reference block in the reference frame based on the position of each reference block, each template prediction region having the same position relative to the corresponding reference block as the template has relative to the current block; determine, according to pixel values in a template prediction region corresponding to each reference block, pixel values in a first interpolation expansion region used for interpolation filtering processing on the corresponding template prediction region; determine, based on the pixel values in the first interpolation expansion region, sub-pixel interpolation of the corresponding template prediction region by interpolation filtering processing as a prediction value of a template corresponding to the corresponding template prediction region. 13.A video decoding apparatus comprising: a decoding unit configured to receive a video bitstream; determine, based on the video bitstream, at least one candidate motion vector for inter-template matching of a current block; construct a template of the current block, the template comprising reconstructed pixel points around the current block; a reference block determination unit configured to determine, according to the at least one candidate motion vector, at least one reference block of the current block in a reference frame; a pixel value determination unit configured to determine, based on a position of each reference block in the at least one reference block, a template prediction region corresponding to each reference block in the reference frame, each template prediction region having a same position relative to the corresponding reference block as the template has relative to the current block; and determine, according to pixel values in a template prediction region corresponding to each reference block, pixel values in a first interpolation expansion region used for interpolation filtering processing on the corresponding template prediction region; a processing unit configured to determine, based on the pixel values in the first interpolation expansion region, sub-pixel interpolation of the corresponding template prediction region by interpolation filtering processing as a prediction value of a template corresponding to the corresponding template prediction region. 14.A video encoding apparatus comprising: a vector determination unit configured to obtain video data, the video data comprising a current block; determine, based on the video data, at least one candidate motion vector for inter-template matching of the current block; and construct a template of the current block, the template comprising pixel points around the current block; a reference block determination unit configured to determine, according to the at least one candidate motion vector, at least one reference block of the current block in a reference frame; a pixel value determination unit configured to determine, according to the at least one candidate motion vector, at least one reference block of the current block in a reference frame; determine, based on a position of each reference block in the at least one reference block, a template prediction region corresponding to each reference block in the reference frame, each template prediction region having a same position relative to the corresponding reference block as the template has relative to the current block; and determine, according to pixel values in a template prediction region corresponding to each reference block, pixel values in a first interpolation expansion region used for interpolation filtering processing on the corresponding template prediction region; a processing unit configured to determine, based on the pixel values in the first interpolation expansion region, sub-pixel interpolation of the corresponding template prediction region by interpolation filtering processing as a prediction value of a template corresponding to the corresponding template prediction region. 15.A computer readable medium having stored thereon a computer program, the computer program being executed by a processor to implement the video decoding method of any one of claims 1 to 11, or to implement the video encoding method of claim 12.

16. An electronic device, comprising: one or more processors; a memory storing one or more computer programs that, when executed by the one or more processors, cause the electronic device to implement the video decoding method of any of claims 1-11 or implement the video encoding method of claim 12.

17. A computer program product, the computer program product comprising a computer program stored in a computer readable storage medium that, when read and executed by a processor of an electronic device, causes the electronic device to perform the video decoding method of any of claims 1-11 or implement the video encoding method of claim 12.

Images