Video processing method, video processing device and code stream generation method

Abstract

To provide a video processing method, a video processing device and a code stream generation method capable of simplifying encoding and decoding operation.SOLUTION: A video processing method capable of simplifying encoding and decoding operation by limiting the number of a reference frame lists that must be scanned in a bidirectional prediction process includes: determining a time motion vector of a current block; determining motion information of a subblock of the current block on the basis of the time motion vector; and performing interframe prediction of the current block on the basis of the motion information of the subblock of the current block.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] Copyright Statement The subject matter disclosed in this patent document contains material that is subject to copyright protection. Such copyright is owned by the Copyright Owner. The Copyright Owner does not object to the copying by anyone of this patent document or this patent disclosure, as it appears in the official records and files of the Patent and Trademark Office.

[0002] The present application relates to the field of video encoding and decoding, and more particularly to a video processing method, a video processing device and a codestream generation method. [Background technology]

[0003] The video encoding process includes an inter-frame prediction process. Inter-frame prediction modes include merge mode and non-merge mode. In merge mode, a merge mode motion vector candidate list must first be constructed, and a motion vector for the current block must be selected from the merge mode motion vector candidate list. The current block may also be referred to as a current coding unit (CU).

[0004] As coding technology advances, alternative / advanced temporal motion vector prediction (ATMVP) technology has been introduced into inter-frame prediction methods. In ATMVP, a current block is divided into multiple sub-blocks and motion information for each sub-block is calculated. ATMVP aims to improve overall video coding performance by introducing sub-block level motion vector prediction.

[0005] The process of finding the motion information of the sub-blocks of the current block using ATMVP technology is relatively complicated and contains some redundant operations, so there is still room for improvement in this process. Summary of the Invention [Means for solving the problem]

[0006] The present application provides a video processing method, a video processing device and a codestream generation method that can simplify encoding and decoding operations.

[0007] In a first aspect, a video processing method is provided, comprising: determining a motion vector of a spatial neighboring block at a specific position of a current block; and for bidirectional prediction, first scanning a reference frame list in a current reference direction of the current block; if a reference frame of the motion vector of the spatial neighboring block of the current block in the current reference direction is the same as a co-located frame of the current block, determining the motion vector of the spatial neighboring block as a temporal motion vector; if the reference frame of the motion vector of the spatial neighboring block in the current reference direction is different from the co-located frame of the current block, continuing scanning a reference frame list in another reference direction; if a reference frame of the motion vector of the spatial neighboring block in the other reference direction is the same as the co-located frame of the current block, determining the motion vector of the spatial neighboring block as the temporal motion vector; and if the reference frame of the motion vector of the spatial neighboring block in the other reference direction is different from the co-located frame of the current block, determining a zero motion vector as the temporal motion vector.

[0008] In a second aspect, a method includes: a memory for storing code; and a processor configured to execute the code stored in the memory, wherein the processor determines a motion vector of a spatial neighboring block at a specific position of a current block; for bidirectional prediction, first scans a reference frame list in a current reference direction of the current block; if a reference frame of the motion vector of the spatial neighboring block of the current block in the current reference direction is the same as a co-located frame of the current block, determines the motion vector of the spatial neighboring block as a temporal motion vector; and if the reference frame of the motion vector of the spatial neighboring block in the other reference direction is the same as the co-located frame of the current block, continue scanning a reference frame list in another reference direction; if the reference frame of the motion vector of the spatial neighboring block in the other reference direction is the same as the co-located frame of the current block, determine the motion vector of the spatial neighboring block as the temporal motion vector; and if the reference frame of the motion vector of the spatial neighboring block in the other reference direction is different from the co-located frame of the current block, determine a zero motion vector as the temporal motion vector.

[0009] In a third aspect, a code stream generation method is provided, the method including: determining a motion vector of a spatial neighboring block at a specific position of a current block; and for bidirectional prediction, first scanning a reference frame list in a current reference direction of the current block; if a reference frame of the motion vector of the spatial neighboring block of the current block in the current reference direction is the same as a co-located frame of the current block, determining the motion vector of the spatial neighboring block as a temporal motion vector; if the reference frame of the motion vector of the spatial neighboring block in the current reference direction is different from the co-located frame of the current block, continuing scanning a reference frame list in another reference direction; if a reference frame of the motion vector of the spatial neighboring block in the other reference direction is the same as the co-located frame of the current block, determining the motion vector of the spatial neighboring block as the temporal motion vector; and if the reference frame of the motion vector of the spatial neighboring block in the other reference direction is different from the co-located frame of the current block, determining a zero motion vector as the temporal motion vector.

[0010] In a fourth aspect, there is provided a video processing device comprising: a memory for storing code; and a processor for executing the code stored in the memory to perform the method of the first, second or third aspect.

[0011] In a fifth aspect, there is provided a computer readable storage medium having stored thereon instructions for carrying out the method of the first, second or third aspect.

[0012] In a sixth aspect, there is provided a computer program product comprising instructions for carrying out the method of the first, second or third aspect.

[0013] By limiting the number of reference frame lists that must be traversed in the bidirectional prediction process, the encoding and decoding operations can be simplified. [Brief explanation of the drawings]

[0014] [Figure 1] 10 is a flowchart for constructing an affine merge candidate list. [Figure 2] FIG. 1 is a schematic diagram of neighboring blocks of a current block. [Figure 3] 1 is a flowchart of the ATMVP implementation process. [Figure 4] 10A and 10B are diagrams illustrating examples of methods for obtaining motion information of sub-blocks of a current block. [Figure 5] 1 is a schematic diagram of the flow of a video processing method provided by an embodiment of the present application; [Figure 6] 1 is a structural schematic diagram of a video processing device provided by an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0015] This application can be applied to multiple video coding standards, including H.264, high efficiency video coding (HEVC), versatile video coding (VVC), audio video coding standard (AVS), AVS+, AVS2, and AVS3.

[0016] The video coding process mainly includes prediction, transformation, quantization, entropy coding, loop filtering, etc. Prediction is an important component of mainstream video coding technology. Prediction can be divided into intra-frame prediction and inter-frame prediction. Inter-frame prediction can be realized by motion compensation. The following describes the motion compensation process with an example.

[0017] For example, an image of one frame can be first divided into one or more coding regions. These coding regions can also be called coding tree units (CTUs). The size of a CTU can be, for example, 64 x 64 or 128 x 128 (unit: pixels; the unit will be omitted in similar descriptions below). Each CTU can be divided into square or rectangular image blocks. These image blocks can also be called coding units (CUs), and the current CU to be coded will be referred to as the current block hereinafter.

[0018] When performing inter-frame prediction on a current block, a similar block of the current block can be found in a reference frame (which may be a nearby reconstructed frame in the time domain) as a predicted block of the current block. The relative displacement between the current block and the similar block is called a motion vector (MV). The process of finding a similar block in the reference frame as a predicted block of the current block is called motion compensation.

[0019] Inter-frame prediction modes include merge mode and non-merge mode. In merge mode, the motion vector (MV) of an image block is the motion vector prediction (MVP) of the image block. Therefore, for merge mode, it is sufficient to transmit the index of the MVP and the index of the reference frame in the code stream. On the other hand, for non-merge mode, not only the MVP and the index of the reference frame must be transmitted in the code stream, but also the motion vector difference (MVD) must be transmitted in the code stream.

[0020] Conventional motion vectors employ a simple translation model, meaning that the motion vector of a current block represents the relative displacement between the current block and a reference block. This type of motion vector has difficulty accurately describing more complex motion situations in video, such as scaling, rotation, and perspective. To describe more complex motion situations, the affine model has been introduced into related coding and decoding standards. The affine model describes the affine motion field of the current block using motion vectors of two or three control points (CPs) of the current block. The two control points may be, for example, the upper left and upper right corners of the current block, and the three control points may be, for example, the upper left, upper right, and lower left corners of the current block.

[0021] The affine model is combined with the merge mode mentioned above to form the affine merge mode. Normally, the merge mode's motion vector candidate list contains the MVP of the image block, while the affine merge mode's motion vector candidate list contains the control point motion vector prediction (CPMVP). Similar to the regular merge mode, the affine merge mode does not require adding the MVD to the code stream; instead, the CPMVP is used as the CPMV of the current block.

[0022] The construction of the affine merge candidate list for the current block is one of the key processes in the affine merge mode. Figure 1 shows one possible construction method for the affine merge candidate list.

[0023] In step S110, the ATMVP is inserted into the affine merge candidate list of the current block.

[0024] The ATMVP includes motion information of the sub-blocks of the current block. In other words, when the ATMVP technique is adopted, an affine merge candidate list is inserted into the motion information of the sub-blocks of the current block, allowing the affine merge mode to perform motion compensation at the sub-block level, thereby improving the overall video coding performance. An embodiment of step S110 will be described in detail later with reference to Figure 3, but will not be described in detail here for the time being.

[0025] The motion information includes one or a combination of multiple types of information: a motion vector, a motion vector differential, a reference frame index value, a reference direction for inter-frame prediction, information on whether intra-frame coding or inter-frame coding is adopted for the image block, and a partition mode for the image block.

[0026] In step S120, the affine candidates to be inherited are inserted into the affine merge candidate list.

[0027] For example, as shown in FIG. 2, the neighboring blocks of the current block are scanned in the order of A1 → B1 → B0 → A0 → B2, and the CPMVs of the neighboring blocks in affine merge mode are adopted as affine candidates for the current block and inserted into the affine merge candidate list of the current block.

[0028] In step S130, it is determined whether the number of affine candidates in the affine merge candidate list is smaller than a predetermined value.

[0029] If the number of affine candidates in the affine merge candidate list has reached a predetermined value, the flow of FIG. 1 ends; if the number of affine candidates in the affine merge candidate list is smaller than the predetermined value, step S140 continues to be executed.

[0030] In step S140, the affine candidates to be configured are inserted into the affine merge candidate list.

[0031] For example, by combining motion information of blocks surrounding the current block, new affine candidates can be constructed, and the constructed and generated affine candidates can be inserted into the affine merge candidate list.

[0032] In step S150, it is determined whether the number of affine candidates in the affine merge candidate list is smaller than a predetermined value.

[0033] If the number of affine candidates in the affine merge candidate list has reached a predetermined value, the flow of FIG. 1 ends; if the number of affine candidates in the affine merge candidate list is smaller than the predetermined value, step S160 continues to be executed.

[0034] In step S160, a 0 vector is inserted into the affine merge candidate list.

[0035] In other words, the affine merge candidate list is padded with a zero vector to reach a predetermined value.

[0036] Hereinafter, an embodiment of step S110 in Fig. 1 will be described in detail with reference to Fig. 3. In some examples, the method of inserting an ATMVP into the affine merge candidate list of the current block described below may not be limited to the example shown in Fig. 1 above.

[0037] As shown in FIG. 3, the embodiment of the ATVMP technique, ie, the method of obtaining motion information of sub-blocks of a current block, can be roughly divided into two steps: steps S310 and S320.

[0038] In step S310, the corresponding block of the current block in the reference frame is determined.

[0039] In the current ATMVP technology, a frame for obtaining motion information of a current frame (the frame in which a current block is located) is called a co-located frame (co-located picture). The co-located frame of the current frame is set when a slice is initialized. Taking forward prediction as an example, the first reference frame list may be a forward reference frame list, or may be a reference frame list including a first set of reference frames. The first set of reference frames includes reference frames that are temporally before and after the current frame. When a slice is initialized, the first frame in the first reference frame list of the current block is usually set as the co-located frame of the current frame.

[0040] The corresponding block of the current block in the reference frame is determined by a temporal motion vector (temp MV). Therefore, to obtain the corresponding block of the current block in the reference frame, this temporal motion vector must be derived first. Next, the derivation process of the temporal motion vector will be described using forward prediction and bidirectional prediction as examples.

[0041] For forward prediction, the number of reference frame lists (which can also be referred to as reference lists or reference image lists) of the current block is 1. The reference frame list of the current block can be referred to as the first reference frame list (reference list 0). In one scenario, this first reference frame list may be a forward reference frame list. The co-located frame of the current frame is usually set to the first frame in the first reference frame list.

[0042] In the process of deriving a temporal motion vector, one embodiment is as follows: first, scan the motion vector candidate list of the current block (this motion vector candidate list can be constructed based on the motion vectors of the image blocks at four adjacent positions in the spatial domain), and select the first candidate motion vector in this motion vector candidate list as the initial temporal motion vector; then, scan the first reference frame list of the current block; if the reference frame of this first candidate motion vector is the same as the co-located frame of the current frame, this first candidate motion vector can be selected as the temporal motion vector; if the reference frame of this first candidate motion vector is different from the co-located frame of the current frame, the temporal motion vector can be set as a 0 vector, and the scanning can be stopped.

[0043] In this embodiment, a motion vector candidate list must be constructed to obtain the first candidate motion vector in the list. In another embodiment, the motion vector of a spatially neighboring block of the current block can be directly taken as the initial temporal motion vector. If the reference frame of the motion vector of this spatially neighboring block is the same as the co-located frame of the current frame, it can be used as the temporal motion vector; otherwise, the temporal motion vector can be set to a 0 vector and scanning can stop. Here, the spatially neighboring block can be any one of the coded blocks surrounding the current block. For example, it can be fixed to the block to the left of the current block, or the block above the current block, or the block to the upper left of the current block.

[0044] For bidirectional prediction, the number of reference frame lists for the current block is two, namely, including a first reference frame list (reference list 0) and a second reference frame list (reference list 1), where, in one scene, the first reference frame list may be a forward reference frame list and the second reference frame list may be a backward reference frame list.

[0045] In one embodiment, the process of deriving a temporal motion vector is as follows: first, scan a current motion vector candidate list, and select a first candidate motion vector in the motion vector candidate list as an initial temporal motion vector. Then, first scan one reference frame list (which may be a first reference frame list or a second reference frame list) of the current block in the current reference direction. If the reference frame of the first candidate motion vector is the same as the co-located frame of the current frame, the first candidate motion vector can be selected as a temporal motion vector. If the reference frame of the first candidate motion vector is different from the co-located frame of the current frame, the reference frame list of the current block in another reference direction is subsequently scanned. Similarly, if the reference frame of the first candidate motion vector in the other reference frame list is the same as the co-located frame of the current frame, the first candidate motion vector can be selected as a temporal motion vector. If the reference frame of the first candidate motion vector is different from the co-located frame of the current frame, the temporal motion vector can be set as a 0 vector and the scanning can be stopped. In addition, in some other situations, both the first reference frame list and the second reference frame list may include reference frames that are temporally before and after the current frame, and the bidirectional prediction refers to selecting reference frames with different reference directions from the first reference frame list and the second reference frame list.

[0046] In this embodiment, to derive the temp MV of ATMVP in bidirectional prediction, it is still necessary to construct a motion vector candidate list. In another embodiment, the motion vector of one spatially neighboring block of the current block can be directly taken as the initial temporal motion vector. For bidirectional prediction, first, one reference frame list (which may be the first reference frame list or the second reference frame list) of the current block in the current reference direction is scanned, and if the reference frame of the motion vector of this spatially neighboring block in this reference direction is the same as the co-located frame of the current frame, it can be taken as the temporal motion vector. Optionally, if the reference frame of the motion vector of this spatially neighboring block in this reference direction is different from the co-located frame of the current frame, the reference frame list of the current block in another reference direction is subsequently scanned. Similarly, if the reference frame of the motion vector of this spatial neighboring block in this other reference frame list is the same as the co-located frame of the current frame, the motion vector of this spatial neighboring block can be set as a temporal motion vector. If the reference frame of the motion vector of this spatial neighboring block is different from the co-located frame of the current frame, the temporal motion vector can be set to a 0 vector and scanning can be stopped. Here, the spatial neighboring block may be any one of the coded blocks surrounding the current block. For example, it may be fixed to the block to the left of the current block, fixed to the block above the current block, or fixed to the block above and to the left of the current block.

[0047] For bidirectional prediction, the scanning order of the first reference frame list and the second reference frame list may be determined by the following rules.

[0048] If the current frame is in a low delay encoding mode and the co-located frame of the current frame is set to the first frame in the second reference frame list, the second reference frame list is scanned first; otherwise, the first reference frame list is scanned first.

[0049] Here, if a low delay encoding mode is adopted for the current frame, it can be represented that the reference frames of the current frame are all before the current frame in the playback order in the video sequence, and if the co-located frame of the current frame is set to the first frame in the second reference frame list, it can be represented that the quantization step size of the first slice in the first reference frame list of the current frame is smaller than the quantization step size of the first slice in the second reference frame list.

[0050] After the temporal motion vector is derived, the corresponding block of the current block can be found in the reference frame using the temporal motion vector.

[0051] In step S320, motion information of the sub-blocks of the current block is obtained based on the corresponding blocks of the current block.

[0052] As shown in Figure 4, the current block can be divided into multiple sub-blocks, and then the motion information of the corresponding block of each sub-block can be determined. It is worth noting that for each sub-block, the motion information of the corresponding block can be determined by the minimum motion information storage unit where it is located.

[0053] The motion information includes one or a combination of multiple types of information: a motion vector, a motion vector differential, a reference frame index value, a reference direction for inter-frame prediction, information on whether intra-frame coding or inter-frame coding is adopted for the image block, and a partition mode for the image block.

[0054] From the ATMVP implementation process depicted in Figure 3, it can be seen that for bidirectional prediction, the worst case scenario is that in the process of deriving a temporal motion vector, even if both reference frame lists are scanned, a temporal motion vector that meets the conditions is still not derived, and in such a situation, scanning the two reference frame lists is redundant.

[0055] Also, in bidirectional prediction, if the encoding mode of the current frame is low delay mode (low delay B) or random access mode (random access), there may be some overlap in the reference frames in the first reference frame list and the second reference frame list, so in the process of obtaining the temporal motion vector, there may be redundant operations in the scanning process for the two reference frame lists.

[0056] Therefore, the temporal motion vector derivation schemes provided by the related art for bi-prediction are relatively complex and leave room for improvement.

[0057] Next, an embodiment of the present invention will be described in detail with reference to FIG.

[0058] 5 is a schematic flowchart of a video processing method provided by an embodiment of the present application, which can be applied to both the encoding side and the decoding side.

[0059] In step S510, obtain the reference frame list of the current block, which includes a first reference frame list and a second reference frame list.

[0060] The current block can also be referred to as a current CU. The reference frame list of the current block includes a first reference frame list and a second reference frame list, which indicates that the current block is going to perform inter-frame bidirectional prediction.

[0061] Optionally, the first reference frame list may be a forward reference frame list, which may be a reference frame list including a first set of reference frames, the first set of reference frames including reference frames that are temporally before and after the current frame.

[0062] Optionally, the second reference frame list may be a backward reference frame list, which may be a reference frame list including a second set of reference frames, the second set of reference frames including reference frames that are temporally before and after the current frame.

[0063] It is important to note that in some situations, both the first reference frame list and the second reference frame list may contain reference frames that are temporally before and after the current frame, and the bidirectional prediction may refer to selecting reference frames with different reference directions from the first reference frame list and the second reference frame list.

[0064] In step S520, a target reference frame list is determined based on the reference frame list of the current block.

[0065] The target reference frame list is one of a first reference frame list and a second reference frame list. This target reference frame list may be selected randomly or based on a certain rule. For example, it may be selected based on at least one of the following rules: if the current frame in which the current block is located adopts a low delay encoding mode and the co-located frame of the current frame is the first frame in the second reference frame list, the second reference frame list is determined as the target reference frame list; or if the current frame in which the current block is located does not adopt a low delay encoding mode or the co-located frame of the current frame is not the first frame in the second reference frame list, the first reference frame list is determined as the target reference frame list.

[0066] In step S530, a temporal motion vector of the current block is determined based on the target reference frame list of the current block.

[0067] In the bidirectional prediction process, the embodiment of the present application determines the temporal motion vector of the current block based on one of the first and second reference frame lists. In other words, regardless of whether a temporal motion vector can be derived from the target reference frame list, the scanning stops immediately after scanning the target reference frame list. In other words, the temporal motion vector of the current block can be determined based only on the target reference frame list.

[0068] For example, first, a first candidate motion vector can be selected from the current motion vector candidate list (this motion vector candidate list can be constructed based on the motion vectors of image blocks at four adjacent positions in the spatial domain), and the reference frame of the first candidate motion vector is searched for in the target reference frame list. If the reference frame of the first candidate motion vector is the same as the co-located frame of the current block, the first candidate motion vector can be determined as a temporal motion vector. If the reference frame of the first candidate motion vector is different from the co-located frame of the current block, the scanning is also stopped, and the other reference frame lists of the current block are not scanned continuously, as illustrated in the embodiment of Figure 3. In this situation, the 0 vector can be the temporal motion vector of the current block.

[0069] In step S540, the motion information of the sub-blocks of the current block is determined based on the temporal motion vector.

[0070] For example, a corresponding block of the current block in a reference frame may be determined based on a temporal motion vector. Then, motion information of a sub-block of the current block may be determined based on the corresponding block of the current block in the reference frame. The motion information may include a combination of one or more of the following: a motion vector, a motion vector differential, a reference frame index value, a reference direction for inter-frame prediction, information on whether the image block is coded using intra-frame coding or inter-frame coding, and a partition mode for the image block. Step S540 may be implemented by referring to step S320 described above, and will not be described again in detail here.

[0071] In step S550, inter-frame prediction is performed on the current block based on the motion information of the sub-blocks of the current block.

[0072] For example, step S550 may include performing inter-frame prediction based on motion information of the sub-blocks of the current block, with the sub-blocks of the current block as units.

[0073] For example, as shown in FIG. 1, motion information of sub-blocks of the current block is inserted as ATMVP into the affine merge candidates list of the current block, and then a complete affine merge candidates list can be constructed as shown in steps S120 to S160 in FIG. 1. Subsequently, the best candidate motion vector can be determined by performing inter-frame prediction on the current block using the candidate motion vectors in the affine merge candidates list. Detailed implementations of step S550 can be performed with reference to related art, and the embodiments of the present application are not limited thereto.

[0074] The embodiments of the present application can simplify the operations on the encoding and decoding side by limiting the number of reference frame lists that must be scanned in the bidirectional prediction process.

[0075] It can be understood that when the method of Fig. 5 is applied to the encoding side and the decoding side, there may be differences in the inter-frame prediction process for the current block described in step S550. For example, when the method of Fig. 5 is applied to the encoding side, performing inter-frame prediction on the current block may include determining a predicted block of the current block and calculating a residual block of the current block based on the initial block and the predicted block of the current block. Also, when the method of Fig. 5 is applied to the decoding side, performing inter-frame prediction on the current block may include determining a predicted block and a residual block of the current block and calculating a reconstructed block of the current block based on the predicted block and the residual block of the current block.

[0076] Above, an embodiment of the method of the present application has been described in detail with reference to Figures 1 to 5. Below, an embodiment of the apparatus of the present application will be described in detail with reference to Figure 6. The description of the embodiment of the method and the description of the embodiment of the apparatus correspond to each other, and therefore it should be understood that the previous embodiment of the method can be referenced for parts not described in detail.

[0077] 6 is a schematic structural diagram of a video processing device provided by an embodiment of the present application. The device 60 in FIG. 6 includes a memory 62 and a processor 64.

[0078] The memory 62 can be used to store the code.

[0079] The processor 64 can be used to execute the code stored in the memory to perform the following operations: obtain a reference frame list for a current block, including a first reference frame list and a second reference frame list; determine a target reference frame list, which is one of the first reference frame list and the second reference frame list, based on the reference frame list for the current block; determine a temporal motion vector for the current block based on the target reference frame list for the current block; determine motion information of a sub-block of the current block based on the temporal motion vector; and perform inter-frame prediction for the current block based on the motion information of the sub-block of the current block.

[0080] Optionally, determining motion information of a sub-block of the current block based on the temporal motion vector includes determining a corresponding block of the current block in a reference frame based on the temporal motion vector, and determining motion information of a sub-block of the current block based on a corresponding block of the current block in the reference frame.

[0081] Optionally, determining a target reference frame list based on the reference frame list of the current block includes at least one of: determining the second reference frame list as the target reference frame list if the current frame in which the current block is located adopts a low delay encoding mode and the co-located frame of the current frame is the first frame in the second reference frame list; and determining the first reference frame list as the target reference frame list if the current frame in which the current block is located does not adopt a low delay encoding mode or the co-located frame of the current frame is not the first frame in the second reference frame list.

[0082] Optionally, the first reference frame list may be a forward reference frame list, which may be a reference frame list including a first set of reference frames, the first set of reference frames including reference frames that are temporally before and after the current frame.

[0083] Optionally, the second reference frame list may be a backward reference frame list, which may be a reference frame list including a second set of reference frames, the second set of reference frames including reference frames that are temporally before and after the current frame.

[0084] It is important to note that in some situations, both the first reference frame list and the second reference frame list may contain reference frames that are temporally before and after the current frame, and the bidirectional prediction may refer to selecting reference frames with different reference directions from the first reference frame list and the second reference frame list.

[0085] Optionally, determining a temporal motion vector of the current block based on the target reference frame list of the current block includes: selecting a first candidate motion vector from the current motion vector candidate list; searching for a reference frame of the first candidate motion vector from the target reference frame list; and determining the first candidate motion vector as the temporal motion vector if the reference frame of the first candidate motion vector is the same as the co-located frame of the current block.

[0086] Optionally, determining a temporal motion vector of the current block based on the target reference frame list of the current block further includes determining the temporal motion vector to be a zero vector if the reference frame of the first candidate motion vector and the co-located frame of the current block are different.

[0087] Optionally, performing inter-frame prediction on the current block includes determining a predicted block of the current block, and calculating a residual block of the current block based on the initial block of the current block and the predicted block.

[0088] Optionally, performing inter-frame prediction on the current block includes determining a predicted block and a residual block of the current block, and calculating a reconstructed block of the current block based on the predicted block and the residual block of the current block.

[0089] Optionally, performing inter-frame prediction on the current block based on motion information of sub-blocks of the current block may include performing inter-frame prediction on a sub-block basis based on motion information of the sub-blocks of the current block.

[0090] The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, they may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded into a computer and executed, they generate all or part of the flows or functions described in the embodiments of the present invention. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored on a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, radio, microwave, etc.) methods. The computer-readable storage medium may be any available medium accessible by a computer, or may be a data storage device, such as a server or data center, that includes one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)).

[0091] Those skilled in the art can imagine that the units and algorithm steps of each example described in the embodiments disclosed herein can be combined and realized by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are ultimately implemented by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can realize the described functions using different methods for each specific application, but such realization should not be considered as departing from the scope of the present application.

[0092] In some embodiments provided herein, it should be understood that the disclosed systems, devices, and methods can be realized in other ways. For example, the device embodiments described above are merely schematic, and the division of the units is merely a logical and functional division. In actual implementation, there may be other division methods, for example, multiple units or assemblies may be combined or integrated into another system. Alternatively, some features may be ignored or not implemented. In other respects, the couplings or direct couplings or communication connections shown or discussed between each other may be indirect couplings or communication connections via some interfaces, devices, or units, which may be electrical, mechanical, or other types.

[0093] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed over multiple network units. Depending on actual needs, some or all of the units can be selected to achieve the objective of the solution of this embodiment.

[0094] Furthermore, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may physically exist alone, or two or more units may be integrated into one unit.

[0095] The above is merely a specific embodiment of the present application, and the scope of protection of the present application is not limited thereto. Anyone skilled in the art can easily think of modifications or replacements within the scope of the technology disclosed in the present application, and all of these must be included in the scope of protection of the present application. Therefore, the scope of protection of the present application must be in accordance with the scope of protection of the claims.

Claims

1. 1. A video processing method comprising: determining a motion vector of a spatially neighboring block at a specific position of the current block; For bidirectional prediction, first scanning a reference frame list in a current reference direction of the current block; determining the motion vector of the spatial neighboring block as a temporal motion vector if the reference frame of the motion vector of the spatial neighboring block of the current block in the current reference direction is the same as the co-located frame of the current block; If the reference frame of the motion vector of the spatial neighboring block in the current reference direction is different from the co-located frame of the current block, continuing to scan a reference frame list in another reference direction; If the reference frame of the motion vector of the spatial neighboring block in another reference direction is the same as the co-located frame of the current block, determining the motion vector of the spatial neighboring block as the temporal motion vector; and determining a zero motion vector as the temporal motion vector if the reference frame of the motion vector of the spatial neighboring block in another reference direction is different from the co-located frame of the current block.

2. 2. A video processing method according to claim 1, comprising: determining a corresponding block of the current block in a reference frame based on the temporal motion vector; determining motion information of sub-blocks of the current block based on the corresponding blocks of the current block in the reference frame.

3. 2. A video processing method according to claim 1, comprising: The spatial neighboring block at the specific position of the current block is a left block, an upper block, or an upper-left block of the current block.

4. A video processing method according to any one of claims 1 to 3, comprising the steps of: determining a prediction block of the current block; calculating a residual block of the current block based on an initial block of the current block and the predicted block.

5. A video processing method according to any one of claims 1 to 3, comprising the steps of: determining a prediction block and a residual block of the current block; calculating a reconstructed block of the current block based on the predicted block and the residual block of the current block.

6. A video processing method according to any one of claims 1 to 3, comprising the steps of: The video processing method further includes performing inter-frame prediction on the current block in sub-block units based on motion information of sub-blocks of the current block.

7. 2. A video processing method according to claim 1, comprising: inserting a candidate into a candidate list of the current block, the candidate indicating determining motion information of a sub-block of the current block using the temporal motion vector; inserting a successor affine candidate into the candidate list for the current block; inserting an affine candidate to be constructed into the candidate list in response to a number of candidates in the candidate list being less than a set value, the affine candidate to be constructed being obtained by combining motion information of neighboring blocks of the current block; After inserting the constructed affine candidates, inserting a zero vector into the candidate list in response to a number of candidates in the candidate list being less than the set value.

8. 8. A video processing method according to claim 7, comprising: A video processing method, wherein inserting the successor affine candidate into the candidate list of the current block includes inserting a control point motion vector of an adjacent block of the current block into the candidate list of the current block as the affine candidate of the current block.

9. 1. A video processing device comprising: a memory for storing the code; a processor configured to execute the code stored in the memory, the processor comprising: determining a motion vector of a spatially neighboring block at a specific position of the current block; For bidirectional prediction, first scanning a reference frame list in a current reference direction of the current block; determining the motion vector of the spatial neighboring block as a temporal motion vector if the reference frame of the motion vector of the spatial neighboring block of the current block in the current reference direction is the same as the co-located frame of the current block; If the reference frame of the motion vector of the spatial neighboring block in the current reference direction is different from the co-located frame of the current block, continuing to scan a reference frame list in another reference direction; If the reference frame of the motion vector of the spatial neighboring block in another reference direction is the same as the co-located frame of the current block, determining the motion vector of the spatial neighboring block as the temporal motion vector; and determining a zero motion vector as the temporal motion vector if the reference frame of the motion vector of the spatial neighboring block in another reference direction is different from the co-located frame of the current block.

10. 1. A codestream generation method, comprising: determining a motion vector of a spatially neighboring block at a specific position of the current block; For bidirectional prediction, firstly, scanning a reference frame list in a current reference direction of the current block; determining the motion vector of the spatial neighboring block as a temporal motion vector if the reference frame of the motion vector of the spatial neighboring block of the current block in the current reference direction is the same as the co-located frame of the current block; If the reference frame of the motion vector of the spatial neighboring block in the current reference direction is different from the co-located frame of the current block, continuing to scan a reference frame list in another reference direction; If the reference frame of the motion vector of the spatial neighboring block in another reference direction is the same as the co-located frame of the current block, determining the motion vector of the spatial neighboring block as the temporal motion vector; and determining a zero motion vector as the temporal motion vector if the reference frame of the motion vector of the spatial neighboring block in another reference direction is different from the co-located frame of the current block.

Images