Video processing method, video processing device, and code stream generation method
By limiting the number of reference frame lists scanned and using time or zero motion vectors, the complexity of motion information search in video encoding is reduced, improving encoding efficiency.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- SZ DJI TECH CO LTD
- Filing Date
- 2025-09-03
- Publication Date
- 2026-06-09
AI Technical Summary
The process of searching for motion information of sub-blocks using advanced temporal motion vector prediction (ATMVP) technology in video encoding is complex and includes redundant operations, necessitating improvements.
A method is introduced to simplify encoding and decoding operations by limiting the number of reference frame lists scanned during bidirectional prediction, determining the motion vector of spatially adjacent blocks, and using a time motion vector or zero motion vector based on frame comparisons.
This approach simplifies the encoding and decoding processes by reducing redundant operations, enhancing video encoding performance and efficiency.
Smart Images
Figure 0007872426000001 
Figure 0007872426000002 
Figure 0007872426000003
Abstract
Description
Technical Field
[0001] Copyright Statement The content disclosed in this patent document includes materials protected by copyright. This copyright is owned by the copyright owner. The copyright owner does not oppose any person from reproducing this patent document or this patent disclosure existing in the official records and documents of the Patent and Trademark Office.
[0002] This application relates to the field of video encoding and decoding, and more specifically, to a video processing method, a video processing apparatus, and a code stream generation method.
Background Art
[0003] The video encoding process includes an inter-frame prediction process. The modes of inter-frame prediction include a merge mode and a non-merge mode. In the merge mode, usually, first, a motion vector candidate list for the merge mode is constructed, and the motion vector of the current block must be selected from the motion vector candidate list for the merge mode. The current block may also be referred to as the current coding unit (CU).
[0004] As encoding technology develops, alternative / advanced temporal motion vector prediction (ATMVP) technology has been introduced into the inter-frame prediction method. In ATMVP technology, the current block is divided into a plurality of sub-blocks, and the motion information of the sub-blocks is calculated. The purpose of ATMVP technology is to improve the overall encoding performance of video by introducing motion vector prediction at the sub-block level.
[0005] The process of searching for the motion information of the sub-blocks of the current block using ATMVP technology is relatively complex, there are some redundant operations, and there is still room for improvement in this process.
Summary of the Invention
[0006] This application provides a video processing method, a video processing device, and a code stream generation method that can simplify encoding and decoding operations.
[0007] In a first embodiment, a video processing method is provided, which includes: determining the motion vector of a spatially adjacent block at a specific position of the current block; first scanning a list of reference frames in the current reference direction of the current block for bidirectional prediction; determining the motion vector of the spatially adjacent block as a time motion vector if the reference frame of the motion vector of the spatially adjacent block in the current reference direction is the same as the collated frame of the current block; continuing to scan a list of reference frames in another reference direction if the reference frame of the motion vector of the spatially adjacent block in the current reference direction is different from the collated frame of the current block; determining the motion vector of the spatially adjacent block as a time motion vector if the reference frame of the motion vector of the spatially adjacent block in another reference direction is the same as the collated frame of the current block; and determining a zero motion vector as a time motion vector if the reference frame of the motion vector of the spatially adjacent block in another reference direction is different from the collated frame of the current block.
[0008] In a second embodiment, the system includes a memory for storing code and a processor configured to execute the code stored in the memory, wherein the processor determines the motion vector of a spatially adjacent block at a specific location of the current block, first scans a list of reference frames in the current reference direction of the current block for bidirectional prediction, determines the motion vector of the spatially adjacent block as a time motion vector if the reference frame of the motion vector of the spatially adjacent block of the current block in the current reference direction is the same as the collated frame of the current block, and the motion vector of the spatially adjacent block in the current reference direction A video processing device is provided that is configured to perform the following actions: if the reference frame of the block is different from the collated frame of the current block, continue scanning the reference frame list in a different reference direction; if the reference frame of the motion vector of the spatially adjacent block in a different reference direction is the same as the collated frame of the current block, determine the motion vector of the spatially adjacent block as the time motion vector; and if the reference frame of the motion vector of the spatially adjacent block in a different reference direction is different from the collated frame of the current block, determine the zero motion vector as the time motion vector.
[0009] In a third aspect, a code stream generation method is provided, which includes: determining the motion vector of a spatially adjacent block at a specific position of the current block; first scanning a list of reference frames in the current reference direction of the current block for bidirectional prediction; determining the motion vector of the spatially adjacent block as a time motion vector if the reference frame of the motion vector of the spatially adjacent block in the current reference direction is the same as the collated frame of the current block; continuing to scan a list of reference frames in another reference direction if the reference frame of the motion vector of the spatially adjacent block in the current reference direction is different from the collated frame of the current block; determining the motion vector of the spatially adjacent block as a time motion vector if the reference frame of the motion vector of the spatially adjacent block in another reference direction is the same as the collated frame of the current block; and determining a zero motion vector as a time motion vector if the reference frame of the motion vector of the spatially adjacent block in another reference direction is different from the collated frame of the current block.
[0010] In a fourth aspect, a video processing device is provided, which includes a memory for storing code and a processor for executing the code stored in the memory to perform the method in the first, second, or third aspect.
[0011] In a fifth aspect, a computer-readable storage medium is provided that stores instructions for performing the method in the first, second, or third aspect.
[0012] In a sixth aspect, a computer program product is provided which includes instructions for performing the method in the first, second, or third aspect.
[0013] By limiting the number of reference frame lists that must be scanned in the bidirectional prediction process, the encoding and decoding operations can be simplified. [Brief explanation of the drawing]
[0014] [Figure 1] This is a flowchart that shows how to create an affine merge candidate list. [Figure 2] This is a schematic diagram of the surrounding blocks of the current block. [Figure 3] This is a flowchart of the ATMVP implementation process. [Figure 4] This diagram shows an example of how to obtain movement information for subblocks within a block. [Figure 5] This is a schematic diagram of the video processing method provided by the embodiment of the present invention. [Figure 6] This is a schematic diagram of the structure of the video processing device provided in the embodiment of the present invention. [Modes for carrying out the invention]
[0015] This invention can be applied to multiple video coding standards, including H.264, high efficiency video coding (HEVC), versatile video coding (VVC), audio video coding standard (AVS), AVS+, AVS2, and AVS3.
[0016] The video coding process primarily involves parts such as prediction, transformation, quantization, entropy coding, and loop filtering. Prediction is a crucial component of mainstream video coding technologies. Prediction can be divided into intra-frame prediction and inter-frame prediction. Inter-frame prediction can be implemented using motion compensation methods. The motion compensation process will be explained below with examples.
[0017] For example, a single frame of an image can first be divided into one or more coding regions. These coding regions can also be called coding tree units (CTUs). The size of a CTU may be, for example, 64x64 or 128x128 (the unit is pixels; units will be omitted in similar descriptions thereafter). Each CTU can be divided into a square or rectangular image block. These image blocks can also be called coding units (CUs), and hereafter, the current CU to be coded will be referred to as the current block.
[0018] When performing inter-frame prediction on the current block, a similar block to the current block can be found in a reference frame (which may be a reconstructed frame in the time domain's vicinity) as the predicted block for the current block. The relative displacement between the current block and the similar block is called the motion vector (MV). The process of finding a similar block as the predicted block for the current block within the reference frame is what is known as motion compensation.
[0019] Interframe prediction modes include merge mode and non-merge mode. In merge mode, the motion vector (MV) of an image block is equivalent to the motion vector prediction (MVP) of the image block. Therefore, for merge mode, it is sufficient to transmit the index of the MVP and the index of the reference frame within the coded stream. In contrast, for non-merge mode, not only must the MVP and the index of the reference frame be transmitted within the coded stream, but the motion vector difference (MVD) must also be transmitted within the coded stream.
[0020] The conventional motion vectors adopt a simple translation model. That is, the motion vector of the current block represents the relative displacement between the current block and the reference block. It is difficult for such a type of motion vector to accurately describe more complex motion situations in the video, such as scaling, rotation, and perspective. To be able to describe more complex motion situations, an affine model (affine model) has been introduced into the related encoding and decoding standards. The affine model describes the affine motion field of the current block by using the motion vectors of two or three control points (control point, CP) of the current block. These two control points may be, for example, the upper left corner and the upper right corner of the current block, and these three control points may be, for example, the upper left corner, the upper right corner, and the lower left corner of the current block.
[0021] Combining the affine model with the aforementioned merge mode forms the affine merge mode. Generally, the MVP of the image block is recorded in the motion vector candidate list (merge candidate list) of the merge mode, and the control point motion vector prediction (CPMVP) is recorded in the motion vector candidate list (affine merge candidate list) of the affine merge mode. Similar to the ordinary merge mode, the affine merge mode does not need to add the MVD to the symbol stream and directly uses the CPMVP as the CPMV of the current block.
[0022] The construction of the affine merge candidate list of the current block is one of the important processes of the affine merge mode. Figure 1 shows one possible construction method of the affine merge candidate list (affine merge candidate list).
[0023] In step S110, the ATMVP is inserted into the affine merge candidate list of the current block.
[0024] What is included in the ATMVP is the motion information of the sub-blocks of the current block. In other words, when adopting the ATMVP technology, the affine merge candidate list is inserted into the motion information of the sub-blocks of the current block, and by enabling the affine merge mode to perform motion compensation at the sub-block level, the overall video encoding performance can be improved. The embodiment of step S110 will be described in detail later while referring to FIG. 3, but will not be elaborated here for the time.
[0025] The motion information includes a combination of one or more types of information among a motion vector, a motion vector difference, a reference frame index value, a reference direction for inter-frame prediction, information on adopting intra-frame encoding or inter-frame encoding for an image block, and a partitioning mode of the image block.
[0026] In step S120, the inherited affine candidates are inserted into the affine merge candidate list.
[0027] For example, as shown in FIG. 2, scan the peripheral blocks of the current block in the order of A1→B1→B0→A0→B2, adopt the CPMV of the peripheral blocks in the affine merge mode as the affine candidates of the current block, and insert them into the affine merge candidate list of the current block.
[0028] In step S130, it is determined whether the number of affine candidates in the affine merge candidate list is smaller than a predetermined value.
[0029] If the number of affine candidates in the affine merge candidate list reaches a predetermined value, the flow in Figure 1 is terminated. If the number of affine candidates in the affine merge candidate list is less than the predetermined value, step S140 is executed.
[0030] Step S140 inserts the constituent affine candidates into the affine merge candidate list.
[0031] For example, by combining the movement information of the surrounding blocks of the current block, new affine candidates can be constructed, and these constructed and generated affine candidates can be inserted into the affine merge candidate list.
[0032] In step S150, it is determined whether the number of affine candidates in the affine merge candidate list is less than a predetermined value.
[0033] If the number of affine candidates in the affine merge candidate list reaches a predetermined value, the flow in Figure 1 is terminated. If the number of affine candidates in the affine merge candidate list is less than the predetermined value, step S160 is executed again.
[0034] In step S160, a 0 vector is inserted into the affine merge candidate list.
[0035] In other words, the affine merge candidate list is padded using a zero vector until it reaches a predetermined value.
[0036] The embodiments of step S110 in Figure 1 will be described in detail below with reference to Figure 3. In some examples, the method of inserting ATMVP into the current block's affine merge candidate list, as described below, is not limited to the embodiment shown in Figure 1 above.
[0037] As shown in Figure 3, the embodiment of the ATVMP technology, that is, the method for acquiring motion information of subblocks of the current block, can be broadly divided into two steps, S310 and S320.
[0038] In step S310, the corresponding block in the reference frame for the current block is determined.
[0039] In current ATMVP technology, the frame used to obtain motion information for the current frame (the frame in which the current block is located) is called a co-located picture. The current frame's co-located picture is set during slice initialization. Taking forward prediction as an example, the first reference frame list may be a forward reference frame list, or it may be a reference frame list containing a first set of reference frames. The first set of reference frames includes reference frames whose temporal order is before and after the current frame. During slice initialization, the first frame in the current block's first reference frame list is usually set as the current frame's co-located picture.
[0040] The corresponding block in the reference frame for the current block is determined by a single time motion vector (temp MV). Therefore, to obtain the corresponding block in the reference frame for the current block, this time motion vector must first be derived. Next, the process of deriving the time motion vector will be explained using forward prediction and bidirectional prediction as examples.
[0041] For forward prediction, the current block's reference frame list (which can also be called the reference list or reference image list) has 1 element. The current block's reference frame list can be called the first reference frame list (reference list 0). In one scenario, this first reference frame list may be the forward reference frame list. The current frame's collocated frame is usually set to the first frame in the first reference frame list.
[0042] In the process of deriving the time motion vector, one embodiment is as follows: First, a list of motion vector candidates for the current block (this list of motion vector candidates can be constructed based on the motion vectors of four adjacent image blocks in the spatial region) is scanned, and the first candidate motion vector in this list is taken as the initial time motion vector. Then, the first reference frame list for the current block is scanned, and if the reference frame of this first candidate motion vector is the same as the collocated frame of the current frame, this first candidate motion vector can be taken as the time motion vector; if the reference frame of this first candidate motion vector is different from the collocated frame of the current frame, the time motion vector can be set to the 0 vector and the scan can be stopped.
[0043] In this embodiment, the first candidate motion vector in the motion vector candidate list must be obtained by constructing a motion vector candidate list. In another embodiment, the motion vector of one spatially adjacent block of the current block can be taken directly as the initial time motion vector. If the reference frame of this spatially adjacent block's motion vector is the same as the collocated frame of the current frame, it can be used as the time motion vector; otherwise, the time motion vector can be set to the 0 vector and scanning can be stopped. Here, the spatially adjacent block may be any one of the encoded blocks around the current block. For example, it may be fixed to the left block of the current block, or to the upper-left block of the current block, or to the upper-left block of the current block.
[0044] For bidirectional prediction, the current number of reference frame lists in a block is two, namely, a first reference frame list (reference list 0) and a second reference frame list (reference list 1). In one scenario, the first reference frame list may be a forward reference frame list, and the second reference frame list may be a backward reference frame list.
[0045] In the process of deriving the time motion vector, one embodiment is as follows: First, the current motion vector candidate list is scanned, and the first candidate motion vector in this motion vector candidate list is taken as the initial time motion vector. Then, first, one reference frame list (which may be the first or second reference frame list) in the current reference direction of the current block is scanned, and if the reference frame of this first candidate motion vector is the same as the collocated frame of the current frame, this first candidate motion vector can be taken as the time motion vector. If the reference frame of this first candidate motion vector is different from the collocated frame of the current frame, the reference frame list in the other reference direction of the current block is scanned. Similarly, if the reference frame of the first candidate motion vector in this other reference frame list is the same as the collocated frame of the current frame, this first candidate motion vector can be taken as the time motion vector. If the reference frame of this first candidate motion vector is different from the collocated frame of the current frame, the time motion vector can be set to the 0 vector and the scanning can be stopped. In addition, in several other scenarios, both the first and second reference frame lists may include reference frames whose temporal order is before and after the current frame, and the bidirectional prediction refers to selecting reference frames with different reference directions from the first and second reference frame lists.
[0046] In this embodiment, to derive the temp MV of the ATMVP in bidirectional prediction, it is still necessary to construct a list of motion vector candidates. In another embodiment, the motion vector of one spatially adjacent block in the current block can be directly taken as the initial time motion vector. For bidirectional prediction, first, one reference frame list (which may be a first or second reference frame list) in the current reference direction of the current block is scanned, and if the reference frame of the motion vector of this spatially adjacent block in this reference direction is the same as the collated frame of the current frame, it can be taken as the time motion vector. Optionally, if the reference frame of the motion vector of this spatially adjacent block in this reference direction is different from the collated frame of the current frame, the reference frame list in another reference direction of the current block is then scanned. Similarly, if the reference frame of the motion vector of this spatially adjacent block in this other reference frame list is the same as the collocated frame of the current frame, then the motion vector of this spatially adjacent block can be the time motion vector. If the reference frame of the motion vector of this spatially adjacent block is different from the collocated frame of the current frame, then the time motion vector can be set to the 0 vector and scanning can be stopped. Here, the spatially adjacent block may be any one of the encoded blocks around the current block. For example, it may be fixed to the block to the left of the current block, or to the block above the current block, or to the upper left of the current block.
[0047] For bidirectional prediction, the traversal order of the first and second reference frame lists can be determined by the following rules.
[0048] If the frame is currently using low-delay encoding mode, and the current frame's collated frame is set as the first frame in the second reference frame list, then the second reference frame list is scanned first; otherwise, the first reference frame list is scanned first.
[0049] Here, if a low-delay encoding mode is used for the current frame, it can be said that the playback order of all the reference frames of the current frame in the video sequence precedes the current frame. If the collated frame of the current frame is set to the first frame in the second reference frame list, it can be said that the quantization step size of the first slice in the first reference frame list of the current frame is smaller than the quantization step size of the first slice in the second reference frame list.
[0050] After deriving the time motion vector, this time motion vector can be used to find the corresponding block of the current block within the reference frame.
[0051] In step S320, the movement information of the subblocks of the current block is obtained based on the corresponding block of the current block.
[0052] As shown in Figure 4, the current block can be divided into multiple subblocks, and then the motion information of the subblocks in their corresponding blocks can be determined. It is worth noting that for each individual subblock, the motion information of its corresponding block can be determined by the smallest motion information storage unit in which it is located.
[0053] The motion information includes a combination of one or more types of information, including motion vectors, motion vector differences, reference frame index values, reference direction for inter-frame prediction, information on whether to employ intra-frame or inter-frame coding for image blocks, and image block segmentation modes.
[0054] From the ATMVP implementation process depicted in Figure 3, the worst-case scenario for bidirectional prediction is that, in the process of deriving the time-motion vector, even after scanning both reference frame lists, a time-motion vector that satisfies the conditions is still not derived. In such a situation, scanning both reference frame lists is redundant.
[0055] Furthermore, in bidirectional prediction, if the encoding mode of the current frame is low-latency mode (low delay B) or random access mode, there may be some overlap in the reference frames in the first and second reference frame lists. Therefore, in the process of obtaining the time-motion vector, redundant operations may exist in the scanning process of the two reference frame lists.
[0056] Therefore, the proposed time-motion vector derivations provided by related technologies for bidirectional prediction are relatively complex and have room for improvement.
[0057] Next, the embodiments of the present application will be described in detail with reference to Figure 5.
[0058] Figure 5 is a schematic flowchart of the video processing method provided by the embodiment of the present invention. The method in Figure 5 can be applied to both the encoding and decoding sides.
[0059] In step S510, the reference frame list for the current block is obtained, which includes the first reference frame list and the second reference frame list.
[0060] The current block can also be called the current CU. The current block's reference frame list includes a first reference frame list and a second reference frame list, indicating that the current block is attempting to perform bidirectional prediction between frames.
[0061] Optionally, the first reference frame list may be a forward reference frame list, which may be a reference frame list containing a first set of reference frames, the first set of reference frames containing reference frames whose temporal order is before the current frame and after the current frame.
[0062] Optionally, the second reference frame list may be a backreference frame list, which may be a reference frame list containing a second set of reference frames, the second set of reference frames containing reference frames whose temporal order is before the current frame and after the current frame.
[0063] It is important to note that in some cases, both the first and second reference frame lists may contain reference frames whose temporal order is before and after the current frame, and the bidirectional prediction may indicate that reference frames with different referencing directions are being selected from the first and second reference frame lists.
[0064] In step S520, the target reference frame list is determined based on the reference frame list of the current block.
[0065] The target reference frame list is either the first reference frame list or the second reference frame list. This target reference frame list may be selected randomly or based on certain rules. For example, it may be selected based on the following rules: if the current frame in which the current block is located employs a low-latency coding mode and the collated frame of the current frame is the first frame in the second reference frame list, then the second reference frame list is determined to be the target reference frame list; or if the current frame in which the current block is located does not employ a low-latency coding mode, or the collated frame of the current frame is not the first frame in the second reference frame list, then the first reference frame list is determined to be the target reference frame list.
[0066] In step S530, the time motion vector of the current block is determined based on the target reference frame list of the current block.
[0067] In the bidirectional prediction process, the embodiment of the present invention determines the time motion vector of the current block based on one of the two reference frame lists, the first reference frame list and the second reference frame list. In other words, regardless of whether or not the time motion vector can be derived from the target reference frame list, the scanning is stopped immediately after scanning the target reference frame list is complete. In other words, the time motion vector of the current block can be determined based solely on the target reference frame list.
[0068] To illustrate with an example, first, a candidate motion vector can be selected from the current motion vector candidate list (this motion vector candidate list can be constructed based on the motion vectors of four adjacent image blocks in the spatial domain). Then, the reference frame of the first candidate motion vector is searched from the target reference frame list. If the reference frame of the first candidate motion vector is the same as the collated frame of the current block, the first candidate motion vector can be determined as the time motion vector. If the reference frame of the first candidate motion vector is different from the collated frame of the current block, the scan is stopped, and the system does not continue scanning another reference frame list of the current block, as depicted in the embodiment of Figure 3. In this situation, the 0 vector can be the time motion vector of the current block.
[0069] In step S540, the motion information of the subblock of the current block is determined based on the time motion vector.
[0070] For example, based on the time motion vector, the corresponding block in the reference frame of the current block can be determined. Then, based on the corresponding block in the reference frame of the current block, motion information of the subblocks of the current block can be determined. The motion information includes a combination of one or more types of information from the motion vector, motion vector difference, reference frame index value, reference direction of interframe prediction, information on whether to employ intraframe coding or interframe coding for the image block, and the division mode of the image block. Step S540 can be implemented by referring to step S320 above, which will not be described in detail again here.
[0071] In step S550, inter-frame predictions are made for the current block based on the movement information of the sub-blocks of the current block.
[0072] For example, step S550 may include performing inter-frame predictions based on the movement information of the subblocks of the current block, using the subblocks of the current block as the unit.
[0073] For example, as shown in Figure 1, the motion information of the subblock of the current block can be inserted as ATMVP into the affine merge candidates list of the current block, and then a complete affine merge candidates list can be constructed as shown in steps S120 to S160 in Figure 1. Subsequently, the best candidate motion vector can be determined by performing interframe prediction on the current block using the candidate motion vectors in this affine merge candidates list. Detailed embodiments of step S550 can be performed by referring to related technologies, and the embodiments of this application are not limited thereto.
[0074] The embodiments of this invention can simplify the coding and decoding operations by limiting the number of reference frame lists that must be scanned in the bidirectional prediction process.
[0075] What is understandable is that when applying the method in Figure 5 to the encoding side and the decoding side, there may be differences in the inter-frame prediction process for the current block described in step S550. For example, when applying the method in Figure 5 to the encoding side, performing inter-frame prediction for the current block may include determining the predicted block for the current block and calculating the residual block for the current block based on the initial block and predicted block for the current block. Similarly, when applying the method in Figure 5 to the decoding side, performing inter-frame prediction for the current block may include determining the predicted block and residual block for the current block and calculating the reconstructed block for the current block based on the predicted block and residual block for the current block.
[0076] The embodiments of the method of this application have been described in detail above with reference to Figures 1 to 5. The embodiments of the apparatus of this application will now be described in detail below with reference to Figure 6. The descriptions of the method embodiments and the apparatus embodiments correspond to each other; therefore, it should be understood that for parts not described in detail, the embodiments of the method described above can be referenced.
[0077] Figure 6 is a schematic diagram of a video processing device provided by an embodiment of the present invention. The device 60 in Figure 6 comprises a memory 62 and a processor 64.
[0078] Memory 62 can be used to store code.
[0079] The processor 64 can be used to perform the following operations by executing the code stored in the memory: obtain a reference frame list of the current block, which includes a first reference frame list and a second reference frame list; determine a target reference frame list, which is one of the first reference frame list and the second reference frame list, based on the reference frame list of the current block; determine the time motion vector of the current block based on the target reference frame list of the current block; determine the motion information of the subblocks of the current block based on the time motion vector; and perform inter-frame prediction for the current block based on the motion information of the subblocks of the current block.
[0080] Optionally, determining motion information of a subblock of the current block based on the time motion vector includes determining the corresponding block of the current block in the reference frame based on the time motion vector, and determining motion information of a subblock of the current block based on the corresponding block in the reference frame.
[0081] Optionally, determining the target reference frame list based on the reference frame list of the current block includes at least one of the following: if the current frame in which the current block is located employs a low-latency coding mode and the collated frame of the current frame is the first frame in the second reference frame list, then determining the second reference frame list as the target reference frame list; or if the current frame in which the current block is located does not employ a low-latency coding mode, or if the collated frame of the current frame is not the first frame in the second reference frame list, then determining the first reference frame list as the target reference frame list.
[0082] Optionally, the first reference frame list may be a forward reference frame list, which may be a reference frame list containing a first set of reference frames, the first set of reference frames containing reference frames whose temporal order is before the current frame and after the current frame.
[0083] Optionally, the second reference frame list may be a backreference frame list, which may be a reference frame list containing a second set of reference frames, the second set of reference frames containing reference frames whose temporal order is before the current frame and after the current frame.
[0084] It is important to note that in some cases, both the first and second reference frame lists may contain reference frames whose temporal order is before and after the current frame, and the bidirectional prediction may indicate that reference frames with different referencing directions are being selected from the first and second reference frame lists.
[0085] Optionally, determining the time motion vector of the current block based on the target reference frame list of the current block includes: selecting a first candidate motion vector from the current motion vector candidate list; searching for the reference frame of the first candidate motion vector in the target reference frame list; and determining the first candidate motion vector as the time motion vector if the reference frame of the first candidate motion vector and the collated frame of the current block are the same.
[0086] Optionally, determining the time motion vector of the current block based on the target reference frame list of the current block further includes determining the time motion vector to a 0 vector if the reference frame of the first candidate motion vector is different from the collated frame of the current block.
[0087] Optionally, performing interframe prediction on the current block includes determining the predicted block of the current block and calculating the residual block of the current block based on the initial block and the predicted block of the current block.
[0088] Optionally, performing interframe prediction on the current block includes determining the predicted block and residual block of the current block, and calculating the reconstructed block of the current block based on the predicted block and residual block of the current block.
[0089] Optionally, performing inter-frame predictions for the current block based on the movement information of the sub-blocks of the current block may include performing inter-frame predictions on a sub-block basis, using the sub-blocks of the current block as the unit and based on the movement information of the sub-blocks of the current block.
[0090] In the embodiments described above, all or part of the invention can be implemented by software, hardware, firmware, or any other combination. When implemented using software, all or part of the invention can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded into a computer and executed, all or part of the flows or functions described in the embodiments of the invention are generated. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, radio, microwave, etc.). The computer-readable storage medium may be any usable medium accessible to a computer, or a data storage device such as a server or data center that includes one or more usable media. The usable media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid state disks (SSDs)).
[0091] Those skilled in the art can conceive that the units and algorithmic steps of each example described in the embodiments disclosed herein can be combined to realize these functions in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are ultimately performed by hardware or software will depend on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to realize the described functions for individual specific applications, but such realizations should not be considered departures from the scope of this application.
[0092] In some embodiments provided herein, it should be understood that the disclosed systems, apparatus and methods can be implemented in other ways. For example, the embodiments of the apparatus described above are schematic, and for example, the division of the units is merely a logical functional division, and there may be other division methods in actual implementation, for example, multiple units or assemblies may be combined or integrated into another system. Or, some features may be ignored or not implemented. In other words, the combinations or direct combinations or communication connections between things shown or considered may be indirect combinations or communication connections through some interfaces, apparatus or units, and may be electrical, mechanical or otherwise.
[0093] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. The objectives of the solution in this embodiment can be achieved by selecting some or all of these units according to the actual needs.
[0094] Furthermore, each functional unit in each embodiment of the present application may be integrated into a single processing unit, and each unit may exist physically independently, or two or more units may be integrated into a single unit.
[0095] The above is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of any modification or substitution within the scope of the art disclosed herein, and such modifications or substitutions must fall within the scope of protection of the present application. Accordingly, the scope of protection of the present application must be equivalent to the scope of protection of the claims.
Claims
1. A video processing method, Currently, the movement vector of a spatially adjacent block at a specific position of the block is determined, For bidirectional prediction, first, the reference frame list in the current reference direction of the current block is scanned, If the reference frame of the motion vector of the spatially adjacent block of the current block in the current reference direction is the same as the collated frame of the current block, then the motion vector of the spatially adjacent block is determined as the time motion vector. If the reference frame of the motion vector of the spatially adjacent block in the current reference direction is different from the collated frame of the current block, the scanning of the reference frame list in another reference direction is continued. If the reference frame of the motion vector of the spatially adjacent block in a different reference direction is the same as the collated frame of the current block, then the motion vector of the spatially adjacent block is determined to be the time motion vector. A video processing method comprising determining a zero motion vector as the time motion vector if the reference frame of the motion vector of the spatially adjacent block in a different reference direction is different from the collated frame of the current block.
2. A video processing method according to claim 1, Determining the corresponding block of the current block in the reference frame based on the aforementioned time motion vector, A video processing method further comprising determining motion information of a subblock of the current block based on the corresponding block of the current block in the reference frame.
3. A video processing method according to claim 1, A video processing method in which the spatially adjacent block at the specific position of the current block is the block to the left, the block above, or the block to the upper left of the current block.
4. A video processing method according to any one of claims 1 to 3, The current block is to determine the predicted block, A video processing method further comprising: calculating the residual block of the current block based on the initial block of the current block and the predicted block.
5. A video processing method according to any one of claims 1 to 3, The prediction block and residual block of the current block are determined, A video processing method further comprising calculating a reconstructed block of the current block based on the predicted block and the residual block of the current block.
6. A video processing method according to any one of claims 1 to 3, A video processing method further comprising performing inter-frame prediction for the current block on a sub-block basis, based on motion information of the sub-blocks of the current block.
7. A video processing method according to claim 1, This indicates inserting a candidate into the candidate list of the current block, and determining the motion information of the subblock of the current block using the time motion vector. Inserting the inherited affine candidate into the candidate list of the current block, In response to the number of candidates in the candidate list being less than a set value, an affine candidate to be constructed is inserted into the candidate list, and the affine candidate to be constructed is obtained by combining the movement information of adjacent blocks of the current block. A video processing method further comprising: inserting the affine candidates to be constructed, and then, in response to the number of candidates in the candidate list being less than the set value, inserting a zero vector into the candidate list.
8. A video processing method according to claim 7, A video processing method in which inserting the inherited affine candidate into the candidate list of the current block includes inserting the control point motion vector of an adjacent block of the current block as the affine candidate of the current block into the candidate list of the current block.
9. A video processing device, Memory for storing code, The processor is configured to execute the code stored in the memory, Currently, the movement vector of a spatially adjacent block at a specific position of the block is determined, For bidirectional prediction, first, the reference frame list in the current reference direction of the current block is scanned, If the reference frame of the motion vector of the spatially adjacent block of the current block in the current reference direction is the same as the collated frame of the current block, then the motion vector of the spatially adjacent block is determined as the time motion vector. If the reference frame of the motion vector of the spatially adjacent block in the current reference direction is different from the collated frame of the current block, the scanning of the reference frame list in another reference direction is continued. If the reference frame of the motion vector of the spatially adjacent block in a different reference direction is the same as the collated frame of the current block, then the motion vector of the spatially adjacent block is determined to be the time motion vector. A video processing device configured to perform the following: determine a zero motion vector as the time motion vector if the reference frame of the motion vector of the spatially adjacent block in a different reference direction is different from the collated frame of the current block.
10. A method for generating a coded stream, Currently, the movement vector of a spatially adjacent block at a specific position of the block is determined, For bidirectional prediction, first, the reference frame list in the current reference direction of the current block is scanned, If the reference frame of the motion vector of the spatially adjacent block of the current block in the current reference direction is the same as the collated frame of the current block, then the motion vector of the spatially adjacent block is determined as the time motion vector. If the reference frame of the motion vector of the spatially adjacent block in the current reference direction is different from the collated frame of the current block, the scanning of the reference frame list in another reference direction is continued. If the reference frame of the motion vector of the spatially adjacent block in a different reference direction is the same as the collated frame of the current block, then the motion vector of the spatially adjacent block is determined to be the time motion vector. A code stream generation method, comprising determining a zero motion vector as the time motion vector if the reference frame of the motion vector of the spatially adjacent block in a different reference direction is different from the collated frame of the current block.