Video image processing method and device

Abstract

Provided is a video image processing method and device, the method comprising: determining a current image block; when the size of the current image block meets a preset condition, determining a temporal candidate motion vector of the current image block according to a temporal motion vector prediction (TMVP) operation and / or an advanced / optional temporal motion vector prediction (ATMVP) operation. The complexity can be reduced while ensuring the encoding and decoding performance.

Description

[0001] Copyright Notice

[0002] The disclosure of this patent document contains material that is subject to copyright protection. The copyright is reserved by the copyright owner. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it exists in the official records and files of the Patent and Trademark Office. Technical Field

[0003] The present application relates to the field of video coding and decoding, and specifically to a video image processing method and device. Background Art

[0004] At present, the main video coding standards all use block-based motion compensation technology in the inter-frame prediction part. Its main principle is to find a most similar block in the coded image for the current image block. This process is called motion compensation. For example, for a frame of image, it is first divided into equal-sized coding areas (Coding Tree Unit, CTU), for example, the size is 64×64 or 128×128. Each CTU can be further divided into square or rectangular coding units (Coding Unit, CU). Each CU searches for the most similar block in the reference frame (generally a reconstructed frame near the current frame in the time domain) as the prediction block of the current CU. The relative displacement between the current block (i.e., the current CU) and the similar block (i.e., the prediction block of the current CU) is called a motion vector (Motion Vector, MV). The process of finding the most similar block in the reference frame as the prediction block of the current block is motion compensation.

[0005] In a current prediction mode, the motion information candidate list of the current CU is usually constructed in two ways. The first is the candidate motion vector in the spatial domain, which usually fills the motion information of the encoded neighboring blocks of the current CU into the candidate list; the second is the candidate motion vector in the temporal domain. The temporal motion vector prediction (TMVP) uses the motion information of the corresponding CU in the adjacent encoded image of the current CU. The motion vector of the current CU is determined according to a candidate motion vector in the motion information candidate list; the prediction block of the current CU is determined according to the motion vector of the current CU.

[0006] There is room for improvement in current forecasting models. Summary of the invention

[0007] The present application provides a video image processing method and device, which can reduce the complexity of ATMVP technology while maintaining the performance gain of existing ATMVP technology.

[0008] In a first aspect, a video image processing method is provided, the method comprising:

[0009] Determine the current image block;

[0010] When the size of the current image block meets a preset condition, the temporal candidate motion vector of the current image block is determined according to the temporal motion vector prediction TMVP operation and / or the advanced / optional temporal motion vector prediction ATMVP operation; wherein,

[0011] The TMVP operation includes:

[0012] Determining related blocks of the current image block in temporally adjacent images;

[0013] Determine a temporal candidate motion vector of the current image block according to the motion vector of the related block;

[0014] The ATMVP operation includes:

[0015] Determining related blocks of the current image block in temporally adjacent images;

[0016] Dividing the current image block into a plurality of sub-image blocks;

[0017] Determining in the correlation block a sub-correlation block corresponding to each of the sub-image blocks in the plurality of sub-image blocks;

[0018] The temporal candidate motion vector of the sub-image block of the current image block is determined according to the motion vector of the sub-correlation block corresponding to each sub-image block.

[0019] The solution provided by this application can simplify existing redundant operations.

[0020] In a second aspect, a video image processing device is provided, the device comprising:

[0021] A memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to:

[0022] Determine the current image block;

[0023] When the size of the current image block meets a preset condition, the temporal candidate motion vector of the current image block is determined according to the temporal motion vector prediction TMVP operation and / or the advanced / optional temporal motion vector prediction ATMVP operation; wherein,

[0024] The TMVP operation includes:

[0025] Determining related blocks of the current image block in temporally adjacent images;

[0026] Determine a temporal candidate motion vector of the current image block according to the motion vector of the related block;

[0027] The ATMVP operation includes:

[0028] Determining related blocks of the current image block in temporally adjacent images;

[0029] Dividing the current image block into a plurality of sub-image blocks;

[0030] Determining in the correlation block a sub-correlation block corresponding to each of the sub-image blocks in the plurality of sub-image blocks;

[0031] The temporal candidate motion vector of the sub-image block of the current image block is determined according to the motion vector of the sub-correlation block corresponding to each sub-image block.

[0032] According to a third aspect, a computer non-volatile storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer implements the method according to the first aspect or any possible implementation of the first aspect.

[0033] According to a fourth aspect, a computer program product comprising instructions is provided, wherein when the instructions are executed by a computer, the computer implements the method according to the first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic flow chart of the video image processing method provided in an embodiment of the present application.

[0035] Figure 2 It is a schematic block diagram of a video image processing device provided in an embodiment of the present application.

[0036] Figure 3 A schematic block diagram of an implementation method of an encoding device or a decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In video coding and decoding, the prediction step is used to reduce redundant information in an image. A prediction block refers to a basic unit used for prediction in a frame of an image. In some standards, the prediction block is also called a prediction unit (PU). Before encoding / compressing a frame of an image, the image is divided into multiple image blocks. Further, each of the multiple image blocks can be divided into multiple image blocks again, and so on. In different coding methods, the number of levels of segmentation can be different, and the operation methods undertaken are also different. In different coding standards, the names of image blocks on the same level may be different. For example, in some video standards, each image block of the multiple image blocks into which a frame of an image is first divided is called a coding tree unit (CTU); each coding tree unit can contain a coding unit (CU) or be divided into multiple coding units again; a coding unit can be divided into one, two, four or other number of prediction units according to the prediction method. In some video standards, the coding tree unit is also called the largest coding unit (LCU).

[0038] Prediction refers to finding image data similar to the prediction block, also called the reference block of the prediction block. The difference between the prediction block and the reference block of the prediction block is encoded / compressed to reduce redundant information in encoding / compression. The difference between the prediction block and the reference block can be the residual obtained by subtracting the corresponding pixel values ​​of the prediction block from those of the reference block. Prediction includes intra-frame prediction and inter-frame prediction. Intra-frame prediction refers to finding the reference block of the prediction block within the frame where the prediction block is located, and inter-frame prediction refers to finding the reference block of the prediction block in frames other than the frame where the prediction block is located.

[0039] In some existing video standards, a prediction unit is the smallest unit in an image, and a prediction unit will not be further divided into multiple image blocks. However, the "image block" or "current image block" mentioned below refers to a prediction unit (or a coding unit), and an image block can be further divided into multiple sub-image blocks, each of which can be further predicted. The current image block is the image block to be encoded (or decoded). The image frame in which the current image block is located is called the current frame. For example, the current image block is a coding unit (CU) in some video standards.

[0040] In this scheme, before predicting the current image block, a motion information candidate list is constructed, and the current image block is predicted based on the candidate motion information selected in the motion information candidate list. The motion information mentioned in this article may include a motion vector, or include a motion vector and reference frame information. The motion information candidate list refers to a set of candidate motion information of the current block, and each candidate motion information in the motion information candidate list may be stored in the same buffer or in different buffers, without limitation. The index of the motion information mentioned below in the motion information candidate list may be the index of the motion information in the set of all candidate motion information of the current block, or may be the index of the motion information in the buffer where it is located, without limitation.

[0041] There are multiple modes for constructing a motion information candidate list. The following first describes multiple modes for constructing a motion information candidate list with examples.

[0042] In the first mode, as a first example, at the encoding end, after the motion information candidate list is constructed, the encoding of the current image block can be completed through the following steps.

[0043] 1) Select the best motion information from the motion information candidate list, determine the motion vector MV1 of the current image block according to the motion information, and obtain the index of the selected motion information in the motion information candidate list.

[0044] 2) According to the motion vector MV1 of the current image block, the predicted image block of the current image block is determined from the reference image (ie, the reference frame), that is, the position of the predicted image block of the current image block in the reference frame is determined.

[0045] 3) Obtain the residual between the current image block and the predicted image block.

[0046] 4) Send the index obtained in step 1) and the residual obtained in step 3) to the decoding end.

[0047] As an example, at the decoding end, the current image block may be decoded through the following steps.

[0048] 1) Receive the residual and index from the encoder.

[0049] 2) A preset method is used to construct a motion information candidate list. The preset method may be consistent with the method used by the encoder to construct the motion information candidate list.

[0050] 3) According to the index, motion information is selected from the motion information candidate list and the motion vector MV1 of the current image block is determined according to the selected motion information.

[0051] 4) According to the motion vector MV1, the predicted image block of the current image block is obtained, and then combined with the residual, the current image block is decoded.

[0052] That is, in the first type of mode, the motion vector of the current image block is equal to the predicted MV (Motion vector prediction, MVP) (such as the motion vector MV1 mentioned above). In some video coding standards, the first type of mode includes merge mode and / or affine merge mode.

[0053] In the second mode, different from the first mode, after the encoder selects the best motion information from the motion information candidate list and determines the predicted MV of the current image block based on the motion information, it also uses the predicted MV as the search starting point for motion search, and records the displacement between the final searched position and the search starting point as the motion vector difference (MVD). Then, based on the predicted MV+MVD of the current image block, the predicted image block of the current image block is determined from the reference image. Therefore, in addition to the index number and residual mentioned in the first mode, the bitstream sent by the encoder to the decoder also includes the MVD. In some video coding and decoding standards, the second mode may include an Advanced Motion Vector Prediction (AMVP) mode.

[0054] The construction methods of motion information candidate lists in different types of modes may be the same or different. A motion information candidate list constructed in the same way may be applicable to only one type of mode or to different types of construction modes. The method for determining one of the candidates in the motion information candidate list may use only one type of mode or to different types of construction modes. No limitation is made here.

[0055] In this solution, two construction methods of motion information candidate lists are provided. For the convenience of description, the motion information candidate lists of the two construction methods are referred to as the first motion vector candidate list and the second motion vector candidate list hereinafter. One difference between the two lists is that at least one candidate in the first motion vector candidate list includes the motion vector of the sub-image block, and each candidate in the second motion vector candidate list includes the motion vector of the image block.

[0056] As mentioned above, the image block and the current image block here are the same type of concepts, both referring to a prediction unit (or a coding unit), and the sub-image block refers to multiple sub-image blocks obtained by segmenting the image block. When using the candidates in the first candidate list of motion vectors for prediction, the reference block of the current image block is determined based on the candidate, and then the residual between the image block and the reference block is calculated. When using the candidates in the second candidate list of motion vectors for prediction, if the candidate used is the motion vector of the sub-image block, the reference block of each sub-image block in the current image block is determined based on the candidate, and then the residual between each sub-image block in the current image block and its reference block is calculated, and the residuals of each sub-image block are spliced ​​into the residual of the current image block.

[0057] The second candidate list of motion vectors of the current image block mentioned in this solution can be applied to the above-mentioned first type of mode and / or second type of mode. For example, in some video coding standards, the second candidate list of motion vectors can be a normal Merge motion information candidate list (Normal Merge Candidate List) in the Merge candidate list. In some video coding standards, the second candidate list of motion vectors can be an AMVP candidate list (AMVP CandidateList). In some video coding standards, the first candidate list of motion vectors can be an affine Merge motion information candidate list (Affine Merge Candidate List) in the Merge candidate list. It should be understood that the second candidate list of motion vectors can also have other names.

[0058] It should be understood that the first candidate list of motion vectors and the second candidate list of motion vectors formed by the scheme provided in the present application can be applied to the encoding end and the decoding end. In other words, the execution subject of the method provided in the present application can be the encoding end or the decoding end.

[0059] In one example, when determining candidates in a motion vector first candidate list and / or a motion vector second candidate list, the candidates can be determined based on a TMVP operation and / or an advanced / alternative temporal motion vector prediction (ATMVP) operation.

[0060] Among them, the ATMVP operation is a motion vector prediction mechanism. The basic idea of ​​ATMVP technology is to perform motion compensation by obtaining the motion information of multiple sub-blocks in the current image block (such as the current CU). The ATMVP operation introduces the motion information of multiple sub-blocks in the current image block as candidates in constructing a candidate list (such as a merge / affine merge candidate list or an AMVP candidate list). The implementation of ATMVP technology can be roughly divided into two steps. The first step is to determine a time domain vector by scanning the candidate motion vector list of the current image block or the motion vector of the adjacent image block of the current image block; the second step is to divide the current image block into N×N sub-blocks (such as sub-CU), determine the corresponding block of each sub-block in the reference frame according to the time domain vector obtained in the first step, and determine the motion vector of each sub-block according to the motion vector of the corresponding block of each sub-block in the reference frame.

[0061] For example, when constructing the first candidate list of motion vectors, the motion vector determined according to the ATMVP operation can be added to the list as a candidate (e.g., as the first candidate). When constructing the second candidate list of motion vectors, the motion vector determined according to the TMVP operation can be added to the list as a candidate. For example, the time domain candidate motion vector determined according to the TMVP operation can be added as a candidate to both the conventional Merge candidate list and the AMVP candidate list. For another example, the time domain candidate motion vector determined according to the TMVP operation can be added as a candidate to the conventional Merge candidate list or to the AMVP candidate list.

[0062] In one example, the TMVP operation includes: determining a related block of the current image block in a temporally adjacent image; and determining a temporal candidate motion vector of the current image block according to a motion vector of the related block.

[0063] In one example, the ATMVP operation includes: determining a related block of a current image block in a time-domain adjacent image; dividing the current image block into a plurality of sub-image blocks; determining a sub-related block corresponding to each of the plurality of sub-image blocks in the related block; and determining a time-domain candidate motion vector of the sub-image block of the current image block based on a motion vector of the sub-related block corresponding to each sub-image block.

[0064] Among them, the time-domain adjacent image mentioned in the TMVP operation and the ATMVP operation can be a reference image that is closest to the image where the current image block is located in time; or, the time-domain adjacent image can be a reference image preset by the codec end; or, the time-domain adjacent image can be a reference image of the current image block, which is a reference image specified in the video parameter set, sequence header, sequence parameter set, image header, image parameter set, and slice header. In one example, the time-domain adjacent image can be a collocated frame of the current image block. The collocated frame is a frame set in the slice-level information header for obtaining motion information for prediction. In some application scenarios, the collocated frame is also called a collocated picture.

[0065] In one example, the related block of the current image block may be a collocated block of the current image block. In some video coding standards, the related block may be referred to as a collocated block or a corresponding block. The collocated block may be an image block having the same position as the current image block in a collocated frame, or an image block having the same position difference as the current image block in a collocated frame.

[0066] The method for determining the related blocks of the current image block in the TMVP operation and the ATMVP operation may be the same or different.

[0067] In one example, the method for determining the related blocks of the current image block in the TMVP operation and the method for determining the related blocks of the current image block in the ATMVP operation are the same, both including: determining the image block at the same position in the image adjacent to the time domain at a specified position in the current image block as the related block of the current image block; or, determining the image block at the same position in the image adjacent to the time domain at a specified position adjacent to the current image block in the spatial domain as the related block of the current image block.

[0068] In an example of the first step of the ATMVP operation and the TMVP operation, all spatial candidate motion vectors currently added to the motion vector merge candidate list are scanned to determine the relevant blocks of the current image block. Compared with this example, the redundant operations in the TMVP operation and the ATMVP operation can be simplified by adopting the method of "determining the image block at the same position in the image adjacent to the temporal domain at the specified position in the current image block as the relevant block of the current image block; or determining the image block at the same position in the image adjacent to the temporal domain at the specified position adjacent to the spatial domain of the current image block as the relevant block of the current image block".

[0069] The size of the related block of the current image block may be the same as the size of the current image block, or the size of the related block of the current image block is a default value.

[0070] The specified position in the current image block may be any position in the current image block, for example, any one of the upper left corner, upper right corner, center, lower left corner, and lower right corner of the current image block. The spatially adjacent specified position in the current image block refers to a specified position in the current image other than the current image block, for example, a specified position adjacent to the current image block.

[0071] Taking the above-mentioned designated position as the upper left corner point of the current image block as an example, there is a pixel point in the time domain adjacent image with the same position as the upper left corner point of the current image block. The encoded / decoded block where the pixel point is located can be used as the related block of the current image block, or, an image block with the pixel point as the upper left corner point and the same size as the current image block or a preset size can be used as the related block of the current image block.

[0072] In an example of ATMVP operation, the size of the sub-image block is set adaptively at the frame level, and the size of the sub-image block defaults to 4×4. When certain conditions are met, the size of the sub-image block is set to 8×8. For example, at the encoding end, when encoding the current image block, the average block size of each sub-image block in the CU when the last coded image block of the same time domain layer is coded in ATMVP mode is calculated. When the average block size is greater than the threshold, the size of the sub-image block of the current image block is set to 8×8, otherwise the default value of 4×4 is used. At present, in the new generation of video coding standards (Versatile Video Coding, VVC), motion vectors are stored in a size of 8×8. It should be understood that when the size of the sub-image block is set to 4×4, the size of the motion vector of the sub-image block (also 4×4) does not meet the storage granularity of the motion vector in the current standard. In addition, in this example of ATMVP operation, when encoding the current image block, it is also necessary to store information on the size of the sub-image block of the last coded image block of the same time domain layer.

[0073] In another example of ATMVP operation, the current image block is a CU, and the sub-image blocks obtained after dividing it can be called sub-CU. Optionally, the size of the sub-image block and / or the size of the related blocks of the sub-image block is fixed to be greater than or equal to 64 pixels. Optionally, the size of the sub-image block and / or the size of the related blocks of the sub-image block are fixed to 8×8 pixels. Fixing the size of the sub-image block of the current image block to 8×8 can, on the one hand, adapt to the storage granularity of the motion vector specified in the video standard VVC, and on the other hand, there is no need to store information on the size of the sub-image block of the previous encoded image block, thereby saving storage space.

[0074] It should be understood that, under the premise of ensuring that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block can also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.

[0075] In some implementations, the storage granularity of the motion vector may not be 8×8, but other values. Optionally, the size of the sub-image block of the current image block is set to be the same as the granularity of the motion vector, which is x×y, where x and y are positive integers.

[0076] In some implementations, when determining the candidates in the first candidate list of motion vectors and / or the second candidate list of motion vectors of the current block, when the size of the current image block meets a preset condition, the time domain candidate motion vector of the current image block in the first candidate list of motion vectors and / or the second candidate list of motion vectors is determined according to the ATMVP operation.

[0077] In some implementations, when determining the candidates in the first candidate list of motion vectors and / or the second candidate list of motion vectors for the current block, when the size of the current image block does not meet a preset condition, the ATMVP operation is turned off, that is, the time domain candidate motion vectors of the current image block in the first candidate list of motion vectors and / or the second candidate list of motion vectors are not determined based on the ATMVP operation.

[0078] In some implementations, when determining the candidates in the first candidate list of motion vectors and / or the second candidate list of motion vectors for each image block, when the size of the current image block meets a preset condition, the time domain candidate motion vector of the current image block in the first candidate list of motion vectors and / or the second candidate list of motion vectors is determined according to the TMVP operation.

[0079] In some implementations, when determining the candidates in the first candidate list of motion vectors and / or the second candidate list of motion vectors for each image block, when the size of the current image block does not meet a preset condition, the TMVP operation is turned off, that is, the time domain candidate motion vector of the current image block in the first candidate list of motion vectors and / or the second candidate list of motion vectors is determined based on the TMVP operation.

[0080] Wherein, in at least one implementation of the four implementations mentioned above, the preset condition may include one condition, or a combination of multiple conditions. For example, when the size of the current image block satisfies the first condition group, the time domain candidate motion vector of the current image block is determined according to the ATMVP operation. When the size of the current image block satisfies the second condition group, the time domain candidate motion vector of the current image block is determined according to the TMVP operation. Wherein, the number of conditions in the first condition group is at least 1. Wherein, the number of conditions in the second condition group is at least 1. Wherein, the first condition group and the second condition group may be completely the same, completely different, or partially the same.

[0081] In one example, the size of the current image block is x1×y1, and the default setting size of the sub-image block of the current image block is x2×y2, wherein x1, x2, y1, and y2 are all positive integers; the preset conditions include: x1 is not less than x2, and / or y1 is not less than y2. For example, when x1 is not less than x2, and / or y1 is not less than y2, the temporal candidate motion vector of the current image block is determined according to the ATMVP operation and / or the TMVP operation.

[0082] In one example, when x1 is less than or equal to x2, and / or y1 is less than or equal to y2, the ATMVP operation is not performed. For example, when constructing the above-mentioned first candidate list of motion vectors, none of the added motion vector candidates is determined according to the ATMVP operation.

[0083] In one example, the size of the current image block is x1×y1, and the preset size is x3×y3; wherein x1, x3, y1, and y3 are all positive integers; and the preset conditions include: x1 is not less than x3, and / or y1 is not less than y3. For example, when x1 is less than or equal to x3, and / or y1 is less than or equal to y3, the temporal candidate motion vector of the current image block is determined according to the ATMV operation and / or the TMVP operation.

[0084] In one example, when x1 is less than or equal to x3, and / or y1 is less than or equal to y3, the TMVP operation is set not to be performed. Skipping the TMVP operation Because in hardware design, it is required that the time to complete encoding or decoding of processing areas of the same size is consistent as much as possible, so for areas containing more small blocks, the required pipeline time is much longer than that of other areas; therefore, saving the pipeline time of small blocks is very meaningful for parallel processing of hardware. When the size of the current image block is small, skipping the TMVP operation can save the pipeline time of small blocks. In addition, the utilization rate of time domain correlation in current coding technology is getting higher and higher, and many time domain prediction technologies are adopted, such as ATMVP. Therefore, for small blocks, the performance impact of skipping TMVP can be ignored.

[0085] When it is mentioned in this document that the TMVP operation is not performed or the TMVP operation is skipped, in the case where both the first category mode and the second category mode motion information candidate lists adopt the TMVP operation to determine candidates, the TMVP operation can be skipped only in the first category mode or only in the second category mode motion information candidate lists; or, the TMVP operation can be skipped in both the first category mode and the second category mode motion information candidate lists.

[0086] In one example, the storage granularity of the motion vector is x3×y3.

[0087] In one example, when the size of the current image block is the same as the first default size, only one of the TMVP operation and the ATMVP operation is performed. For example, it is set not to perform the ATMVP operation or not to perform the TMVP operation. For example, only perform the ATMVP operation. For example, it is set not to perform the TMVP operation. In one example, the first default size can be the same as the size of the storage granularity of the motion vector. When the size of the current image block is the same as the first default size, there is a certain redundancy between the ATMVP technology and the TMVP technology. Both technologies derive a set of time domain motion information for the current image block. Therefore, it is set not to perform one of the operations, skipping some redundant operations, which can effectively save encoding and decoding time. In some implementations, ATMVP uses the candidate motion vectors in the merge list of the current image block when deriving, while TMVP directly derives the candidate motion vectors from the fixed position in the adjacent encoded image. In this case, the motion vector derived by the ATMVP technology is more effective and adaptive than the TMVP technology to a certain extent, so it is set not to perform the TMVP operation.

[0088] In one example, when the current image block contains a storage granularity of a motion vector, the temporal candidate motion vector of the current image block is determined according to the ATMVP operation and / or the TMVP operation. That is, when the size of the current image block can cover the storage granularity of a motion vector, the temporal candidate motion vector of the current image block is determined according to the ATMVP operation and / or the TMVP operation.

[0089] In one example, when the current image block cannot contain the storage granularity of a motion vector or the size of the current image block is equal to the storage granularity of a motion vector, it is set not to perform the TMVP operation. The TMVP technology derives a set of temporal motion information. When the current image block cannot contain the storage granularity of a motion vector, the derived motion vectors may be the same, causing unnecessary division operations.

[0090] In one example, the preset condition includes: the number of pixels of the current image block is greater than or greater than or equal to a preset value. In one example, when the number of pixels of the current image block is less than or less than or equal to the preset value, the TMVP operation and / or the ATMVP operation is not performed. For example, the preset value may be 32 or 64.

[0091] In a specific example, when the width or height of the current CU block is less than 8, when the width and height of the current CU block are both equal to 8, and when the width or height of the current CU block is less than 8 or when the width and height of the current CU block are both equal to 8, it is set not to perform the TMVP operation. Since some redundant operations are skipped, the encoding and decoding time can be effectively saved.

[0092] In some implementations, in the process of constructing the second candidate list of motion vectors for the current image block, the TMVP operation process is turned off, that is, the TMVP technology is not used to determine the time domain candidate motion vectors added to the second candidate list of motion vectors. Taking into account other operations related to the added time domain motion information, such as ATMVP operation, HMVP operation, etc., the effect of the TMVP technology in the current construction process is greatly reduced, and it has a certain redundancy with the above technologies, that is, the same motion information may be exported in some cases, resulting in the candidate list construction process being too cumbersome and inefficient. In one example, the TMVP process is turned off in the process of constructing the merge candidate motion vector list for the current image block. In this case, the time domain related information is still effectively utilized, and the merge candidate list construction process is simpler, which can reduce the complexity of the codec end to a certain extent.

[0093] In the motion compensation prediction stage, the previous mainstream video coding standards only applied the translation motion model. In the real world, there are too many forms of motion, such as zooming in / out, rotation, perspective motion and other irregular motions. In order to improve the efficiency of inter-frame prediction, the affine motion compensation model can be introduced in the encoding and decoding technology. Affine motion compensation describes the affine motion field of the image block through the MV of a set of control points. In an example, the affine motion compensation model adopts a four-parameter Affine model, and the group of control points includes two control points (for example, the upper left corner and the upper right corner of the image block). In an example, the affine motion compensation model adopts a six-parameter Affine model, and the group of control points includes three control points (for example, the upper left corner, the upper right corner and the lower left corner of the image block).

[0094] In one implementation, when constructing the first candidate list of motion vectors, the added candidates may be a set of MVs of control points, or referred to as control point motion vector prediction (CPMVP). Optionally, the first candidate list of motion vectors may be used in Merge mode, specifically, may be referred to as Affine Merge mode; correspondingly, the first candidate list of motion vectors may be referred to as affine mergecandidate list. In Affine Merge mode, the prediction in the first candidate list of motion vectors is directly used as the CPMV (Control point motion vector) of the current image block, i.e., there is no need to perform an affine motion estimation process.

[0095] In one implementation, the candidates determined according to the ATMVP technique may be added to the first candidate list of motion vectors.

[0096] In one example, the control point motion vector group of the related blocks of the current image block is added as a candidate to the first candidate list of motion vectors. When the candidate in the first list of motion vectors is used for prediction, the current image block is predicted according to the control point motion vector group of the related blocks of the current image block.

[0097] In one example, as described above, the representative motion vector of the relevant block of the current image block is added as a candidate to the first candidate list of motion vectors. Further, optionally, the candidate is also marked as determined according to the ATMVP technology. When the candidate in the first candidate list of motion vectors is used for prediction, the relevant block of the current image block is determined according to the mark and the candidate, and the current image block and the relevant block are divided into multiple sub-image blocks in the same way, and each sub-image block in the current image block corresponds to each sub-image block in the relevant block one by one; the motion vectors of the corresponding sub-image blocks in the current image block are predicted according to the motion vectors of each sub-image block in the relevant block.

[0098] Wherein, optionally, when a sub-image block with an unavailable motion vector appears in the relevant block, the representative motion vector of the relevant block is used to replace the unavailable motion vector, and the corresponding sub-image block in the current image block is predicted. Optionally, when the representative motion vectors of the relevant blocks are all unavailable, the candidate determined according to the ATMVP technology is abandoned to be added to the second candidate list of the motion vector. In one example, when the sub-image block in the relevant block is unavailable, or the sub-image block in the relevant block adopts the intra-frame coding mode, the sub-image block with an unavailable motion vector in the relevant block is determined.

[0099] Wherein, optionally, each candidate in the first candidate list of motion vectors includes a motion vector of a group of control points; when the representative motion vector of the relevant block of the current image block is added to the first candidate list of motion vectors, in order to ensure the consistency of the data format, the representative motion vector of the relevant block can be inserted as the motion vector of each control point in the candidate (that is, the motion vector of each control point in the candidate is assigned to the representative motion vector of the relevant block).

[0100] Optionally, the representative motion vector of the relevant block of the current image block may refer to a motion vector at the center position of the relevant block, or other motion vectors representing the relevant block, which is not limited here.

[0101] like Figure 1 As shown, an embodiment of the present application also provides a video image processing method, which includes the following steps.

[0102] S110, determining a current image block.

[0103] S120, when the size of the current image block meets a preset condition, determining the time domain candidate motion vector of the current image block according to a time domain motion vector prediction TMVP operation and / or an advanced / optional time domain motion vector prediction ATMVP operation.

[0104] The TMVP operation includes: determining a related block of the current image block in an image adjacent to the current image block in the time domain; and determining a temporal candidate motion vector of the current image block according to a motion vector of the related block.

[0105] The ATMVP operation includes: determining a related block of a current image block in a time-domain adjacent image; dividing the current image block into a plurality of sub-image blocks; determining a sub-related block corresponding to each of the plurality of sub-image blocks in the related block; and determining a time-domain candidate motion vector of the current image block based on a motion vector of a sub-related block corresponding to each sub-image block.

[0106] about Figure 1 The video image processing method shown can be referred to the above description and will not be repeated here.

[0107] Combination of the above Figure 1 The method embodiment of the present application is described below. Figure 1 The device embodiment corresponding to the method embodiment shown. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the contents not described in detail can be referred to the previous method embodiment, and will not be repeated here for the sake of brevity.

[0108] Figure 2FIG. 2 is a schematic block diagram of a video image processing device 200 provided in an embodiment of the present application. The device 200 is used to perform the following steps: Figure 1 The method embodiment shown in the figure. The device 200 includes the following units.

[0109] A first determination module 210, configured to determine a current image block;

[0110] The second determination module 220 is used to determine the time domain candidate motion vector of the current image block according to the time domain motion vector prediction TMVP operation and / or the advanced / optional time domain motion vector prediction ATMVP operation when the size of the current image block meets the preset conditions.

[0111] The TMVP operation includes: determining a related block of the current image block in an image adjacent to the current image block in the time domain; and determining a temporal candidate motion vector of the current image block according to a motion vector of the related block.

[0112] The ATMVP operation includes: determining a related block of a current image block in a time-domain adjacent image; dividing the current image block into a plurality of sub-image blocks; determining a sub-related block corresponding to each of the plurality of sub-image blocks in the related block; and determining a time-domain candidate motion vector of the current image block based on a motion vector of a sub-related block corresponding to each sub-image block.

[0113] It should be understood that the first determination module and the second determination module in this embodiment can be implemented by a processor.

[0114] The embodiment of the present application also provides a video image processing device. The device can be used to execute the method embodiment described above. The device includes a processor and a memory, the memory is used to store instructions, the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory causes the processor to execute the method according to the method embodiment described above.

[0115] Optionally, the apparatus may further include a communication interface for communicating with an external device. For example, the processor is used to control the communication interface to receive and / or send signals.

[0116] The device provided in the present application can be applied to an encoder or a decoder.

[0117] Figure 3A schematic block diagram of an implementation of an encoding device or decoding device (referred to as decoding device 1100) provided in the present application. The decoding device 1100 may include a processor 1110, a memory 1130, and a bus system 1150. The processor and the memory are connected via a bus system, the memory is used to store instructions, and the processor is used to execute instructions stored in the memory. The memory of the encoding device stores program code, and the processor can call the program code stored in the memory to execute various video encoding or decoding methods described in the present application, especially the inter-frame prediction method described in the present application. To avoid repetition, it is not described in detail here.

[0118] In the embodiment of the present application, the processor 1110 may be a CPU, or other general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0119] The memory 1130 may include ROM or RAM. Any other suitable type of storage device may also be used as the memory 1130. The memory 1130 may include code and data 1131 accessed by the processor 1110 using the bus 1150. The memory 1130 may further include an operating system 1133 and an application 1135, which includes at least one program that allows the processor 1110 to perform the video encoding or decoding method described in the present application (especially the inter-frame prediction method described in the present application). For example, the application 1135 may include applications 1 to N, which further include a video encoding or decoding application (referred to as a video decoding application) that executes the video encoding or decoding method described in the present application.

[0120] The bus system 1150 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as the bus system 1150 in the figure.

[0121] Optionally, the decoding device 1100 may further include one or more output devices, such as a display 1170. In one example, the display 1170 may be a touch-sensitive display that combines a display with a touch-sensitive unit operable to sense touch input. The display 1170 may be connected to the processor 1110 via the bus 1150.

[0122] An embodiment of the present application also provides a computer storage medium on which a computer program is stored. When the computer program is executed by a computer, the computer executes the method provided in the above method embodiment.

[0123] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to execute the method provided in the above method embodiment.

[0124] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0125] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0126] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0127] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0129] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video image encoding method, characterized in that: include: Determine the current image block; If the number of pixels of the current image block does not meet the preset condition, close the time domain candidate motion vector of the current image block determined according to the time domain motion vector prediction TMVP operation; wherein the TMVP operation is different from the advanced / optional time domain motion vector prediction ATMVP operation; Encoding the current image block; The TMVP operation includes: Determining related blocks of the current image block in temporally adjacent images; Determine a temporal candidate motion vector of the current image block according to the motion vector of the related block; The ATMVP operation includes: Dividing the current image block into a plurality of sub-image blocks; Determining, in the correlation block of the current image block, a sub-correlation block corresponding to each of the sub-image blocks of the plurality of sub-image blocks; The temporal candidate motion vector of the sub-image block of the current image block is determined according to the motion vector of the sub-correlation block corresponding to each sub-image block.

2. The video image encoding method according to claim 1, characterized in that: The preset condition includes: the number of pixels of the current image block is greater than or equal to a preset value.

3. The video image encoding method according to claim 2, characterized in that: The preset value is equal to 32.

4. The video image encoding method according to claim 1, characterized in that: The step of determining the temporal candidate motion vector of the current image block according to the TMVP operation includes: The TMVP operation is skipped in the motion information candidate list of the first type mode or the second type mode; or, the TMVP operation is skipped in both the motion information candidate list of the first type mode and the second type mode.

5. The video image encoding method according to claim 4, characterized in that: The first type of mode includes a merge mode and / or an affine merge mode, and / or the second type of mode includes an advanced motion vector prediction AMVP mode.

6. The video image encoding method according to claim 1, characterized in that: The method further includes: outputting a code stream obtained by encoding the current image block.

7. An encoder, characterized in that include: A memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to: Determine the current image block; If the number of pixels of the current image block does not meet the preset condition, close the time domain candidate motion vector of the current image block determined according to the time domain motion vector prediction TMVP operation; wherein the TMVP operation is different from the advanced / optional time domain motion vector prediction ATMVP operation; Encoding the current image block; The TMVP operation includes: Determining related blocks of the current image block in temporally adjacent images; Determine a temporal candidate motion vector of the current image block according to the motion vector of the related block; The ATMVP operation includes: Dividing the current image block into a plurality of sub-image blocks; Determining, in the correlation block of the current image block, a sub-correlation block corresponding to each of the sub-image blocks of the plurality of sub-image blocks; The temporal candidate motion vector of the sub-image block of the current image block is determined according to the motion vector of the sub-correlation block corresponding to each sub-image block.

8. The encoder according to claim 7, characterized in that The preset condition includes: the number of pixels of the current image block is greater than or equal to a preset value.

9. The encoder according to claim 8, characterized in that The preset value is equal to 32.

10. A video image decoding method, characterized in that: include: Determine the current image block; If the number of pixels of the current image block does not meet the preset condition, close the time domain candidate motion vector of the current image block determined according to the time domain motion vector prediction TMVP operation; wherein the TMVP operation is different from the advanced / optional time domain motion vector prediction ATMVP operation; Decoding the current image block; The TMVP operation includes: Determining related blocks of the current image block in temporally adjacent images; Determine a temporal candidate motion vector of the current image block according to the motion vector of the related block; The ATMVP operation includes: Dividing the current image block into a plurality of sub-image blocks; Determining, in the correlation block of the current image block, a sub-correlation block corresponding to each of the sub-image blocks of the plurality of sub-image blocks; The temporal candidate motion vector of the sub-image block of the current image block is determined according to the motion vector of the sub-correlation block corresponding to each sub-image block.

11. The video image decoding method according to claim 10, characterized in that: The preset condition includes: the number of pixels of the current image block is greater than or equal to a preset value.

12. The video image decoding method according to claim 11, characterized in that: The preset value is equal to 32.

13. The video image decoding method according to claim 10, characterized in that: The step of determining the temporal candidate motion vector of the current image block according to the TMVP operation includes: The TMVP operation is skipped in the motion information candidate list of the first type mode or the second type mode; or, the TMVP operation is skipped in both the motion information candidate list of the first type mode and the second type mode.

14. The video image decoding method according to claim 13, characterized in that: The first type of mode includes a merge mode and / or an affine merge mode, and / or the second type of mode includes an advanced motion vector prediction AMVP mode.

15. A decoder, characterized in that: include: A memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to: Determine the current image block; If the number of pixels of the current image block does not meet the preset condition, close the time domain candidate motion vector of the current image block determined according to the time domain motion vector prediction TMVP operation; wherein the TMVP operation is different from the advanced / optional time domain motion vector prediction ATMVP operation; Decoding the current image block; The TMVP operation includes: Determining related blocks of the current image block in temporally adjacent images; Determine a temporal candidate motion vector of the current image block according to the motion vector of the related block; The ATMVP operation includes: Dividing the current image block into a plurality of sub-image blocks; Determining, in the correlation block of the current image block, a sub-correlation block corresponding to each of the sub-image blocks of the plurality of sub-image blocks; The temporal candidate motion vector of the sub-image block of the current image block is determined according to the motion vector of the sub-correlation block corresponding to each sub-image block.

16. The decoder according to claim 15, characterized in that The preset condition includes: the number of pixels of the current image block is greater than or equal to a preset value.

17. The decoder according to claim 16, characterized in that The preset value is equal to 32.

18. An encoder, characterized in that: include: A memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to: Determine the current image block; If the number of pixels of the width of the current image block is less than or equal to 8 or the number of pixels of the height of the current image block is less than or equal to 8, turning off the determination of the temporal candidate motion vector of the current image block according to the advanced / optional temporal motion vector prediction ATMVP operation; Encoding the current image block; The ATMVP operation includes: Determining related blocks of the current image block in temporally adjacent images; Dividing the current image block into a plurality of sub-image blocks; Determining in the correlation block a sub-correlation block corresponding to each of the sub-image blocks in the plurality of sub-image blocks; The temporal candidate motion vector of the sub-image block of the current image block is determined according to the motion vector of the sub-correlation block corresponding to each sub-image block.

19. A method for generating a bit stream, characterized in that: include: Determine the current image block; If the number of pixels of the current image block does not meet the preset condition, close the time domain candidate motion vector of the current image block determined according to the time domain motion vector prediction TMVP operation; wherein the TMVP operation is different from the advanced / optional time domain motion vector prediction ATMVP operation; Encoding the current image block to generate a bit stream; The TMVP operation includes: Determining related blocks of the current image block in temporally adjacent images; Determine a temporal candidate motion vector of the current image block according to the motion vector of the related block; The ATMVP operation includes: Dividing the current image block into a plurality of sub-image blocks; Determining, in the correlation block of the current image block, a sub-correlation block corresponding to each of the sub-image blocks of the plurality of sub-image blocks; The temporal candidate motion vector of the sub-image block of the current image block is determined according to the motion vector of the sub-correlation block corresponding to each sub-image block.

20. A method for generating a bit stream, characterized in that: include: Determine the current image block; If the number of pixels of the width of the current image block is less than or equal to 8 or the number of pixels of the height of the current image block is less than or equal to 8, turning off the determination of the temporal candidate motion vector of the current image block according to the advanced / optional temporal motion vector prediction ATMVP operation; Encoding the current image block to generate a bit stream; The ATMVP operation includes: Dividing the current image block into a plurality of sub-image blocks; Determining, in the correlation block of the current image block, a sub-correlation block corresponding to each of the sub-image blocks of the plurality of sub-image blocks; The temporal candidate motion vector of the sub-image block of the current image block is determined according to the motion vector of the sub-correlation block corresponding to each sub-image block.

Images