Video image processing method and device

Abstract

Provided are a video image processing method and apparatus, comprising: sequentially scanning N neighboring blocks among M preset neighboring blocks of a current image block, determining a target neighboring block based on the scanning results, where N is less than M; determining a related block of the current image block based on a motion vector of the target neighboring block, the current image block, and a reference image of the current image block; dividing the current image block and the related block into multiple sub-image blocks in the same manner, with each sub-image block in the current image block corresponding to each sub-image block in the related block; and predicting the corresponding sub-image blocks in the current image block based on the motion vectors of each sub-image block in the related block. This method can reduce complexity while ensuring encoding and decoding performance.

Description

[0001] This application is a divisional application of the Chinese patent application "Video Image Processing Method and Device" with application date of October 30, 2018 and application number 201880016545.8.

[0002] Copyright Notice

[0003] The disclosure of this patent document contains material that is subject to copyright protection. The copyright is reserved by the copyright owner. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the official records and files of the Patent and Trademark Office. Technical Field

[0004] The present application relates to the field of video coding and decoding, and in particular to a video image processing method and device. Background Art

[0005] Currently, major video coding standards all use block-based motion compensation technology for inter-frame prediction. The main principle is to find the most similar block in the coded image for the current image block. This process is called motion compensation. For example, a frame is first divided into coding tree units (CTUs) of equal size, such as 64×64 or 128×128. Each CTU can be further divided into square or rectangular coding units (CUs). Each CU searches for the most similar block in a reference frame (generally a reconstructed frame near the current frame in the temporal domain) as the prediction block for the current CU. The relative displacement between the current block (i.e., the current CU) and the similar block (i.e., the prediction block for the current CU) is called a motion vector (MV). The process of finding the most similar block in the reference frame as the prediction block for the current block is motion compensation.

[0006] In current technology, a motion vector candidate list of the current CU, also called a merge candidate list, is usually constructed based on the motion vectors of the encoded neighboring blocks of the current CU; the best candidate motion vector is selected from the merge candidate list as the motion vector of the current CU; and the prediction block of the current CU is determined based on the motion vector of the current CU.

[0007] Advanced / Alternative Temporal Motion Vector Prediction (ATMVP) is a motion vector prediction mechanism. The basic concept of ATMVP is to perform motion compensation by obtaining motion information from multiple sub-blocks within the current CU. ATMVP incorporates the motion information of multiple sub-blocks within the current CU as candidates when constructing a candidate list (such as a merge candidate list or an AMVP (Advanced Motion Vector Prediction) candidate list). ATMVP implementation can be roughly divided into two steps. The first step is to determine a temporal vector by scanning the candidate motion vector list of the current CU or the motion vectors of the current CU's neighboring image blocks. The second step is to divide the current CU into N×N (N defaults to 4) sub-CUs. The temporal vectors obtained in the first step are used to determine the corresponding block in the reference frame for each sub-block. The motion vectors of each sub-block are then determined based on the motion vectors of the corresponding blocks in the reference frame.

[0008] In the first step of the current ATMVP technology, there is still room for improvement in the process of determining a time domain vector by scanning the candidate motion vector list of the current CU or the motion vectors of the adjacent image blocks of the current CU. Summary of the Invention

[0009] The present application provides a video image processing method and device, which can reduce the complexity of ATMVP technology while maintaining the performance gain of existing ATMVP technology.

[0010] In a first aspect, a video image processing method is provided, the method comprising:

[0011] Scanning N neighboring blocks among the preset M neighboring blocks of the current image block in sequence, and determining a target neighboring block according to the scanning results, where N is less than M;

[0012] determining a related block of the current image block according to a motion vector of the target neighboring block, the current image block, and a reference image of the current image block;

[0013] Dividing the current image block and the related block into a plurality of sub-image blocks in the same manner, wherein each sub-image block in the current image block corresponds to each sub-image block in the related block in a one-to-one manner;

[0014] The corresponding sub-image blocks in the current image block are predicted respectively according to the motion vectors of the sub-image blocks in the related blocks.

[0015] In the solution provided by this application, when obtaining a reference motion vector for the current image block, only N (N is less than M) candidate motion vectors out of the M candidate motion vectors already obtained are scanned sequentially. Compared to the prior art, this can reduce the number of scans of candidate motion vectors in the process of obtaining a reference motion vector for the current image block. It should be understood that applying the solution provided by this application to the first step of the existing ATMVP technology can simplify the redundant operations that exist therein.

[0016] In a second aspect, a video image processing method is provided, the method comprising:

[0017] Determine M neighboring blocks of the current image block according to M candidates in the second candidate list of the motion vector of the current image block;

[0018] Scanning N adjacent blocks in the M adjacent blocks in sequence, and determining a target adjacent block according to the scanning results, where N is less than M;

[0019] determining a related block of the current image block according to a motion vector of the target neighboring block, the current image block, and a reference image of the current image block;

[0020] determining a specific candidate in a first candidate list of motion vectors of the current image block according to a related block of the current image block;

[0021] When it is determined to adopt the specific candidate, the current image block and the related block are divided into a plurality of sub-image blocks in the same manner, and each sub-image block in the current image block corresponds to each sub-image block in the related block in a one-to-one manner;

[0022] The corresponding sub-image blocks in the current image block are predicted respectively according to the motion vectors of the sub-image blocks in the related blocks.

[0023] According to a third aspect, a video image processing device is provided, the device comprising:

[0024] a construction module, configured to sequentially scan N neighboring blocks among M preset neighboring blocks of a current image block, determine a target neighboring block based on the scanning results, where N is less than M; determine a related block of the current image block based on a motion vector of the target neighboring block, the current image block, and a reference image of the current image block; and divide the current image block and the related block into a plurality of sub-image blocks in the same manner, with each sub-image block in the current image block corresponding to each sub-image block in the related block in a one-to-one manner;

[0025] The prediction module is configured to predict the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.

[0026] According to a fourth aspect, a video image processing apparatus is provided, the apparatus comprising:

[0027] a construction module configured to determine M neighboring blocks of the current image block based on M candidates in a second candidate list for a motion vector of the current image block; sequentially scan N neighboring blocks among the M neighboring blocks and determine a target neighboring block based on the scanning results, where N is less than M; determine a related block of the current image block based on the motion vector of the target neighboring block, the current image block, and a reference image of the current image block; determine a specific candidate in a first candidate list for a motion vector of the current image block based on the related block of the current image block; and when the specific candidate is determined to be used, divide the current image block and the related block into a plurality of sub-image blocks in the same manner, with each sub-image block in the current image block corresponding one-to-one to each sub-image block in the related block;

[0028] The prediction module is configured to predict the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.

[0029] In a fifth aspect, a video image processing device is provided, which includes a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to execute the method in the first aspect or any possible implementation of the first aspect.

[0030] In the sixth aspect, a video image processing device is provided, which includes a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to execute the method in the second aspect or any possible implementation of the second aspect.

[0031] In a seventh aspect, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer implements the method in the first aspect or any possible implementation of the first aspect.

[0032] In an eighth aspect, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer implements the method in the second aspect or any possible implementation of the second aspect.

[0033] In a ninth aspect, a computer program product comprising instructions is provided, wherein when the instructions are executed by a computer, the computer implements the method in the first aspect or any possible implementation of the first aspect.

[0034] In a tenth aspect, a computer program product comprising instructions is provided, wherein when the instructions are executed by a computer, the computer implements the method in the second aspect or any possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic flow chart of the video image processing method provided in an embodiment of the present application.

[0036] Figure 2 This is a schematic diagram of obtaining candidate motion vectors of a current block through neighboring blocks of the current image block.

[0037] Figure 3 It is a schematic diagram of scaling the candidate motion vectors.

[0038] Figure 4 This is another schematic flowchart of the video image processing method provided in an embodiment of the present application.

[0039] Figure 5 It is a schematic block diagram of a video image processing device provided in an embodiment of the present application.

[0040] Figure 6 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.

[0041] Figure 7 It is a schematic flow chart of the video image processing method provided in an embodiment of the present application.

[0042] Figure 8 FIG. 4 is another schematic flow chart of the video image processing method provided in the embodiment of the present invention.

[0043] Figure 9 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.

[0044] Figure 10 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.

[0045] Figure 11 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.

[0046] Figure 12 FIG. 1 is a schematic diagram of obtaining candidates for the first candidate list of motion vectors.

[0047] Figure 13 FIG. 4 is another schematic diagram of constructing candidates for the first candidate list of motion vectors.

[0048] Figure 14 and Figure 15It is a schematic flow chart of the video image processing method provided in an embodiment of the present application.

[0049] Figure 16 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.

[0050] Figure 17 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In video coding and decoding, the prediction step is used to reduce redundant information in an image. A prediction block is the basic unit used for prediction within a frame. In some standards, this prediction block is also called a prediction unit (PU). Before encoding / compressing a frame, the image is divided into multiple image blocks. Further, each of these multiple image blocks can be further divided into multiple image blocks, and so on. Different coding methods can use different levels of division and perform different operations. Different coding standards may use different names for image blocks at the same level. For example, in some video standards, each of the multiple image blocks into which a frame is initially divided is called a coding tree unit (CTU); each CTU can contain one coding unit (CU) or be further divided into multiple coding units. A coding unit can be divided into one, two, four, or another number of prediction units depending on the prediction method. In some video standards, this CTU is also called the largest coding unit (LCU).

[0052] Prediction refers to searching for image data similar to the prediction block, also known as the reference block of the prediction block. Redundant information in encoding / compression is reduced by encoding / compressing the difference between the prediction block and the reference block. The difference between the prediction block and the reference block can be the residual obtained by subtracting the corresponding pixel values ​​of the prediction block from those of the reference block. Prediction includes intra-frame prediction and inter-frame prediction. Intra-frame prediction refers to searching for the reference block of the prediction block within the frame where the prediction block is located, while inter-frame prediction refers to searching for the reference block of the prediction block in frames other than the frame where the prediction block is located.

[0053] In some existing video standards, a prediction unit (PU) is the smallest unit in an image and is not further divided into multiple image blocks. Hereinafter, "image block" or "current image block" refers to a prediction unit (or coding unit). Furthermore, an image block can be further divided into multiple sub-image blocks, each of which can be further predicted.

[0054] In this solution, before predicting the current image block, a motion vector candidate list is constructed. The current image block is predicted based on the candidate motion vectors selected from this list. The motion vector candidate list has multiple modes. The following example illustrates these modes.

[0055] In the first mode, as a first example, at the encoding end, after the motion vector candidate list is constructed, the encoding of the current image block can be completed through the following steps.

[0056] 1) Select the best motion vector (denoted as MV1) from the motion vector candidate list, use the selected MV1 as the motion vector of the current image block, and obtain the index of MV1 in the motion vector candidate list.

[0057] 2) According to the motion vector MV1 of the current image block, the predicted image block of the current image block is determined from the reference image (ie, the reference frame), that is, the position of the predicted image block of the current image block in the reference frame is determined.

[0058] 3) Obtain the residual between the current image block and the predicted image block.

[0059] 4) Sending the index of the motion vector MV1 of the current image block in the motion vector candidate list and the residual obtained in step 3) to the decoding end.

[0060] As an example, at the decoding end, the current image block can be decoded through the following steps.

[0061] 1) Receive the residual and the index of the motion vector of the current image block in the motion vector candidate list from the encoder.

[0062] 2) Obtaining a motion vector candidate list by the method according to the embodiment of the present application. The motion vector candidate list obtained by the decoding end is consistent with the motion vector candidate list obtained by the encoding end.

[0063] 3) According to the index, obtain the motion vector MV1 of the current image block from the motion vector candidate list.

[0064] 4) According to the motion vector MV1, the predicted image block of the current image block is obtained, and then combined with the residual, the current image block is decoded.

[0065] That is, in the first type of mode, the motion vector of the current image block is equal to the predicted MV (Motion vector prediction, MVP). In some standards, the first type of mode is also called Merge mode.

[0066] In the second mode, unlike the first, after the encoder selects the optimal motion vector MV1 from the motion vector candidate list, it also performs a motion search using MV1 as the search starting point. The displacement between the final searched position and the search starting point is recorded as the motion vector difference (MVD). Then, based on the motion vector MV1 + MVD of the current image block, the predicted image block of the current image block is determined from the reference image. The encoder also sends the MVD to the decoder. In some standards, this second mode is also called AMVP mode (i.e., normal inter-frame prediction mode).

[0067] The motion vector candidate lists in different types of modes can be constructed in the same or different ways. A motion vector candidate list constructed in the same way can be applied to only one type of mode or to different types of construction modes, which is not limited here.

[0068] This solution provides two methods for constructing motion vector candidate lists. For ease of description, these two methods are referred to below as the first motion vector candidate list and the second motion vector candidate list. One difference between these two lists is that at least one candidate in the first motion vector candidate list includes the motion vector of a sub-image block, while each candidate in the second motion vector candidate list includes the motion vector of an image block. As mentioned above, the image block and the current image block are synonymous, both referring to a prediction unit (or a coding unit), and the sub-image block refers to multiple sub-image blocks obtained by segmenting the image block. When using a candidate in the first motion vector candidate list for prediction, the reference block for the current image block is determined based on the candidate, and then the residual between the image block and the reference block is calculated. When using a candidate in the second motion vector candidate list for prediction, if the candidate used is a sub-image block's motion vector, the reference block for each sub-image block in the current image block is determined based on the candidate, and then the residual between each sub-image block in the current image block and its reference block is calculated. The residuals of the sub-image blocks are concatenated to form the residual of the current image block.

[0069] In determining the candidates in the first candidate list of motion vectors and / or the second candidate list of motion vectors, one of the candidates can be determined according to the ATMVP technique. In one example, when constructing the first candidate list of motion vectors, the motion vector determined according to the ATMVP technique can be added to the list as the first candidate. In one example, when constructing the second candidate list of motion vectors, after adding candidates to the second candidate list of motion vectors based on the motion vectors of a preset number of spatially adjacent blocks at a preset position of the current image block, the motion vector determined according to the ATMVP technique can be added to the list as a candidate. Of course, the order in which the candidates are added to the two candidate lists can be other orders, and there is no limitation on this.

[0070] The following first uses an example to illustrate how to determine one of the candidates based on the ATMVP technology in combination with the construction method and use method of the second motion vector candidate list.

[0071] In describing the construction method of the second candidate list of motion vectors, the motion vector is explained here for ease of understanding. The motion vector of an image block can contain two pieces of information: 1) the image pointed to by the motion vector; 2) the displacement. The motion vector of an image block means the image block in the image pointed to by the motion vector that has the displacement with the image block. For an encoded / decoded image block, the meaning contained in its motion vector includes: the reference image of the encoded / decoded image block, and the displacement of the reference block of the encoded / decoded image block relative to the encoded / decoded image block. It should be noted that the reference block of an image block mentioned in this article refers to the image block used to calculate the residual of the image block.

[0072] Figure 1 A schematic flow chart of a video image processing method provided in an embodiment of the present application is provided. The method comprises the following steps.

[0073] S110 , determining M candidate motion vectors for adding to a second candidate list of motion vectors of a current image block.

[0074] The current image block is the image block to be encoded (or decoded). The image frame in which the current image block is located is called the current frame. For example, the current image block is a coding unit (CU).

[0075] For example, the second motion vector candidate list for the current image block may be a Merge candidate list or an AMVP candidate list. For example, the second motion vector candidate list may be a normal motion vector candidate list (Nomal Merge List) in the Merge candidate list. It should be understood that the second motion vector candidate list may also have other names.

[0076] The M candidate motion vectors may be determined based on the motion vectors of the M neighboring blocks of the current image block within the current frame. The neighboring blocks may be image blocks that are adjacent to or spaced a certain distance from the current image block in the current frame. It should be understood that these M neighboring blocks are image blocks that have already been encoded (or decoded) within the current frame.

[0077] As an example, Figure 2 As shown, the M neighboring blocks of the current image block are Figure 2 As shown in FIG, the image blocks are located at four positions A1 (left) → B1 (top) → B0 (top right) → A0 (bottom left) around the current image block. Based on the motion vectors of the image blocks at these four positions, M (i.e., M is equal to 4) candidate motion vectors for the current image block are determined.

[0078] In addition, when an unavailable neighboring block appears in the M neighboring blocks, or a neighboring block using intra-frame coding mode appears in the M neighboring blocks, the motion vector of the unavailable neighboring block or the neighboring block using intra-frame coding mode is unavailable. In this case, the motion vector of the unavailable neighboring block is not used as a candidate motion vector, and the unavailable motion vector is not added to the second candidate motion vector list of the current image block.

[0079] As a possible implementation, after step S110 is completed, the M candidate motion vectors have been added to the second candidate list of motion vectors. In step S120, the second candidate list of motion vectors may be directly scanned.

[0080] S120 , sequentially scan N candidate motion vectors among the M candidate motion vectors, and determine a reference motion vector according to the scanning results, where N is less than M. Here, M and N are both natural numbers.

[0081] Regardless of whether all of the M candidate motion vectors are added to the second motion vector candidate list, or whether some of the M candidate motion vectors are unavailable, resulting in only some of the M candidate motion vectors being added to the second motion vector candidate list, the N candidate motion vectors in the M candidate motion vectors are fixedly scanned sequentially. Fixedly scanning the N candidate motion vectors in the M candidate motion vectors sequentially can refer to fixedly scanning the candidate motion vectors in the N candidate motion vectors that have been added to the candidate motion vector list; or it can refer to fixedly scanning the N candidate motion vectors in the M candidate motion vectors that have been added to the candidate motion vector list.

[0082] The process of determining the reference motion vector according to the scanning results of the N candidate motion vectors may be to judge the N candidate motion vectors in sequence based on a preset condition, and determine the reference motion vector according to the judgment results.

[0083] As an example, the preset conditions include: the image block can obtain or not adopt the intra-frame prediction coding mode, and the reference frame pointed to by the candidate motion vector is the same as the reference image of the current image block.

[0084] Among them, the reference image of the current image block is the reference image with the shortest time distance to the image where the current image block is located; or, the reference image of the current image block is the reference image preset by the codec end; or, the reference image of the current image block is the reference image specified in the video parameter set, sequence header, sequence parameter set, image header, image parameter set, and slice header.

[0085] For example, the reference image of the current image block is the collocated frame of the current image block, which is a frame set in the slice-level information header for obtaining motion information for prediction. In some application scenarios, the collocated frame is also called a collocated picture.

[0086] It should be understood that, depending on future technological evolution, this preset condition may be given other different definitions, and the corresponding solutions also fall within the scope of protection of this application.

[0087] The process of determining the reference motion vector according to the scanning results of N candidate motion vectors will be described in detail below.

[0088] In step S120 , only N motion vectors among the M candidate motion vectors obtained in step S110 are scanned, which can reduce the number of scans.

[0089] Optionally, in step S120 , the first N candidate motion vectors among the M candidate motion vectors may be scanned in sequence.

[0090] Optionally, in step S120, the last N candidate motion vectors among the M candidate motion vectors may be scanned in sequence; or the middle N candidate motion vectors among the M candidate motion vectors may be scanned in sequence. This application does not limit this.

[0091] As an example, in step S120 , some of the M candidate motion vectors are scanned sequentially.

[0092] As another example, in step S120, some of the candidate motion vectors currently added to the second candidate list of motion vectors are scanned in sequence.

[0093] S130 , determining candidate motion vectors to be added to the second candidate list of motion vectors according to the reference motion vector, the current image block, and the reference image of the current image block.

[0094] The second candidate list of motion vectors for the current image block includes the M candidate motion vectors determined in step S110 and the candidate motion vector determined in step S130. In one example, after determining the M+1th candidate motion vector to be added to the second candidate list of motion vectors in step S130, other candidate motion vectors to be added to the second candidate list of motion vectors are determined according to other methods, which are not limited here.

[0095] After constructing the second candidate list of motion vectors, Figure 1 As shown, the method may further include: S140, determining the motion vector of the current image block according to the second candidate list of motion vectors obtained in step S130.

[0096] It should be understood that the solution provided in this application can be applied to the ATMVP technology. In the first step of the existing ATMVP technology, the temporal vector of the current image block is obtained by scanning all spatial candidate motion vectors currently added to the second candidate motion vector list. For example, the second candidate motion vector list is usually filled with four spatial candidate motion vectors. In this case, the following situation may occur: four candidate motion vectors need to be scanned to obtain the temporal vector of the current image block.

[0097] In the embodiment of the present application, when obtaining the reference motion vector for the current image block, only N (N is less than M) candidate motion vectors of the already obtained M candidate motion vectors are scanned sequentially. Compared to the prior art, this can reduce the number of scans of candidate motion vectors during the process of obtaining the reference motion vector for the current image block. It should be understood that applying the solution provided by the present application to the first step of the existing ATMVP technology can simplify the redundant operations that exist therein.

[0098] The applicant selected the official test sequence as the test sequence on the latest reference software VTM-2.0 of Versatile Video Coding, and tested the solution provided in this application with RA configuration and LDB configuration. The test results showed that after reducing the number of scans, the performance gain of ATMVP technology can still be maintained.

[0099] Therefore, the solution provided in this application can reduce the complexity of ATMVP technology while maintaining the performance gain of existing ATMVP technology.

[0100] It should be understood that the second candidate list of motion vectors formed by the construction scheme provided in this application can be applied to both the encoding end and the decoding end. In other words, the execution subject of the method provided in this application can be the encoding end or the decoding end.

[0101] As an example, the second candidate list of motion vectors formed by the construction solution provided in this application can be applied to the first type of mode mentioned above (such as the Merge mode).

[0102] Optionally, in this embodiment, in step S110, four candidate motion vectors for adding to the second candidate list of motion vectors for the current image block are determined based on the motion vectors of four neighboring blocks of the current image block in the current frame, that is, M is equal to 4. In step S120, N candidate motion vectors among the four candidate motion vectors are scanned, where N is less than 4.

[0103] For example, N is equal to 1. For example, in step S120, only the first candidate motion vector in the second candidate motion vector list is scanned. For another example, N is equal to 2 or 3.

[0104] The following describes a method for determining the reference motion vector of the current image block according to the scanning results of the N candidate motion vectors in step S120.

[0105] In step S120, N of the M candidate motion vectors are determined one by one to determine whether they meet a preset condition, and a reference motion vector is determined based on the determination result. This article describes the preset condition as if the reference frame pointed to by the candidate motion vector is the same as the reference image of the current image block.

[0106] Optionally, in step S120, N candidate motion vectors are scanned in sequence. When the first candidate motion vector that meets the preset conditions is scanned, that is, when the reference frame pointed to by the first candidate motion vector is the same as the co-located frame of the current frame, the scanning is stopped, and the reference motion vector is determined based on the scanned first candidate motion vector that meets the preset conditions.

[0107] It should be understood that when the first candidate motion vector meeting the preset conditions is scanned, the number of scans may be equal to N or less than N. For example, when the first scanned candidate motion vector meets the preset conditions, scanning is stopped and this candidate motion vector is used as the reference motion vector of the current image block.

[0108] Optionally, in step S120, when no candidate motion vector meeting the preset conditions is scanned among the N candidate motion vectors, that is, when the reference frames pointed to by the N candidate motion vectors are all different from the co-located frames of the current image block, the default value is used as the value of the reference motion vector.

[0109] For example, the default value is (0, 0), that is, the reference motion vector is (0, 0). It should be understood that the default value may have other definitions according to actual conditions.

[0110] Optionally, in step S120, when no candidate motion vector meeting the preset conditions is scanned among the N candidate motion vectors, that is, when the reference frames pointed to by the N candidate motion vectors are all different from the co-located frames of the current image block, the specific candidate motion vector in the second candidate list of motion vectors is scaled, and the reference motion vector is determined based on the specific candidate motion vector after scaling.

[0111] The specific candidate motion vector may be the first motion vector or the last motion vector obtained in a scanning order among the N candidate motion vectors.

[0112] The specific candidate motion vector may also be a motion vector obtained in other scanning orders among the N candidate motion vectors.

[0113] When the preset condition is defined as that the reference frame pointed to by the candidate motion vector is the same as the reference frame of the current image block, the specific candidate motion vector in the second candidate list of motion vectors is scaled, and the reference motion vector is determined based on the scaled specific candidate motion vector, including: scaling the specific candidate motion vector in the second candidate list of motion vectors so that the reference frame pointed to by the scaled specific candidate motion vector is the same as the reference image of the current image block; and using the scaled specific candidate motion vector as the reference motion vector.

[0114] like Figure 3 As shown, curr_pic represents the image where the current image block is located, col_pic represents the collocated picture of the current image block, and neigh_ref_pic represents the reference frame pointed to by the specific candidate motion vector. In one implementation, the scaling ratio of the specific motion vector is determined based on the temporal distance between the reference image neigh_ref_pic pointed to by the specific candidate motion vector and the image curr_pic where the image block corresponding to the specific motion vector is located, as well as the temporal distance between the reference image col_pic of the current image block and the image curr_pic where the current image block is located.

[0115] It should be understood that the degree of motion between image frames is less different. In the case of intense motion between the current frame and its collocated frame, if the motion vector (0, 0) is used as the basis for locating the corresponding block of the current block, the motion between frames is not taken into account. Instead, it is directly assumed that the absolute coordinates of the current block in the collocated frame have not changed. In fact, there is a high probability that the coordinates of the current block in the collocated frame are different from those in the current frame, and therefore a large deviation will occur.

[0116] In an embodiment of the present application, if no candidate motion vector is found among the N candidate motion vectors whose reference frame is the same as the collocated frame of the current frame, one of the N candidate motion vectors is scaled so that its reference frame is the same as the collocated frame of the current frame, and then the scaled candidate motion vector is used as the motion vector of the current image block. This can improve the accuracy of the motion vector of the current image block.

[0117] Optionally, when N is an integer less than M and greater than 1, the specific candidate motion vector in this embodiment may be a candidate motion vector whose reference frame and the co-located frame of the current image block are closest in time domain among the N candidate motion vectors.

[0118] It should be understood that selecting a candidate motion vector with the closest distance between the reference frame and the co-located frame of the current frame among the N candidate motion vectors for scaling processing can reduce the time consumption of scaling processing, thereby improving the efficiency of obtaining the motion vector of the current image block.

[0119] Optionally, when N is an integer smaller than M and greater than 1, the specific candidate motion vector in this embodiment may also be any one of the N candidate motion vectors.

[0120] It should be understood that when N is equal to 1, the specific candidate motion vector in this embodiment is the scanned candidate motion vector.

[0121] Optionally, as an embodiment, N is equal to 1, and in step S120, a reference motion vector of the current image block is obtained by scanning a candidate motion vector in the second candidate list of motion vectors. When the reference frame pointed to by the scanned candidate motion vector is different from the collocated frame of the current frame where the current image block is located, the candidate motion vector is scaled so that the reference frame of the scaled candidate motion vector is the same as the collocated frame of the current frame; the candidate motion vector after the scaling is used as the reference motion vector of the current image block. When the reference frame of the scanned candidate motion vector is the same as the collocated frame of the current frame, the candidate motion vector is used as the motion vector of the current image block.

[0122] In this embodiment, a motion vector for the current image block is obtained by scanning a candidate motion vector from the candidate motion vector list, effectively reducing the number of times candidate motion vectors are scanned during the process of obtaining the motion vector for the current image block. When the reference frame of the scanned candidate motion vector differs from the collocated frame of the current frame, the candidate motion vector is scaled so that its reference frame is the same as the collocated frame of the current frame. This scaled candidate motion vector is then used as the motion vector for the current image block, thereby improving the accuracy of the motion vector for the current image block. Therefore, compared to the prior art, the solution provided by the embodiments of the present application can both simplify the process of determining the motion vector for the current image block and improve the accuracy of the motion vector for the current image block.

[0123] It should be understood that when the definition of the preset condition changes, the process of scaling the specific candidate motion vector in the second candidate list of motion vectors must also change accordingly, that is, to ensure that the specific candidate motion vector after scaling meets the preset condition.

[0124] The following describes the process of determining candidate motion vectors to be added to the second candidate list of motion vectors according to the reference motion vector, the current image block, and the reference image of the current image block in step S130.

[0125] Optionally, as an implementation method, determining candidate motion vectors to be continued to be added to the second candidate list of motion vectors based on the reference motion vector, the current image block, and the reference image of the current image block includes: dividing the current image block into multiple sub-image blocks; determining related blocks of the sub-image blocks in the reference image of the current image block based on the reference motion vector; and determining candidate motion vectors to be continued to be added to the second candidate list of motion vectors based on the motion vectors of the related blocks.

[0126] In one example, the motion vector of the associated block of each sub-image block in the current image block is added as a candidate to the second candidate list of motion vectors. When the candidate is used for prediction, the sub-image block is predicted based on the motion vector of the associated block of each sub-image block in the current image block.

[0127] In one example, a representative motion vector of a block associated with a current image block is added as a candidate to a second candidate list of motion vectors, and the candidate is marked as determined according to the ATMVP technique. When using the candidate for prediction, the associated block of the current image block is determined based on the mark and the candidate. The current image block and the associated block are divided into multiple sub-image blocks in the same manner, with each sub-image block in the current image block corresponding to each sub-image block in the associated block. Based on the motion vectors of each sub-image block in the associated block, prediction is performed for the corresponding sub-image blocks in the current image block. Optionally, when a sub-image block in the associated block has an unavailable motion vector, the representative motion vector of the associated block is used to replace the unavailable motion vector, and prediction is performed for the corresponding sub-image block in the current image block. Optionally, when a sub-image block in the associated block has an unavailable motion vector and the representative motion vectors of the associated block are all unavailable, the candidate determined according to the ATMVP technique is discarded from being added to the second candidate list of motion vectors. In one example, when a sub-image block in a related block is unavailable, or a sub-image block in a related block adopts an intra-frame coding mode, it is determined that a sub-image block with an unavailable motion vector appears in the related block.

[0128] Optionally, the representative motion vector of the relevant block of the current image block may refer to the motion vector of the center position of the relevant block, or other motion vectors representing the relevant block, which is not limited here.

[0129] In some video coding standards, the related block may be referred to as a collocated block or a corresponding block.

[0130] For example, if the current image block is a CU, the sub-image blocks obtained by dividing it may be referred to as sub-CUs. Optionally, the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be greater than or equal to 64 pixels. Optionally, the size of the sub-image block and / or the size of the block associated with the sub-image block are both fixed to 8×8 pixels.

[0131] In current ATMVP technology, the sub-image block size is adaptively set at the frame level. The default sub-image block size is 4×4, and when certain conditions are met, the sub-image block size is set to 8×8. For example, on the encoder side, when encoding the current image block, the average block size of each sub-image block in the CU is calculated when the previously coded image block on the same temporal layer is coded in ATMVP mode. When the average block size exceeds a threshold, the sub-image block size of the current image block is set to 8×8; otherwise, the default value of 4×4 is used. Currently, in the next-generation video coding standard (Versatile Video Coding, VVC), motion vectors are stored in an 8×8 size. It should be understood that when the sub-image block size is set to 4×4, the size of the motion vector for this sub-image block (also 4×4) does not meet the motion vector storage granularity in the current standard. Furthermore, in current ATMVP technology, when encoding the current image block, information about the sub-image block size of the previously coded image block on the same temporal layer also needs to be stored.

[0132] In an embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8. On the one hand, it can adapt to the storage granularity of the motion vector specified in the video standard VVC. On the other hand, there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.

[0133] It should be understood that, under the premise that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block may also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.

[0134] Optionally, as another implementation method, determining candidate motion vectors to be continued to be added to the second candidate list of motion vectors based on the reference motion vector, the current image block, and the reference image of the current image block includes: determining related blocks of the current image block in the reference image of the current image block based on the reference motion vector; and determining candidate motion vectors to be continued to be added to the second candidate list of motion vectors based on the motion vector of the related blocks.

[0135] In encoding / decoding technology, an already encoded / decoded image is generally used as a reference image for the current encoding / decoding. In some embodiments, a reference image may be constructed to improve the similarity between the reference image and the current image to be encoded / decoded.

[0136] For example, video content contains a specific encoding / decoding scenario in which the background remains essentially unchanged, while only the foreground changes or moves. Video surveillance, for example, falls into this category. In video surveillance scenarios, the surveillance camera is typically stationary or moves only slowly, so the background can be considered essentially unchanged. In contrast, objects such as people or vehicles captured in the video footage frequently move or change, so the foreground can be considered constantly changing. For such scenarios, a specific reference image can be created that contains only high-quality background information. This specific reference image can include multiple image blocks, each of which is extracted from a previously decoded image. Different image blocks within this specific reference image may be extracted from different previously decoded images. During inter-frame prediction, the background portion of the image to be encoded / decoded can be referenced by this specific reference image. This reduces residual information in inter-frame prediction, thereby improving encoding / decoding efficiency.

[0137] The above is a specific example of a specific reference image. In some implementations, the specific reference image has at least one of the following properties: a composite reference, a long-term reference image, and an image that is not output. The image that is not output refers to an image that is not output for display; generally speaking, the image that is not output exists as a reference image for other images. For example, the specific reference image can be a constructed long-term reference image, or a composite frame that is not output, or a long-term reference image that is not output, etc. In some implementations, the composite frame is also called a composite reference frame.

[0138] In some implementations, a non-specific reference image may be a reference image that does not have at least one of the following properties: a construction frame, a long-term reference image, or an image that is not output. For example, the specific reference image may include reference images other than the construction frame, or reference images other than the long-term reference image, or reference images other than the image that is not output, or reference images other than the constructed long-term reference image, or reference images other than the construction frame that is not output, or reference images other than the long-term reference image that is not output, etc.

[0139] In some implementations, when images in a video can be used as reference images, a distinction can be made between long-term reference images and short-term reference images. The short-term reference image is a concept corresponding to the long-term reference image. A short-term reference image exists in a reference image buffer for a period of time. After a decoded reference image following the short-term reference image is moved into and out of the reference image buffer, the short-term reference image is removed from the reference image buffer. The reference image buffer may also be referred to as a reference image list cache, reference image list, reference frame list cache, or reference frame list, and is collectively referred to herein as the reference image buffer.

[0140] The long-term reference image (or part of the data in the long-term reference image) can always exist in the reference image buffer. The long-term reference image (or part of the data in the long-term reference image) is not affected by the movement of the decoded reference image in and out of the reference image buffer. The long-term reference image (or part of the data in the long-term reference image) will only be moved out of the reference image buffer when the decoding end issues an update instruction operation.

[0141] Short-term reference images and long-term reference images may be called differently in different standards. For example, in standards such as H.264 / Advanced Video Coding (AVC) or H.265 / HEVC, short-term reference images are called short-term reference frames, and long-term reference images are called long-term reference frames. For example, in standards such as the audio video coding standard (AVS) 1-P2, AVS2-P2, and the Institute of Electrical and Electronics Engineers (IEEE) 1857.9-P4, long-term reference images are called background pictures. For example, in standards such as VP8 and VP9, ​​long-term reference images are called golden frames.

[0142] It should be understood that the use of specific terms in the embodiments of this application does not necessarily mean that they must be applied to specific scenarios. For example, calling a long-term reference image a long-term reference frame does not mean that it must be used in technologies corresponding to standards such as H.264 / AVC or H.265 / HEVC.

[0143] The long-term reference image mentioned above can be constructed from image blocks extracted from multiple decoded images, or can be obtained by updating an existing reference frame (e.g., a pre-stored reference frame) using multiple decoded images. Of course, the constructed specific reference image can also be a short-term reference image. Alternatively, the long-term reference image may not be a constructed reference image.

[0144] In the above implementation, the specific reference image may include a long-term reference image, and the non-specific reference image may include a short-term reference image.

[0145] Optionally, the type of reference frame can be identified by a special field in the codestream structure.

[0146] Optionally, when it is determined that the reference image is a long-term reference image, the reference image is determined to be a specific reference image; or, when it is determined that the reference image is a frame that is not output, the reference image is determined to be a specific reference image; or, when it is determined that the reference image is a construction frame, the reference image is determined to be a specific reference image; or, when it is determined that the reference image is a frame that is not output and further determined to be a construction frame, the reference image is determined to be a specific reference image.

[0147] Optionally, various types of reference images may have corresponding identifiers. In this case, the decoding end may determine whether the reference image is a specific reference image based on the identifier of the reference image.

[0148] In some implementations, when it is determined that the reference image has an identifier of a long-term reference image, the reference image is determined to be a specific reference image.

[0149] In some implementations, when it is determined that the reference image has a flag indicating that it is not to be output, the reference image is determined to be a specific reference image.

[0150] In some implementations, when it is determined that the reference image has an identifier of a constructed frame, the reference image is determined to be a specific reference image.

[0151] In some implementations, the reference image is determined to be a specific reference image when it is determined that the reference image has at least two of the following three flags: a long-term reference image flag, a flag indicating that the image is not output, and a flag indicating that the image is a constructed frame or a synthesized reference frame. For example, when it is determined that the reference image has a flag indicating that the image is not output and a flag indicating that the image is a constructed frame, the reference image is determined to be a specific reference image.

[0152] Specifically, an image may have an identifier indicating whether it is an output frame. When an image is indicated as not being output, it indicates that the frame is a reference image. Furthermore, a determination is made as to whether the frame has an identifier indicating a construction frame. If so, the reference image is determined to be a specific reference image. If an image is indicated as being output, the determination as to whether it is a construction frame may be omitted, and the frame may be directly determined to be not a specific reference image. Alternatively, if an image is indicated as not being output but has an identifier indicating that it is not a construction frame, the frame may be determined to be not a specific reference image.

[0153] Optionally, when it is determined by parsing parameters from a picture header, a picture parameter set (PPS), and a slice header that the reference image satisfies one of the following conditions, the reference image is determined to be a specific reference image:

[0154] The reference image is a long-term reference image;

[0155] The reference image is a construction reference image;

[0156] The reference image is an image that is not output;

[0157] When the reference image is an image that is not output, it is further determined that the reference image is a constructed reference image.

[0158] In some implementations of the embodiments of the present application, in the process of determining the motion vector of the current image block, it is involved to use the motion vector of an image block on another image to determine the motion vector of the image block. For convenience of description, the image block is referred to as the first image block, and the image block on the other image to be used is referred to as the temporal reference block or related block of the first image block. It is understandable that the first image block and the temporal reference block (or related block) of the first image block are located on different images. Therefore, in the process of determining the motion vector of the first image block using the motion vector of the temporal reference block (or related block), it may be necessary to scale the motion vector of the temporal reference block (or related block). For convenience of description, the term "related block" is uniformly used herein.

[0159] For example, when the ATMVP technology is applied in constructing an AMVP candidate list, when determining the motion vector of a block related to the current image block according to the ATMVP technology, it is necessary to scale the motion vector of the related block, and then determine the motion vector of the current image block based on the scaled motion vector. Generally speaking, the scaling ratio of the motion vector of the related block is determined based on the temporal distance between the reference image pointed to by the motion vector of the related block and the image where the related block is located, as well as the temporal distance between the reference image of the current image block and the image where the current image block is located.

[0160] In one example, the motion vector of the relevant block is called MV 2, and the reference frame index value of the reference image pointed to by the motion vector MV 2 is x. The reference frame index value x is the difference between the sequence number (for example, POC) of the reference image pointed to by MV 2 and the sequence number of the image where the relevant block is located. The reference frame index value of the reference image of the first image block is called y. The reference frame index value y is the difference between the sequence number of the reference image of the first image block and the sequence number of the image where the first image block is located. Then, the scaling ratio of the motion vector MV 2 is y / x. Optionally, the product of the motion vector MV 2 and y / x can be used as the motion vector of the first image block.

[0161] However, when the motion vector MV 2 of the relevant block points to a specific reference image, or when the reference image of the first image block is a specific reference image, since the temporal distance between the specific reference image and the image where the first image block is located is not clearly defined, scaling the motion vector MV 2 of the relevant block is meaningless.

[0162] Optionally, in this embodiment, when the motion vector of the current image block is determined based on the motion vector of the related block, specifically: when the motion vector of the related block points to a specific reference image, or the reference image of the current image block is a specific reference image, the motion vector of the current image block is determined based on the motion vector of the processed related block, wherein the motion vector of the processed related block is the same as the motion vector of the related block before processing.

[0163] For example, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0164] Optionally, in this embodiment, when the motion vector of the current image block is determined based on the motion vector of the related block, specifically: when the motion vector of the related block points to a specific reference image, or the reference image of the current image block is a specific reference image, the determination of the motion vector of the current image block based on the motion vector of the related block is abandoned.

[0165] In some embodiments, step S120 includes: if no candidate motion vector meeting a preset condition is scanned among the N candidate motion vectors, scaling a specific candidate motion vector in the second candidate list of motion vectors, and determining a reference motion vector based on the scaled specific candidate motion vector. In this case, optionally, the method further includes: when the specific candidate motion vector points to a specific reference image, or the reference image of the current image block is a specific reference image, determining candidate motion vectors to continue adding to the second candidate list of motion vectors based on the processed specific candidate motion vector, wherein the processed specific candidate motion vector is the same as the specific candidate motion vector before the processing.

[0166] The processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0167] In some embodiments, step S120 includes: if no candidate motion vector meeting a preset condition is scanned among the N candidate motion vectors, scaling a specific candidate motion vector in the second candidate motion vector list, and determining a reference motion vector based on the scaled specific candidate motion vector. In this case, the method may further include: when the specific candidate motion vector points to a specific reference image, or the reference image of the current image block is the specific reference image, abandoning the determination of candidate motion vectors to be added to the second candidate motion vector list based on the specific candidate motion vector.

[0168] As can be seen from the above, in the embodiment of the present application, when obtaining the reference motion vector for the current image block, only N (N is less than M) candidate motion vectors out of the M candidate motion vectors already obtained are scanned sequentially. Compared to the prior art, this can reduce the number of scans of candidate motion vectors in the process of obtaining the reference motion vector for the current image block. It should be understood that applying the solution provided by the present application to the existing ATMVP technology can simplify the redundant operations that exist therein.

[0169] When no candidate motion vector whose reference frame is the same as the collocated frame of the current frame is scanned among the N candidate motion vectors, one of the N candidate motion vectors is scaled so that its reference frame is the same as the collocated frame of the current frame, and then the scaled candidate motion vector is used as the motion vector of the current image block, which can improve the accuracy of the motion vector of the current image block.

[0170] Dividing the current image block into sub-image blocks of 8×8 size can, on the one hand, adapt to the storage granularity of motion vectors specified in the video standard VVC, and on the other hand, there is no need to store the size information of the sub-blocks of the previous encoded image block, thus saving storage space.

[0171] like Figure 4 As shown, the embodiment of the present application also provides a video image processing method, which includes the following steps:

[0172] S410 , obtaining M candidate motion vectors for adding to a second candidate list of motion vectors of a current image block.

[0173] Step S410 corresponds to step S110 described above. Please refer to the above for detailed description, which will not be repeated here.

[0174] S420 , sequentially scanning at least some of the M candidate motion vectors, and determining a reference motion vector of the current image block according to the scanning result.

[0175] As an optional implementation, some of the M candidate motion vectors are scanned sequentially, and the reference motion vector of the current image block is determined based on the scan results. In this implementation, step S420 may correspond to step S120 described above, and the detailed description is as above.

[0176] As another optional implementation manner, all candidate motion vectors among the M candidate motion vectors are scanned in sequence, and the reference motion vector of the current image block is determined according to the scanning results.

[0177] It should be understood that, in step S420, the specific method of determining the reference motion vector of the current image block according to the scanning result can refer to the relevant description in the above embodiment, which will not be repeated here.

[0178] S430: Divide the current image block into a plurality of sub-image blocks, wherein the size of the sub-image blocks is fixed to be greater than or equal to 64 pixels.

[0179] For example, the current image block is a CU, and the sub-image blocks obtained by dividing it can be called sub-CUs.

[0180] S440 , determining a related block of the sub-image block in the reference image of the current image block according to the reference motion vector.

[0181] The reference image of the current image block may be a co-located frame of the current image block.

[0182] S450 , determining candidate motion vectors to be added to the second candidate list of motion vectors according to the motion vector of the relevant block.

[0183] In the current ATMVP technology, the size of the sub-image block is set adaptively at the frame level. The size of the sub-image block defaults to 4×4. When certain conditions are met, the size of the sub-image block is set to 8×8. For example, at the encoding end, when encoding the current image block, the average block size of each sub-image block in the CU when the previous coded image block of the same time domain layer is encoded in ATMVP mode is calculated. When the average block size is greater than the threshold, the size of the sub-image block of the current image block is set to 8×8, otherwise the default value of 4×4 is used. In other words, in the prior art, when encoding the current image block, it is also necessary to store information about the size of the sub-image block of the previous coded image block of the same time domain layer.

[0184] In an embodiment of the present application, the size of the sub-image block of the current image block is fixed to be greater than or equal to 64 pixels, and there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.

[0185] Optionally, in this embodiment, the size of the sub-image block and / or the size of the block associated with the sub-image block are both fixed to 8×8 pixels.

[0186] In current ATMVP technology, the sub-image block size is adaptively set at the frame level. The default sub-image block size is 4×4, and when certain conditions are met, the sub-image block size is set to 8×8. For example, on the encoder side, when encoding the current image block, the average block size of each sub-image block in the CU is calculated when the previously coded image block on the same temporal layer is coded in ATMVP mode. When the average block size exceeds a threshold, the sub-image block size of the current image block is set to 8×8; otherwise, the default value of 4×4 is used. Currently, in the next-generation video coding standard (Versatile Video Coding, VVC), motion vectors are stored in an 8×8 size. It should be understood that when the sub-image block size is set to 4×4, the size of the motion vector for this sub-image block (also 4×4) does not meet the motion vector storage granularity in the current standard. Furthermore, in current ATMVP technology, when encoding the current image block, information about the sub-image block size of the previously coded image block on the same temporal layer also needs to be stored.

[0187] In an embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8. On the one hand, it can adapt to the storage granularity of the motion vector specified in the video standard VVC. On the other hand, there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.

[0188] It should be understood that, under the premise that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block may also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.

[0189] Optionally, in step S420, at least part of the candidate motion vectors are scanned in sequence. When the first candidate motion vector meeting the preset conditions is scanned, the scanning is stopped, and a reference motion vector is determined based on the scanned first candidate motion vector meeting the preset conditions.

[0190] Determining the reference motion vector according to the scanned first candidate motion vector that meets the preset conditions may include: taking the first candidate motion vector that meets the preset conditions as the target adjacent block.

[0191] Optionally, the preset condition includes: a reference image of the candidate motion vector is the same as a reference image of the current image block.

[0192] Optionally, step S450 includes: when the motion vector of the relevant block points to a specific reference image, or the reference image of the current image block is a specific reference image, determining the candidate motion vectors to be added to the second candidate list of motion vectors based on the motion vector of the processed relevant block, wherein the motion vector of the processed relevant block is the same as the motion vector of the relevant block before processing.

[0193] For example, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0194] Optionally, step S450 includes: when the motion vector of the relevant block points to a specific reference image, or the reference image of the current image block is a specific reference image, abandoning the candidate motion vectors determined based on the motion vector of the relevant block to be added to the second candidate list of motion vectors.

[0195] therefore, Figure 4 In the embodiment shown, the current image block is divided into sub-image blocks of 8×8 size. On the one hand, this can adapt to the storage granularity of motion vectors specified in the video standard VVC. On the other hand, there is no need to store information about the size of the sub-blocks of the previous encoded image block, thereby saving storage space.

[0196] The above describes how to determine the candidate to be added to the second motion vector candidate list according to the ATMVP technology. In some implementations, other candidates may be added to the second motion vector candidate list, which is not limited here.

[0197] Combined with the above Figure 1 and Figure 4 Having described the method embodiments of the present application, the following describes the corresponding device embodiments of the above method embodiments. It should be understood that the description of the device embodiments corresponds to the description of the method embodiments. Therefore, for matters not described in detail, reference can be made to the above method embodiments. For the sake of brevity, they will not be repeated here.

[0198] Figure 5 Schematic block diagram of a video image processing device 500 provided in an embodiment of the present application. The device 500 is used to perform the following Figure 1 The method embodiment shown in FIG. 5 includes the following units.

[0199] An acquiring unit 510 is configured to acquire M candidate motion vectors for adding to a second candidate list of motion vectors of a current image block;

[0200] a determining unit 520 configured to sequentially scan N candidate motion vectors among the M candidate motion vectors and determine a reference motion vector according to the scanning results, where N is less than M;

[0201] The determining unit 520 is further configured to determine, based on the reference motion vector, the current image block, and the reference image of the current image block, candidate motion vectors to be added to the second candidate list of motion vectors;

[0202] The determining unit 520 is further configured to determine the motion vector of the current image block according to the second candidate list of motion vectors.

[0203] In the first step of the existing ATMVP technology, the temporal vector of the current image block is obtained by scanning all candidate motion vectors currently added to the second motion vector candidate list. For example, the second motion vector candidate list is usually filled with four candidate motion vectors. In this case, the following situation may occur: four candidate motion vectors need to be scanned to obtain the temporal vector of the current image block.

[0204] In the embodiment of the present application, when obtaining the reference motion vector for the current image block, only N (N is less than M) candidate motion vectors of the already obtained M candidate motion vectors are scanned sequentially. Compared to the prior art, this can reduce the number of scans of candidate motion vectors during the process of obtaining the reference motion vector for the current image block. It should be understood that applying the solution provided by the present application to the first step of the existing ATMVP technology can simplify the redundant operations that exist therein.

[0205] The applicant selected the official test sequence as the test sequence on the latest reference software VTM-2.0 of Versatile Video Coding, and tested the solution provided in this application with RA configuration and LDB configuration. The test results showed that after reducing the number of scans, the performance gain of ATMVP technology can still be maintained.

[0206] Therefore, the solution provided in this application can reduce the complexity of ATMVP technology while maintaining the performance gain of existing ATMVP technology.

[0207] Optionally, as an embodiment, the acquisition unit 510 is configured to acquire M candidate motion vectors for adding to the second candidate list of motion vectors of the current image block according to the motion vectors of M neighboring blocks of the current image block in the current frame.

[0208] Optionally, as an embodiment, the neighboring block is an image block that is adjacent to the current image block or has a certain positional distance from the current image block in the current frame.

[0209] Optionally, as an embodiment, the determining unit 520 is configured to sequentially scan the first N candidate motion vectors among the M candidate motion vectors.

[0210] Optionally, as an embodiment, M is equal to 4 and N is less than 4.

[0211] Optionally, as an embodiment, N is equal to 1 or 2.

[0212] Optionally, as an embodiment, the determining unit 520 is configured to, based on a preset condition, sequentially scan N candidate motion vectors among the M candidate motion vectors, and determine the reference motion vector according to the scanning result.

[0213] Optionally, as an embodiment, the preset condition includes: a candidate motion vector pointing to a reference frame that is the same as a reference image of the current image block.

[0214] Optionally, as an embodiment, the determination unit 520 is used to scan N candidate motion vectors in sequence, stop scanning when the first candidate motion vector that meets the preset conditions is scanned, and determine the reference motion vector based on the scanned first candidate motion vector that meets the preset conditions.

[0215] Optionally, as an embodiment, the determination unit 520 is used to, when no candidate motion vector meeting the preset conditions is scanned among the N candidate motion vectors, scale the specific candidate motion vector in the second candidate list of motion vectors, and determine the reference motion vector based on the specific candidate motion vector after scaling.

[0216] Optionally, as an embodiment, the specific candidate motion vector is the first motion vector or the last motion vector obtained in a scanning order among the N candidate motion vectors.

[0217] Optionally, as an embodiment, the determination unit 520 is used to scale a specific candidate motion vector in the second candidate list of motion vectors so that the reference frame pointed to by the scaled specific candidate motion vector is the same as the reference image of the current image block; and the scaled specific candidate motion vector is used as a reference motion vector.

[0218] Optionally, as an embodiment, the determining unit 520 is configured to use a default value as a reference motion vector when no candidate motion vector meeting a preset condition is scanned among the N candidate motion vectors.

[0219] Optionally, as an embodiment, the default value is the motion vector (0, 0).

[0220] Optionally, as an embodiment, the determination unit 520 is used to divide the current image block into multiple sub-image blocks; determine the relevant blocks of the sub-image blocks in the reference image of the current image block based on the reference motion vector; and determine the candidate motion vectors to be continued to be added to the second candidate list of motion vectors based on the motion vectors of the relevant blocks.

[0221] Optionally, as an embodiment, the size of the sub-image block and / or the size of the associated block of the sub-image block is fixed to be greater than or equal to 64 pixels.

[0222] Optionally, as an embodiment, the current image block is a coding unit CU.

[0223] Optionally, as an embodiment, the determination unit 520 is used to determine a related block of the current image block in the reference image of the current image block based on the reference motion vector; and determine a candidate motion vector to be added to the second candidate list of motion vectors based on the motion vector of the related block.

[0224] Optionally, as an embodiment, the determination unit 520 is used to determine, when the motion vector of the relevant block points to a specific reference image, or the reference image of the current image block is a specific reference image, the candidate motion vector to be added to the second candidate list of motion vectors based on the motion vector of the processed relevant block, wherein the motion vector of the processed relevant block is the same as the motion vector of the relevant block before processing.

[0225] Optionally, as an embodiment, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0226] Optionally, as an embodiment, the determination unit 520 is used to abandon determining the candidate motion vector to be added to the second candidate list of motion vectors based on the motion vector of the relevant block when the motion vector of the relevant block points to a specific reference image, or the reference image of the current image block is a specific reference image.

[0227] Optionally, as an embodiment, the determination unit 520 is used to determine, when a specific candidate motion vector points to a specific reference image, or the reference image of the current image block is a specific reference image, the candidate motion vector to be continued to be added to the second candidate list of motion vectors based on the processed specific candidate motion vector, wherein the processed specific candidate motion vector is the same as the specific candidate motion vector before processing.

[0228] Optionally, as an embodiment, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0229] Optionally, as an embodiment, the determination unit 520 is used to abandon determining the candidate motion vector to be added to the second candidate list of motion vectors based on the specific candidate motion vector when the specific candidate motion vector points to a specific reference image, or the reference image of the current image block is a specific reference image.

[0230] Optionally, as an embodiment, the second motion vector candidate list is a Merge candidate list.

[0231] Optionally, as an embodiment, the reference image of the current image block is a co-located frame of the current image block.

[0232] Optionally, as an embodiment, the size of the sub-image block and / or the size of the block associated with the sub-image block are both fixed to 8×8 pixels.

[0233] It should be understood that both the acquiring unit 510 and the determining unit 520 in this embodiment can be implemented by a processor.

[0234] like Figure 6 As shown, the embodiment of the present application further provides a video image processing device 600. The device 600 is used to perform the following Figure 4 The method embodiment shown in FIG. 6 includes the following units.

[0235] An acquiring unit 610 is configured to acquire M candidate motion vectors for adding to a second candidate list of motion vectors of a current image block;

[0236] a determining unit 620 configured to sequentially scan at least some of the M candidate motion vectors and determine a reference motion vector of a current image block based on the scanning result;

[0237] a dividing unit 630, configured to divide the current image block into a plurality of sub-image blocks, wherein the size of the sub-image blocks is fixed to be greater than or equal to 64 pixels;

[0238] The determining unit 620 is further configured to determine a related block of the sub-image block in the reference image of the current image block according to the reference motion vector;

[0239] The determining unit 620 is further configured to determine, according to the motion vector of the relevant block, candidate motion vectors to be added to the second candidate list of motion vectors.

[0240] In the current ATMVP technology, the size of the sub-image block is set adaptively at the frame level. The size of the sub-image block defaults to 4×4. When certain conditions are met, the size of the sub-image block is set to 8×8. For example, at the encoding end, when encoding the current image block, the average block size of each sub-image block in the CU when the previous coded image block of the same time domain layer is encoded in ATMVP mode is calculated. When the average block size is greater than the threshold, the size of the sub-image block of the current image block is set to 8×8, otherwise the default value of 4×4 is used. In other words, in the prior art, when encoding the current image block, it is also necessary to store information about the size of the sub-image block of the previous coded image block of the same time domain layer.

[0241] In an embodiment of the present application, the size of the sub-image block of the current image block is fixed to be greater than or equal to 64 pixels, and there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.

[0242] Optionally, as an embodiment, the size of the sub-image block and / or the size of the block associated with the sub-image block are both fixed to 8×8 pixels.

[0243] Currently, in the new generation video coding standard (Versatile Video Coding, VVC), motion vectors are stored in an 8×8 size. In the embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8. On the one hand, it can adapt to the storage granularity of motion vectors specified in the video standard VVC. On the other hand, there is no need to store the size information of the sub-image block of the previous encoded image block, thus saving storage space.

[0244] Optionally, as an embodiment, the determination unit 620 is used to scan at least part of the candidate motion vectors in sequence, stop scanning when the first candidate motion vector that meets the preset conditions is scanned, and determine the reference motion vector based on the scanned first candidate motion vector that meets the preset conditions.

[0245] Optionally, as an embodiment, the determining unit 620 is configured to use the first candidate motion vector that meets a preset condition as the target neighboring block.

[0246] Optionally, as an embodiment, the preset condition includes: a reference image of the candidate motion vector is the same as a reference image of the current image block.

[0247] It should be understood that the obtaining unit 610 , the determining unit 620 , and the dividing unit 630 in this embodiment can all be implemented by a processor.

[0248] In the above description, the motion vector of an image block contains two pieces of information: 1) the image to which the motion vector points; 2) the displacement. In some application scenarios, the motion vector of an image block only contains the information of "displacement". The image block also provides index information for indicating the reference image of the image block. For an encoded / decoded image block, the meaning contained in its motion vector includes: the displacement of the reference block of the encoded / decoded image block on the reference image relative to the image block that is at the same position as the encoded / decoded image block and is located in the reference image. When determining the reference block of the encoded / decoded image block, it is necessary to determine the reference block of the encoded / decoded image block through the index information of the reference image of the encoded / decoded image block and the motion vector of the encoded / decoded image block. Then, in Figure 1 In the video image processing method shown, step S120 does not scan the candidate motion vectors in the second candidate list of motion vectors, but directly scans the image block corresponding to the candidate motion vector. The video image processing method is provided below for the new definition of the motion vector (i.e., including "displacement" information but not including "the image pointed to"). It should be noted that the methods for determining candidate motion vectors based on the ATMVP technology provided for the two different meanings of "motion vector" are basically the same, and the above explanations are also applicable to the video image processing method provided below. The main difference is that when constructing the second candidate list of motion vectors and determining the candidates to be added to the second candidate list of motion vectors according to the ATMVP technology, the video image processing method described above scans the motion vectors already added to the second candidate list of motion vectors, while the video image processing method provided below scans the image blocks corresponding to the motion vectors already added to the second candidate list of motion vectors.

[0249] like Figure 7 As shown, an embodiment of the present application provides a video image processing method, which includes the following steps.

[0250] S710: Determine M neighboring blocks of the current image block.

[0251] The current image block is an image block to be encoded (or decoded). For example, the current image block is a coding unit (CU).

[0252] The image frame where the current image block is located is called the current frame.

[0253] The neighboring blocks are image blocks that are adjacent to the current image block or have a certain positional distance from the current image block in the current image.

[0254] The M neighboring blocks are image blocks that have been encoded (or decoded) in the current frame.

[0255] As an example, Figure 2 As shown, press Figure 2 The order of the image blocks located at the four positions A1 (left) → B1 (upper) → B0 (upper right) → A0 (lower left) around the current image block shown in is used to determine the four neighboring blocks of the current image block in sequence.

[0256] S720 , sequentially scan N adjacent blocks among the M adjacent blocks, and determine a target adjacent block according to the scanning results, where N is less than M.

[0257] The process of determining the target neighboring block according to the scanning results of the N neighboring blocks may be to judge the N neighboring blocks in sequence based on preset conditions, and determine the target neighboring block according to the judgment results.

[0258] As an example, the preset condition is defined as that the reference image of the neighboring block is the same as the reference image of the current image block.

[0259] Among them, the reference image of the current image block is the reference image with the shortest time distance to the image where the current image block is located; or, the reference image of the current image block is the reference image preset by the codec end; or, the reference image of the current image block is the reference image specified in the video parameter set, sequence header, sequence parameter set, image header, image parameter set, and slice header.

[0260] For example, the reference image of the current image block is the collocated frame of the current image block, and the collocated frame is a frame set in the slice-level information header and used to obtain motion information for prediction.

[0261] It should be understood that, depending on future technological evolution, this preset condition may be given other different definitions, and the corresponding solutions also fall within the scope of protection of this application.

[0262] The process of determining the target neighboring block according to the scanning results of N neighboring blocks will be described in detail below.

[0263] In step S720, only N adjacent blocks among the M adjacent blocks obtained in step S710 are scanned, which can reduce the number of scans.

[0264] Optionally, in step S720, the first N adjacent blocks among the M adjacent blocks may be scanned in sequence.

[0265] When the M neighboring blocks of the current image block are determined sequentially according to a preset order in step S710 , the first N neighboring blocks obtained in step S720 refer to the N neighboring blocks first determined according to the preset order.

[0266] Optionally, in step S720, the last N adjacent blocks among the M adjacent blocks may be scanned in sequence; or the middle N adjacent blocks among the M adjacent blocks may be scanned in sequence. This application does not impose any limitation on this.

[0267] S730 , determining a related block of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block.

[0268] S740 , encoding / decoding the current image block according to the motion vector of the related block.

[0269] Optionally, step S740 includes: determining a reference block of the current image block according to the motion vector of the relevant block and the reference image.

[0270] For example, step S740 includes: constructing a candidate block list for the current image block, where the candidate blocks in the candidate block list include M adjacent blocks and related blocks; and encoding and decoding the current image block according to the reference blocks of the candidate blocks in the candidate block list.

[0271] In one example, the candidate block list is a merge candidate list of the current image block. In another example, the candidate block list is an AMVP candidate list of the current image block.

[0272] On the encoder side, the index of the candidate block for the current block is written into the bitstream. On the decoder side, after obtaining the index, the candidate block corresponding to the index is found in the candidate block list. The reference block of the current image block is determined based on the reference block of the candidate block, or the motion vector of the current image block is determined based on the motion vector of the candidate block.

[0273] For example, the reference block of the candidate block is directly used to determine the reference block of the current image block, or the motion vector of the candidate block is directly used to determine the motion vector of the current image block. In another example, the encoder also writes the MVD of the current block into the bitstream. After obtaining the MVD, the decoder adds the motion vector of the candidate block to the MVD as the motion vector of the current block, and then determines the reference block of the current block based on this motion vector and the reference image of the current block.

[0274] In an embodiment of the present application, in the process of obtaining the target neighboring block of the current image block, only N (N is less than M) neighboring blocks among the M neighboring blocks that have been obtained are scanned in sequence. Compared with the prior art, the number of scans of candidate neighboring blocks in the process of obtaining the target neighboring block of the current image block can be reduced, thereby reducing complexity.

[0275] Optionally, in this embodiment, in step S710 , four neighboring blocks of the current image block in the current frame are determined, that is, M is equal to 4. In step S720 , N neighboring blocks of the four neighboring blocks are scanned, where N is less than 4.

[0276] For example, N is equal to 1. For example, in step S720, only the first neighboring block among the four neighboring blocks is scanned.

[0277] For another example, N is equal to 2 or 3.

[0278] The following describes the method of determining the target neighboring block according to the scanning results of the N neighboring blocks in step S720.

[0279] Optionally, in step S720, N adjacent blocks are scanned in sequence. When the first adjacent block meeting the preset condition is scanned, the scanning is stopped, and a target adjacent block is determined based on the first scanned adjacent block meeting the preset condition.

[0280] For example, the preset condition is defined as the reference image of the neighboring block being the same as the reference image of the current image block.

[0281] It should be understood that in future evolving technologies, the preset conditions may be given other definitions.

[0282] Hereinafter, description will be given by taking the case where the preset condition is defined as the reference image of the neighboring block being the same as the reference image of the current image block as an example.

[0283] For example, the first neighboring block that meets the preset conditions is used as the target neighboring block.

[0284] Optionally, when in step S720, no neighboring block meeting the preset conditions is scanned among the N neighboring blocks, the method further includes: scaling the motion vector of a specific neighboring block among the M neighboring blocks, and encoding / decoding the current image block according to the scaled motion vector.

[0285] For example, the reference block of the current image block is determined according to the scaled motion vector and the reference image of the current image block.

[0286] Optionally, the specific neighboring block is the first neighboring block or the last neighboring block obtained in a scanning order among the N neighboring blocks.

[0287] The specific neighboring block may also be a neighboring block obtained in other scanning orders among the N neighboring blocks.

[0288] Optionally, encoding / decoding the current image block according to the scaled motion vector includes: scaling the motion vector of a specific neighboring block so that the reference frame pointed to by the scaled motion vector is the same as the reference image of the current image block; and using the image block pointed to by the scaled motion vector in the reference image of the current image block as the reference block of the current image block.

[0289] Optionally, when no adjacent block meeting the preset condition is scanned among the N adjacent blocks in step S720, the default block is used as a candidate reference block for the current image block.

[0290] For example, the default block is the image block pointed to by the motion vector (0, 0).

[0291] The following describes the process of determining the related blocks of the current image block according to the motion vector of the target neighboring block, the current image block, and the reference image of the current image block in step S730.

[0292] Optionally, as an implementation method, determining the relevant blocks of the current image block based on the motion vector of the target neighboring block, the current image block, and the reference image of the current image block includes: dividing the current image block into multiple sub-image blocks; and determining the relevant blocks of the sub-image blocks in the reference image of the current image block based on the motion vector of the target neighboring block, wherein the relevant blocks of the current image block include the relevant blocks of the sub-image blocks.

[0293] The related block may be referred to as a collocated block or a corresponding block.

[0294] For example, the current image block is a CU, and the sub-image blocks obtained by dividing it can be called sub-CUs.

[0295] Optionally, the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be greater than or equal to 64 pixels.

[0296] Optionally, the size of the sub-image block and / or the size of the block associated with the sub-image block are both fixed to 8×8 pixels.

[0297] In current ATMVP technology, the sub-image block size is adaptively set at the frame level. The default sub-image block size is 4×4, and when certain conditions are met, the sub-image block size is set to 8×8. For example, on the encoder side, when encoding the current image block, the average block size of each sub-image block in the CU is calculated when the previously coded image block on the same temporal layer is coded in ATMVP mode. When the average block size exceeds a threshold, the sub-image block size of the current image block is set to 8×8; otherwise, the default value of 4×4 is used. Currently, in the next-generation video coding standard (Versatile Video Coding, VVC), motion vectors are stored in an 8×8 size. It should be understood that when the sub-image block size is set to 4×4, the size of the motion vector for this sub-image block (also 4×4) does not meet the motion vector storage granularity in the current standard. Furthermore, in current ATMVP technology, when encoding the current image block, information about the sub-image block size of the previously coded image block on the same temporal layer also needs to be stored.

[0298] In an embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8. On the one hand, it can adapt to the storage granularity of the motion vector specified in the video standard VVC. On the other hand, there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.

[0299] It should be understood that, under the premise that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block may also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.

[0300] Optionally, as another implementation method, determining the relevant blocks of the current image block based on the motion vector of the target neighboring block, the current image block and the reference image of the current image block includes: determining the relevant blocks of the current image block in the reference image of the current image block based on the motion vector of the target neighboring block.

[0301] Optionally, step S740 includes: when the reference image of the relevant block is a specific reference image, or the reference image of the current image block is a specific reference image, determining a candidate reference block of the current image block based on the motion vector of the processed relevant block and the reference image of the current image block; wherein the motion vector of the processed relevant block is the same as the motion vector of the relevant block before processing.

[0302] For example, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0303] Optionally, step S740 includes: when the reference image of the relevant block is a specific reference image, or the reference image of the current block is a specific reference image, abandoning determining the candidate reference block of the current image block according to the motion vector of the relevant block.

[0304] In some embodiments, step S720 includes: when the motion vector of a specific neighboring block points to a specific reference image, or the reference image of the current image block is a specific reference image, determining the reference block of the current image block based on the motion vector of the processed related block and the reference image of the current image block; wherein the motion vector of the processed related block is the same as the motion vector of the related block before processing.

[0305] The processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0306] As can be seen from the above, in the embodiment of the present application, in the process of obtaining the target neighboring block of the current image block, only N (N is less than M) neighboring blocks among the M neighboring blocks that have been obtained are scanned in sequence. Compared with the prior art, the number of scans of candidate neighboring blocks in the process of obtaining the target neighboring block of the current image block can be reduced, thereby reducing complexity.

[0307] When no adjacent block whose reference frame is the same as the collocated frame of the current frame is scanned among the N adjacent blocks, the motion vector of one of the N adjacent blocks is scaled so that its reference frame is the same as the collocated frame of the current frame, and then the scaled motion vector is used as the motion vector of the current image block, thereby improving the accuracy of the motion vector of the current image block.

[0308] Dividing the current image block into sub-image blocks of 8×8 size can, on the one hand, adapt to the storage granularity of motion vectors specified in the video standard VVC, and on the other hand, there is no need to store the size information of the sub-blocks of the previous encoded image block, thus saving storage space.

[0309] In some implementations of the embodiments of the present application, in the process of determining the motion vector of the current image block, it is involved to use the motion vector of an image block on another image to determine the motion vector of the image block. For convenience of description, the image block is referred to as the first image block, and the image block on the other image to be used is referred to as the temporal reference block or related block of the first image block. It is understandable that the first image block and the temporal reference block (or related block) of the first image block are located on different images. Therefore, in the process of using the motion vector of the temporal reference block (or related block) to determine the motion vector of the first image block, it may be necessary to scale the motion vector of the temporal reference block (or related block). For convenience of description, the term "related block" is uniformly used herein.

[0310] For example, when the ATMVP technology is applied in constructing an AMVP candidate list, after determining the related blocks of the current image block according to the ATMVP technology, when determining the motion vector of the current image block according to the motion vector of the related block, it is necessary to scale the motion vector of the related block, and then determine the motion vector of the current image block based on the scaled motion vector. Generally speaking, the scaling ratio of the motion vector of the related block is determined based on the temporal distance between the reference image pointed to by the motion vector of the related block and the image where the related block is located, as well as the temporal distance between the reference image of the current image block and the image where the current image block is located.

[0311] In one example, the motion vector of the relevant block is called MV 2, and the reference frame index value of the reference image pointed to by the motion vector MV 2 is x. The reference frame index value x is the difference between the sequence number (for example, POC) of the reference image pointed to by MV 2 and the sequence number of the image where the relevant block is located. The reference frame index value of the reference image of the first image block is called y. The reference frame index value y is the difference between the sequence number of the reference image of the first image block and the sequence number of the image where the first image block is located. Then, the scaling ratio of the motion vector MV 2 is y / x. Optionally, the product of the motion vector MV 2 and y / x can be used as the motion vector of the first image block.

[0312] However, when the motion vector MV 2 of the relevant block points to a specific reference image, or when the reference image of the first image block is a specific reference image, since the temporal distance between the specific reference image and the image where the first image block is located is not clearly defined, scaling the motion vector MV 2 of the relevant block is meaningless.

[0313] Optionally, in this embodiment, when the motion vector of the current image block is determined based on the motion vector of the related block, specifically: when the motion vector of the related block points to a specific reference image, or the reference image of the current image block is a specific reference image, the motion vector of the current image block is determined based on the motion vector of the processed related block, wherein the motion vector of the processed related block is the same as the motion vector of the related block before processing.

[0314] For example, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0315] Optionally, in this embodiment, when the motion vector of the current image block is determined based on the motion vector of the related block, specifically: when the motion vector of the related block points to a specific reference image, or the reference image of the current image block is a specific reference image, the determination of the motion vector of the current image block based on the motion vector of the related block is abandoned.

[0316] like Figure 8 As shown, an embodiment of the present application also provides a video image processing method, which includes the following steps.

[0317] S810: Determine M neighboring blocks of the current image block.

[0318] Step S810 may correspond to step S710 in the above embodiment.

[0319] S820 , sequentially scan at least some of the M neighboring blocks, and determine a target neighboring block according to the scanning result.

[0320] Optionally, some of the M neighboring blocks are scanned in sequence, and the target neighboring block is determined according to the scanning result.

[0321] Optionally, all the M neighboring blocks are scanned in sequence, and the target neighboring block is determined according to the scanning result.

[0322] S830: Divide the current image block into a plurality of sub-image blocks, wherein the size of the sub-image blocks is fixed to be greater than or equal to 64 pixels.

[0323] S840 , determining a related block of the current image block in a reference image of the current image block according to the motion vector of the target neighboring block and the sub-image block.

[0324] Optionally, the reference image of the current image block is a reference image that is closest in time distance to the image where the current image block is located.

[0325] Optionally, the reference image of the current image block is a reference image preset by the codec.

[0326] Optionally, the reference image of the current image block is a reference image specified in a video parameter set, a sequence header, a sequence parameter set, a picture header, a picture parameter set, or a slice header.

[0327] S850: Encode / decode the current image block according to the motion vector of the related block.

[0328] In an embodiment of the present application, the size of the sub-image block of the current image block is fixed to be greater than or equal to 64 pixels, and there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.

[0329] Optionally, in this embodiment, the size of the sub-image block and / or the size of the temporal reference block of the sub-image block are both fixed to 8×8 pixels.

[0330] Currently, in the new generation video coding standard (Versatile Video Coding, VVC), motion vectors are stored in an 8×8 size. In the embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8. On the one hand, it can adapt to the storage granularity of motion vectors specified in the video standard VVC. On the other hand, there is no need to store the size information of the sub-image block of the previous encoded image block, thus saving storage space.

[0331] It should be understood that, under the premise that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block may also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.

[0332] Optionally, step S820 includes: scanning at least part of the adjacent blocks in sequence, stopping the scanning when the first adjacent block meeting the preset condition is scanned, and determining the target adjacent block according to the scanned first adjacent block meeting the preset condition.

[0333] For example, the first neighboring block that meets the preset conditions is used as the target neighboring block.

[0334] For example, the preset condition is defined as: the reference image of the neighboring block is the same as the reference image of the current image block.

[0335] Optionally, step S840 includes: determining related blocks of the sub-image block in a reference image of the current image block according to the motion vector of the target neighboring block and the sub-image block, wherein the related blocks of the current image block include related blocks of the sub-image block.

[0336] Combined with the above Figure 7 and Figure 8 The method embodiment of the present application is described below. Figure 7 and Figure 8 The method embodiment shown corresponds to the device embodiment. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so that the content not described in detail can be referred to the previous method embodiment, and for the sake of brevity, it will not be repeated here.

[0337] Figure 9 The schematic block diagram of the video image processing device 900 provided in the embodiment of the present application is shown in FIG. Figure 7 The method embodiment shown in FIG. 9 includes the following units.

[0338] An acquisition unit 910 is configured to acquire M neighboring blocks of a current image block;

[0339] a determining unit 920 configured to sequentially scan N adjacent blocks among the M adjacent blocks and determine a target adjacent block based on the scanning results, where N is less than M;

[0340] The determining unit 920 is further configured to determine a related block of the current image block based on the motion vector of the target neighboring block, the current image block, and a reference image of the current image block;

[0341] The encoding / decoding unit 930 is configured to encode / decode the current image block according to the motion vector of the related block.

[0342] In an embodiment of the present application, in the process of obtaining the target neighboring block of the current image block, only N (N is less than M) neighboring blocks among the M neighboring blocks that have been obtained are scanned in sequence. Compared with the prior art, the number of scans of candidate neighboring blocks in the process of obtaining the target neighboring block of the current image block can be reduced, thereby reducing complexity.

[0343] Optionally, as an embodiment, M is equal to 4 and N is less than 4.

[0344] Optionally, as an embodiment, N is equal to 1 or 2.

[0345] Optionally, as an embodiment, the determining unit 920 is configured to sequentially scan the first N adjacent blocks among the M adjacent blocks.

[0346] Optionally, as an embodiment, the acquisition unit 910 is configured to sequentially acquire M neighboring blocks of the current image block in a preset order; the first N neighboring blocks refer to the N neighboring blocks first determined in the preset order.

[0347] Optionally, as an embodiment, the determination unit 920 is configured to scan N adjacent blocks in sequence, stop scanning when the first adjacent block meeting the preset conditions is scanned, and determine the target adjacent block based on the first adjacent block meeting the preset conditions.

[0348] Optionally, as an embodiment, the determining unit 920 is configured to use the first neighboring block that meets a preset condition as the target neighboring block.

[0349] Optionally, as an embodiment, the preset condition includes: the reference image of the neighboring block is the same as the reference image of the current image block.

[0350] Optionally, as an embodiment, the encoding / decoding unit 930 is configured to determine a reference block of the current image block according to a motion vector of a related block and a reference image.

[0351] Optionally, as an embodiment, the encoding / decoding unit 930 is used to construct a candidate block list for the current image block, where the candidate blocks in the candidate block list include M adjacent blocks and related blocks; and encode and decode the current image block according to the reference blocks of the candidate blocks in the candidate block list.

[0352] Optionally, as an embodiment, the encoding / decoding unit 930 is also used to, when no neighboring block meeting the preset conditions is scanned among the N neighboring blocks, scale the motion vector of a specific neighboring block among the M neighboring blocks, and encode / decode the current image block based on the scaled motion vector.

[0353] Optionally, as an embodiment, the encoding / decoding unit 930 is configured to determine a reference block of the current image block according to the scaled motion vector and the reference image of the current image block.

[0354] Optionally, as an embodiment, the specific neighboring block is the first neighboring block or the last neighboring block obtained in a scanning order among the N neighboring blocks.

[0355] Optionally, as an embodiment, the encoding / decoding unit 930 is used to scale the motion vector of a specific neighboring block so that the reference frame pointed to by the scaled motion vector is the same as the reference image of the current image block; and the image block pointed to by the scaled motion vector in the reference image of the current image block is used as the reference block of the current image block.

[0356] Optionally, as an embodiment, the determining unit 920 is configured to use a default block as a reference block for the current image block when no adjacent block meeting a preset condition is scanned among the N adjacent blocks.

[0357] Optionally, as an embodiment, the default block is the image block pointed to by the motion vector (0, 0).

[0358] Optionally, as an embodiment, the determining unit 920 is configured to:

[0359] Divide the current image block into multiple sub-image blocks;

[0360] According to the motion vector of the target neighboring block, a related block of the sub-image block is determined in the reference image of the current image block, and the related blocks of the current image block include the related blocks of the sub-image block.

[0361] Optionally, as an embodiment, the size of the sub-image block and / or the size of the associated block of the sub-image block is fixed to be greater than or equal to 64 pixels.

[0362] Optionally, as an embodiment, the current image block is a coding unit CU.

[0363] Optionally, as an embodiment, the determining unit 920 is configured to determine a related block of the current image block in a reference image of the current image block according to a motion vector of a target neighboring block.

[0364] Optionally, as an embodiment, the neighboring block is an image block that is adjacent to the current image block or has a certain positional distance from the current image block on the current image.

[0365] Optionally, as an embodiment, the encoding / decoding unit 930 is configured to, when the reference image of the relevant block is a specific reference image, or the reference image of the current image block is a specific reference image, determine the reference block of the current image block according to the processed motion vector of the relevant block and the reference image of the current image block;

[0366] The motion vector of the relevant block after processing is the same as the motion vector of the relevant block before processing.

[0367] Optionally, as an embodiment, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0368] Optionally, as an embodiment, the encoding / decoding unit 930 is configured to abandon determining the reference block of the current image block based on the motion vector of the relevant block when the reference image of the relevant block is a specific reference image or the reference image of the current block is a specific reference image.

[0369] Optionally, as an embodiment, the determination unit 920 is used to: when the motion vector of a specific neighboring block points to a specific reference image, or the reference image of the current image block is a specific reference image, determine the reference block of the current image block based on the motion vector of the processed related block and the reference image of the current image block; wherein the motion vector of the processed related block is the same as the motion vector of the related block before processing.

[0370] Optionally, as an embodiment, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.

[0371] It should be understood that the acquisition unit 910, the determination unit 920 and the encoding / decoding unit 930 in this embodiment can all be implemented by a processor.

[0372] like Figure 10 As shown, the embodiment of the present application further provides a video image processing device 1000. The device 1000 is used to perform the following Figure 8 The method embodiment shown in FIG. 1 includes the following units.

[0373] An acquisition unit 1010 is configured to acquire M neighboring blocks of a current image block;

[0374] A determining unit 1020 is configured to sequentially scan at least some of the M neighboring blocks and determine a target neighboring block based on the scanning result;

[0375] a dividing unit 1030, configured to divide the current image block into a plurality of sub-image blocks, wherein the size of the sub-image blocks is fixed to be greater than or equal to 64 pixels;

[0376] The determining unit 1020 is further configured to determine a related block of the current image block in the reference image of the current image block according to the motion vector of the target neighboring block and the sub-image block;

[0377] The encoding / decoding unit 1040 is configured to encode / decode the current image block according to the motion vector of the related block.

[0378] In an embodiment of the present application, the size of the sub-image block of the current image block is fixed to be greater than or equal to 64 pixels, and there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.

[0379] Optionally, as an embodiment, the size of the sub-image block and / or the size of the temporal reference block of the sub-image block are both fixed to 8×8 pixels.

[0380] Currently, in the new generation video coding standard (Versatile Video Coding, VVC), motion vectors are stored in an 8×8 size. In the embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8. On the one hand, it can adapt to the storage granularity of motion vectors specified in the video standard VVC. On the other hand, there is no need to store the size information of the sub-image block of the previous encoded image block, thus saving storage space.

[0381] It should be understood that, under the premise that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block may also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.

[0382] Optionally, as an embodiment, at least part of the M neighboring blocks are scanned in sequence, and the target neighboring block is determined based on the scanning results, including: scanning at least part of the neighboring blocks in sequence, stopping the scanning when the first neighboring block that meets the preset conditions is scanned, and determining the target neighboring block based on the first scanned neighboring block that meets the preset conditions.

[0383] Optionally, as an embodiment, the determining unit 1020 is configured to use the first neighboring block that meets a preset condition as the target neighboring block.

[0384] Optionally, as an embodiment, the preset condition includes: the reference image of the neighboring block is the same as the reference image of the current image block.

[0385] Optionally, as an embodiment, the determination unit 1020 is used to determine the relevant blocks of the sub-image block in the reference image of the current image block based on the motion vector of the target neighboring block and the sub-image block, wherein the relevant blocks of the current image block include the relevant blocks of the sub-image block.

[0386] It should be understood that the obtaining unit 1010 , the determining unit 1020 , the dividing unit 1030 , and the encoding / decoding unit 1040 in this embodiment can all be implemented by a processor.

[0387] like Figure 11 As shown, an embodiment of the present application further provides a video image processing device 1100. Device 1100 can be used to perform the method embodiment described above. Device 1100 includes a processor 1110 and a memory 1120. Memory 1120 is used to store instructions. Processor 1110 is used to execute the instructions stored in memory 1120. Execution of the instructions stored in memory 1120 causes processor 1110 to perform the method according to the method embodiment described above.

[0388] Alternatively, as Figure 11 As shown, the apparatus 1100 may further include a communication interface 1130 for communicating with an external device. For example, the processor 1110 is configured to control the communication interface 1130 to receive and / or send signals.

[0389] The apparatuses 500, 600, 900, 1000 and 1100 provided in the present application can be applied to an encoder or a decoder.

[0390] The second motion vector candidate list is explained above, and the first motion vector candidate list will be explained below.

[0391] In the motion compensation prediction stage, previous mainstream video coding standards only applied translational motion models. However, in the real world, there are many forms of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In order to improve the efficiency of inter-frame prediction, an affine motion compensation model can be introduced into the coding and decoding technology. Affine motion compensation describes the affine motion field of the image block through the MV of a set of control points. In one example, the affine motion compensation model adopts a four-parameter Affine model, and the set of control points includes two control points (for example, the upper left corner and the upper right corner of the image block). In another example, the affine motion compensation model adopts a six-parameter Affine model, and the set of control points includes three control points (for example, the upper left corner, the upper right corner, and the lower left corner of the image block).

[0392] In one implementation, when constructing the first candidate list of motion vectors, the added candidates may be a set of MVs of control points, or referred to as control point motion vector prediction (CPMVP). Optionally, the first candidate list of motion vectors may be used in Merge mode, specifically, in Affine Merge mode; correspondingly, the first candidate list of motion vectors may be referred to as affine mergecandidate list. In Affine Merge mode, the prediction in the first candidate list of motion vectors is directly used as the CPMV (Control point motion vector) of the current image block, i.e., there is no need to perform an affine motion estimation process.

[0393] In one implementation, the candidates determined according to the ATMVP technique may be added to the first candidate list of motion vectors.

[0394] In one example, the control point motion vector group of the blocks related to the current image block is added as a candidate to the first motion vector candidate list. When the candidate in the first motion vector candidate list is used for prediction, the current image block is predicted based on the control point motion vector group of the blocks related to the current image block.

[0395] In one example, as described above, the representative motion vector of the relevant block of the current image block is added as a candidate to the first candidate list of motion vectors. Furthermore, optionally, the candidate is also marked as determined according to the ATMVP technology. When the candidate in the first candidate list of motion vectors is used for prediction, the relevant block of the current image block is determined based on the mark and the candidate, and the current image block and the relevant block are divided into multiple sub-image blocks in the same manner, and each sub-image block in the current image block corresponds to each sub-image block in the relevant block; based on the motion vector of each sub-image block in the relevant block, the motion vector of the corresponding sub-image block in the current image block is predicted.

[0396] Optionally, when a sub-image block with an unavailable motion vector appears in the relevant block, the representative motion vector of the relevant block is used to replace the unavailable motion vector to predict the corresponding sub-image block in the current image block. Optionally, when all representative motion vectors of the relevant blocks are unavailable, the candidate determined according to the ATMVP technology is abandoned from being added to the second candidate list of the motion vector. In one example, when the sub-image block in the relevant block is unavailable, or the sub-image block in the relevant block uses an intra-frame coding mode, the sub-image block in the relevant block with an unavailable motion vector is determined.

[0397] In which, optionally, each candidate in the first candidate list of motion vectors includes the motion vectors of a group of control points; when the representative motion vector of the relevant block of the current image block is added to the first candidate list of motion vectors, in order to ensure the consistency of the data format, the representative motion vector of the relevant block can be inserted as the motion vector of each control point in the candidate (that is, the motion vector of each control point in the candidate is assigned to the representative motion vector of the relevant block).

[0398] Optionally, the representative motion vector of the relevant block of the current image block may refer to the motion vector of the center position of the relevant block, or other motion vectors representing the relevant block, which is not limited here.

[0399] According to the above description of the second motion vector candidate list, when determining candidates according to the ATMVP technique, it is necessary to determine the related blocks of the current image block. In this solution, when determining candidates to be added to the first motion vector candidate list according to the ATMVP technique, there are two methods for determining the related blocks of the current image block:

[0400] Method 1: Scan N neighboring blocks among the preset M neighboring blocks of the current image block in sequence, determine the target neighboring block based on the scanning results, where N is less than M and M is less than or equal to 4; and determine the related blocks of the current image block based on the motion vector of the target neighboring block, the current image block, and the reference image of the current image block.

[0401] Method 2: Determine M neighboring blocks of the current image block based on M candidates in a second candidate list for the motion vector of the current image block; sequentially scan N neighboring blocks of the M neighboring blocks and determine a target neighboring block based on the scanning results, where N is less than M and M is less than or equal to 4; and determine a related block of the current image block based on the motion vector of the target neighboring block, the current image block, and a reference image of the current image block. The M candidates in the second candidate list for the motion vector may refer to the M neighboring blocks of the current image block.

[0402] Among them, for the description of the two steps of "determining the target neighboring block based on the scanning results" and "determining the related blocks of the current image block based on the motion vector of the target neighboring block, the current image block and the reference image of the current image block" in Method 1 and Method 2, please refer to the explanation above and will not be repeated here.

[0403] In one implementation, a method for determining candidates to be added to a first motion vector candidate list includes: determining, from neighboring blocks of a current image block, a control point motion vector group of neighboring blocks predicted using an affine transformation mode in a specific scanning order; and adding each determined control point motion vector group of the neighboring blocks as a candidate to the first motion vector candidate list.

[0404] In one example, the neighboring block predicted using the affine change mode means that the motion vector of the neighboring block is determined based on the candidates in the affine merge candidate list. In other words, the candidates come from the affine motion model of the spatially neighboring blocks of the current image block using the affine mode; that is, the CPMV of the spatially neighboring blocks using the affine mode is used as the CPMVP of the current block.

[0405] In one example, the control point motion vector group may include the motion vectors of two control points of the neighboring block (for example, the upper left corner and upper right corner of the neighboring block), or include the motion vectors of three control points of the neighboring block (for example, the upper left corner, upper right corner and lower left corner of the image block), depending on whether the four-parameter Affine model or the six-parameter Affine model is adopted.

[0406] In one example, determining a control point motion vector group of a neighboring block for prediction using an affine transformation mode in a specific scanning order includes:

[0407] Determine a control point motion vector group of a first adjacent block in a first scanning order among adjacent blocks on the left side of the current image block;

[0408] Determining a control point motion vector group of a second adjacent block in an upper adjacent block of the current image block according to a second scanning order;

[0409] The control point motion vector group of the first neighboring block and the control point motion vector group of the second neighboring block are added to the first motion vector candidate list.

[0410] For example, Figure 12 As shown, Figure 12 This is a schematic diagram of obtaining candidates for the first candidate list of motion vectors using neighboring blocks of the current image block. On the left side of the current image block, the image block A is scanned sequentially in the order of image block D and image block E, and the control point motion vector group of the first image block that meets the preset conditions is added as a candidate to the first candidate list of motion vectors. On the upper side of the current image block, the image block B is scanned sequentially in the order of image block C, and the control point motion vector group of the first image block that meets the preset conditions is added as a candidate to the first candidate list of motion vectors. Optionally, if no image block that meets the threshold condition is found in this scanning sequence, determining a candidate in this scanning sequence is abandoned.

[0411] In one implementation, a method for determining a candidate to be added to a first candidate list of motion vectors includes:

[0412] constructing motion vectors of some control points of the current image block according to neighboring blocks of the control points;

[0413] The motion vectors of some control points of the current image block are added to the first motion vector candidate list.

[0414] That is, in this implementation, a candidate is added to the first motion vector candidate list by constructing a candidate. In one example, before adding the candidate to the first motion vector candidate list by constructing a candidate, it is first determined whether the number of candidates in the first motion vector candidate list has reached a preset value (e.g., 5). If the number has not reached the preset value, the candidate is added to the first motion vector candidate list by constructing a candidate.

[0415] In one example, the constructed candidate is obtained by combining motion information of neighboring blocks of some control points of the current image block and adding the result as CPMVP to the first candidate list of motion vectors.

[0416] like Figure 13 As shown, Figure 13This is a schematic diagram of constructing the candidates for the first candidate list of motion vectors through the neighboring blocks of the current image block. The current image block has a total of four control points, namely CP1, CP2, CP3, and CP4. Among them, image blocks A0 and A1 are the spatial neighboring blocks of CP1; image blocks A2, B2, and B3 are the spatial neighboring blocks of CP2; image blocks B0 and B1 are the spatial neighboring blocks of CP2, and T is the temporal neighboring block of CP4. The coordinates of control points CP1, CP2, CP3, and CP4 are: (0, 0), (W, 0), (H, 0), and (W, H), respectively, where W and H represent the width and height of the current CU, respectively. The priority for obtaining the motion information of the neighboring blocks of each control point is:

[0417] For CP1, the acquisition priority is: B2->B3->A2. When B2 is available, the MV of B2 is used as the MV of control point CP1; when B2 is unavailable, the MV of B3 is used as the MV of control point CP1; if both B2 and B3 are unavailable, the MV of A1 is used as the MV of control point CP1; if B2, B3, and A1 are all unavailable, the motion information of control point CP1 is unavailable.

[0418] Similarly, for CP2, the acquisition priority is: B1->B0; for CP3, the acquisition priority is: A1->A0; for CP4, the MV of T is directly used as the MV of control point CP4.

[0419] The constructed MV is inserted only when the MVs of the current CU's control points (six-parameter model: CP0, CP1, and CP2; four-parameter model: CP0 and CP1) are all available. Otherwise, skip to the next step. After obtaining the MVs of all control points (if any), the MVs of the control points are combined in different ways to obtain multiple affine candidates. The combination methods are as follows:

[0420] If a four-parameter affine model is used, combining two of the four control point MVs yields one or more candidates. Two combinations are selected: {CP1, CP2} and {CP1, CP3}. The {CP1, CP3} combination transforms the MVs of the two selected control points into the MVs of the top-left and top-right control points of the current CU (CP1 and CP2) according to the four-parameter model.

[0421] If the six-parameter affine model is used, one or more candidates can be obtained by combining three of the MVs of the four control points. Four combinations are available: {CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, and {CP1, CP3, CP4}. The combinations {CP1, CP2, CP3}, {CP2, CP3, CP4}, and {CP1, CP3, CP4} require converting the MVs of the three selected control points into the MVs of the top-left, top-right, and bottom-left control points of the current CU (CP1, CP2, and CP3) based on the six-parameter model.

[0422] In one example, if different combinations of MVs (2 or 3) use different reference frames, the candidate generated by the combination is considered unusable.

[0423] In one implementation, a method for determining candidates to be added to a first motion vector candidate list includes: using a default vector for padding. Optionally, the default vector may be a zero vector or another vector. Optionally, after determining candidates to be added to the first motion vector candidate list using other methods, determining whether the number of candidates currently added to the first candidate list has reached a preset value; if not, padding the first candidate list with the default vector until the number of candidates in the first candidate list reaches the preset value.

[0424] When predicting the current image block using a candidate in the first motion vector candidate list, if the candidate used is at least one candidate other than the candidate determined using the ATMVP technique, a motion vector of a sub-image block in the current image block is derived based on the candidate using an affine motion model. If the candidate used is a candidate determined using the ATMVP technique, then as described above, a reference block for each sub-image block in the current image block is determined based on the motion vector of each sub-image block in the relevant block, the reference blocks of each sub-image block are concatenated into a reference block for the current image block, and a residual of the current image block is calculated based on the reference block.

[0425] The following combination Figure 14 and Figure 15 A video image processing method provided in an embodiment of the present application is described by way of example. Figure 14 As shown, the method includes the following steps.

[0426] S1410 : Scan N neighboring blocks among the preset M neighboring blocks of the current image block in sequence, and determine a target neighboring block according to the scanning results, where N is less than M. Optionally, M is less than or equal to 4.

[0427] S1420 : Determine a related block of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block.

[0428] S1430: Divide the current image block and the related block into a plurality of sub-image blocks in the same manner, with each sub-image block in the current image block corresponding to each sub-image block in the related block in a one-to-one manner.

[0429] S1440 , predicting the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.

[0430] right Figure 14 The explanation of the video image processing method shown can be found above and will not be repeated here.

[0431] like Figure 15 As shown, the method includes the following steps.

[0432] S1510 , determining M neighboring blocks of the current image block according to M candidates in a second candidate list of the motion vector of the current image block.

[0433] S1520 : Scan N adjacent blocks in the M adjacent blocks in sequence, and determine a target adjacent block according to the scanning result, where N is less than M. Optionally, M is less than or equal to 4.

[0434] S1530 : Determine a related block of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block.

[0435] S1540: Determine a specific candidate in a first candidate list of motion vectors of the current image block based on a related block of the current image block, wherein the specific candidate may be the candidate determined based on the ATMVP technology mentioned above.

[0436] S1550: When it is determined to adopt the specific candidate, the current image block and the related block are divided into multiple sub-image blocks in the same manner, and each sub-image block in the current image block corresponds to each sub-image block in the related block in a one-to-one manner.

[0437] S1560 , predicting the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.

[0438] right Figure 15 The explanation of the video image processing method shown can be found above and will not be repeated here.

[0439] Figure 16Schematic block diagram of a video image processing device 1600 provided in an embodiment of the present application. The device 1600 is used to perform the following Figure 14 The method embodiment shown in FIG. 1600 includes the following units.

[0440] A construction module 1610 sequentially scans N neighboring blocks among M preset neighboring blocks of a current image block, determines a target neighboring block based on the scanning results, where N is less than M; determines a related block of the current image block based on a motion vector of the target neighboring block, the current image block, and a reference image of the current image block; and divides the current image block and the related block into a plurality of sub-image blocks using the same method, with each sub-image block in the current image block corresponding to each sub-image block in the related block.

[0441] The prediction module 1620 predicts the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.

[0442] In one example, N is equal to 1 or 2.

[0443] In one example, the prediction module is further configured to: before predicting the corresponding sub-image blocks in the current image block based on the motion vectors of the sub-image blocks in the relevant block, add the representative motion vector of the relevant block as a candidate to a first candidate list of motion vectors;

[0444] When it is determined to adopt the candidate, the prediction module predicts the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the relevant block.

[0445] In one example, predicting the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks includes:

[0446] The motion vectors of the sub-image blocks in the related blocks are used as the motion vectors of the corresponding sub-image blocks in the current image block.

[0447] In one example, the representative motion vector of the relevant block is added as the first candidate to the first candidate list of motion vectors.

[0448] In one example, the representative motion vector of the relevant block includes a motion vector at a center position of the relevant block.

[0449] In one example, the prediction module is further used to: when a sub-image block with an unavailable motion vector appears in the related block, use the representative motion vector of the related block as the motion vector of the sub-image block with an unavailable motion vector, and predict the corresponding sub-image block in the current image block.

[0450] In one example, the prediction module is further used to: when a sub-image block with an unavailable motion vector appears in the related block and the representative motion vector of the related block is unavailable, abandon the prediction of the corresponding sub-image blocks in the current image block based on the motion vectors of each sub-image block in the related block.

[0451] In one example, the prediction module is further configured to: when the sub-image block in the relevant block is unavailable or the sub-image block in the relevant block adopts intra-frame coding mode, determine a sub-image block in the relevant block having an unavailable motion vector.

[0452] In one example, the construction module is further configured to: determine other candidates, and add the other candidates to the first motion vector candidate list, wherein at least one of the other candidates includes a motion vector of a sub-image block.

[0453] In one example, the construction module is further configured to: when it is determined to adopt one of the other candidates, determine the motion vector of the sub-image block in the current image block according to the adopted candidate.

[0454] In one example, the at least one candidate includes motion vectors of a set of control points.

[0455] In one example, the prediction module is also used to:

[0456] When it is determined to adopt a candidate among the at least one candidate, performing an affine transformation on the adopted candidate according to an affine transformation model;

[0457] The sub-image block in the current image block is predicted according to the candidate after the affine transformation.

[0458] In one example, when the affine transformation model includes a four-parameter affine transformation model, each of the at least one candidate includes motion vectors of two control points;

[0459] When the affine transformation model includes a six-parameter affine transformation model, each of the at least one candidate includes motion vectors of three control points.

[0460] In one example, the construction module is further configured to: determine, from neighboring blocks of the current image block, in a specific scanning order, a control point motion vector group of the neighboring blocks that are predicted using an affine transformation mode;

[0461] Each determined control point motion vector group of the neighboring block is added as a candidate to the first motion vector candidate list.

[0462] In one example, determining, from neighboring blocks of the current image block in a specific scanning order, a control point motion vector group of neighboring blocks predicted using an affine transformation mode includes:

[0463] Determine a control point motion vector group of a first adjacent block in a first scanning order among adjacent blocks on the left side of the current image block;

[0464] Determine a control point motion vector group of a second adjacent block in an upper adjacent block of the current image block according to a second scanning order;

[0465] The control point motion vector group of the first neighboring block and the control point motion vector group of the second neighboring block are added to the first motion vector candidate list.

[0466] In one example, the construction module is further configured to: construct motion vectors of some control points of the current image block according to neighboring blocks of the control points;

[0467] The motion vectors of some control points of the current image block are added to the first motion vector candidate list.

[0468] In one example, constructing motion vectors of some control points of the current image block based on neighboring blocks of the control points includes:

[0469] For each control point in the partial control points, specific neighboring blocks of the control point are scanned in sequence according to a third scanning order, and a motion vector of the specific neighboring block that meets a preset condition is used as the motion vector of the control point.

[0470] In one example, the building blocks are also used to:

[0471] When the motion vectors of some control points point to different reference frames respectively, adding the motion vectors of some control points of the current image block to the first motion vector candidate list is abandoned.

[0472] In one example, when the number of candidates in the first motion vector candidate list is greater than a preset value, adding the motion vectors of some control points of the current image block to the first motion vector candidate list is abandoned.

[0473] In one example, the building blocks are also used to:

[0474] Constructing a second candidate list of motion vectors, wherein the candidate added to the second candidate list of motion vectors is a motion vector of an image block;

[0475] When it is determined to adopt a candidate in the second candidate list of motion vectors, the motion vector of the current image block is determined according to the motion vector of the candidate.

[0476] In one example, determining the motion vector of the current image block according to the motion vector of the candidate includes:

[0477] The confirmed candidate is used as the motion vector of the current image block, or the confirmed candidate is scaled and used as the motion vector of the current image block.

[0478] In one example, constructing the second candidate list of motion vectors includes:

[0479] Candidates to be added to the second candidate list of motion vectors are determined according to motion vectors of a plurality of neighboring blocks of the current image block on the current image.

[0480] In one example, the plurality of neighboring blocks of the current image block on the current image include the preset M neighboring blocks.

[0481] In one example, the building blocks are also used to:

[0482] Sequentially taking the motion vectors of the preset M neighboring blocks as M candidates in a preset order, and adding them to the second motion vector candidate list;

[0483] The N adjacent blocks refer to the N adjacent blocks first determined according to the preset order.

[0484] In one example, the building blocks are also used to:

[0485] When the motion vectors of one or more neighboring blocks among the M neighboring blocks are unavailable, candidates to be added to the second motion vector candidate list determined based on the motion vectors of the one or more neighboring blocks are abandoned.

[0486] In one example, sequentially scanning N adjacent blocks among the M adjacent blocks and determining a target adjacent block according to the scanning results includes:

[0487] The N adjacent blocks are scanned in sequence. When a first adjacent block meeting a preset condition is scanned, the scanning is stopped, and a target adjacent block is determined according to the first adjacent block meeting the preset condition.

[0488] In one example, determining the target adjacent block according to the first adjacent block that meets a preset condition found during scanning includes:

[0489] The first adjacent block that meets the preset conditions is used as the target adjacent block.

[0490] In one example, the preset conditions include:

[0491] The reference image of the neighboring block is the same as the reference image of the current image block.

[0492] In one example, the construction module is further used to scale the motion vector of a specific neighboring block among the M neighboring blocks when no neighboring block that meets the preset conditions is scanned among the N neighboring blocks, and the prediction module is further used to predict the current image block based on the scaled motion vector.

[0493] In one example, predicting the current image block according to the scaled motion vector includes:

[0494] A reference block of the current image block is determined according to the scaled motion vector and the reference image of the current image block.

[0495] In one example, the specific neighboring block is the first neighboring block or the last neighboring block obtained in a scanning order among the N neighboring blocks.

[0496] In one example, scaling the motion vector of a specific neighboring block among the M neighboring blocks and predicting the current image block based on the scaled motion vector includes:

[0497] Scaling the motion vector of the specific neighboring block so that the reference frame pointed to by the scaled motion vector is the same as the reference image of the current image block;

[0498] The image block pointed to by the motion vector after the scaling process in the reference image of the current image block is used as the reference block of the current image block.

[0499] In one example, when no adjacent block meeting the preset condition is scanned among the N adjacent blocks, a default block is used as a reference block for the current image block.

[0500] In one example, the default block is the image block pointed to by the motion vector (0, 0).

[0501] In one example, the size of the sub-image block and / or the size of a block related to the sub-image block is fixed to be greater than or equal to 64 pixels.

[0502] In one example, the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to 8×8 pixels, or 16×4 pixels, or 4×16 pixels.

[0503] In one example, the current image block is a coding unit CU.

[0504] In one example, determining the related blocks of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block includes:

[0505] According to the motion vector of the target neighboring block, a related block of the current image block is determined in a reference image of the current image block.

[0506] In one example, the adjacent block is an image block that is adjacent to the current image block or has a certain positional distance from the current image block on the current image.

[0507] In one example, predicting the current image block according to the motion vector of the related block includes:

[0508] When the reference image of the relevant block is a specific reference image, or the reference image of the current image block is a specific reference image, determining a reference block of the current image block according to the processed motion vector of the relevant block and the reference image of the current image block;

[0509] The motion vector of the related block after the processing is the same as the motion vector of the related block before the processing.

[0510] In one example, the processed motion vector of the relevant block includes:

[0511] A motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or

[0512] The motion vector of the relevant block skips the scaling step.

[0513] In one example, predicting the current image block according to the motion vector of the related block includes:

[0514] When the reference image of the related block is a specific reference image, or the reference image of the current block is a specific reference image, determining the reference block of the current image block according to the motion vector of the related block is abandoned.

[0515] In one example, the building blocks are also used to:

[0516] When the motion vector of the specific neighboring block points to a specific reference image, or the reference image of the current image block is a specific reference image, determining the reference block of the current image block according to the processed motion vector of the related block and the reference image of the current image block;

[0517] The motion vector of the related block after the processing is the same as the motion vector of the related block before the processing.

[0518] In one example, the processed motion vector of the relevant block includes:

[0519] A motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or

[0520] The motion vector of the relevant block skips the scaling step.

[0521] In one example, M is less than or equal to 4.

[0522] Figure 17 Schematic block diagram of a video image processing device 1700 provided in an embodiment of the present application. The device 1700 is used to perform the following Figure 15 The method embodiment shown in FIG. 1700 includes the following units.

[0523] A construction module 1710 is configured to determine M neighboring blocks of the current image block based on M candidates in a second candidate list for a motion vector of the current image block; sequentially scan N neighboring blocks among the M neighboring blocks and determine a target neighboring block based on the scanning results, where N is less than M; determine a related block of the current image block based on the motion vector of the target neighboring block, the current image block, and a reference image of the current image block; determine a specific candidate in a first candidate list for a motion vector of the current image block based on the related block of the current image block; and when the specific candidate is determined to be used, divide the current image block and the related block into a plurality of sub-image blocks in the same manner, with each sub-image block in the current image block corresponding one-to-one to each sub-image block in the related block;

[0524] The prediction module 1720 is configured to predict the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.

[0525] In one example, at least one candidate in the first candidate list of motion vectors includes a motion vector of a sub-image block, and each candidate in the second candidate list of motion vectors includes a motion vector of an image block.

[0526] In one example, N is equal to 1 or 2.

[0527] In one example, the M candidates include motion vectors of M neighboring blocks of the current image block on the current image.

[0528] In one example, sequentially scanning N adjacent blocks among the M adjacent blocks and determining a target adjacent block according to the scanning results includes:

[0529] The N adjacent blocks are scanned in sequence. When a first adjacent block meeting a preset condition is scanned, the scanning is stopped, and a target adjacent block is determined according to the first adjacent block meeting the preset condition.

[0530] In one example, determining the target adjacent block according to the first adjacent block that meets a preset condition found during scanning includes:

[0531] The first adjacent block that meets the preset conditions is used as the target adjacent block.

[0532] In one example, the preset conditions include:

[0533] The reference image of the neighboring block is the same as the reference image of the current image block.

[0534] In one example, the construction module is further used to scale the motion vector of a specific neighboring block among the M neighboring blocks when no neighboring block that meets the preset conditions is scanned among the N neighboring blocks, and the prediction module is further used to predict the current image block based on the scaled motion vector.

[0535] In one example, predicting the current image block according to the scaled motion vector includes:

[0536] A reference block of the current image block is determined according to the scaled motion vector and the reference image of the current image block.

[0537] In one example, the specific neighboring block is the first neighboring block or the last neighboring block obtained in a scanning order among the N neighboring blocks.

[0538] In one example, scaling the motion vector of a specific neighboring block among the M neighboring blocks and predicting the current image block based on the scaled motion vector includes:

[0539] Scaling the motion vector of the specific neighboring block so that the reference frame pointed to by the scaled motion vector is the same as the reference image of the current image block;

[0540] The image block pointed to by the motion vector after the scaling process in the reference image of the current image block is used as the reference block of the current image block.

[0541] In one example, when no adjacent block meeting the preset condition is scanned among the N adjacent blocks, a default block is used as a reference block for the current image block.

[0542] In one example, the default block is the image block pointed to by the motion vector (0, 0).

[0543] In one example, the size of the sub-image block and / or the size of a block related to the sub-image block is fixed to be greater than or equal to 64 pixels.

[0544] In one example, the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to 8×8 pixels, or 16×4 pixels, or 4×16 pixels.

[0545] In one example, the current image block is a coding unit CU.

[0546] In one example, determining the related blocks of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block includes:

[0547] According to the motion vector of the target neighboring block, a related block of the current image block is determined in a reference image of the current image block.

[0548] In one example, the adjacent block is an image block that is adjacent to the current image block or has a certain positional distance from the current image block on the current image.

[0549] In one example, predicting the current image block according to the motion vector of the related block includes:

[0550] When the reference image of the relevant block is a specific reference image, or the reference image of the current image block is a specific reference image, determining a reference block of the current image block according to the processed motion vector of the relevant block and the reference image of the current image block;

[0551] The motion vector of the related block after the processing is the same as the motion vector of the related block before the processing.

[0552] In one example, the processed motion vector of the relevant block includes:

[0553] A motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or

[0554] The motion vector of the relevant block skips the scaling step.

[0555] In one example, predicting the current image block according to the motion vector of the related block includes:

[0556] When the reference image of the related block is a specific reference image, or the reference image of the current block is a specific reference image, determining the reference block of the current image block according to the motion vector of the related block is abandoned.

[0557] In one example, the construction module is further configured to: when the motion vector of the specific neighboring block points to a specific reference image, or the reference image of the current image block is a specific reference image, determine the reference block of the current image block according to the processed motion vector of the related block and the reference image of the current image block;

[0558] The motion vector of the related block after the processing is the same as the motion vector of the related block before the processing.

[0559] In one example, the processed motion vector of the relevant block includes:

[0560] A motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or

[0561] The motion vector of the relevant block skips the scaling step.

[0562] In one example, predicting the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks includes:

[0563] The motion vectors of the sub-image blocks in the related blocks are used as the motion vectors of the corresponding sub-image blocks in the current image block.

[0564] In one example, determining a specific candidate in a first candidate list of motion vectors of the current image block according to a related block of the current image block includes:

[0565] The representative motion vector of the block related to the current image block is added to the first motion vector candidate list as the specific candidate.

[0566] In one example, the representative motion vector of the relevant block is added as the first candidate to the first candidate list of motion vectors.

[0567] In one example, the representative motion vector of the relevant block includes a motion vector at a center position of the relevant block.

[0568] In one example, the prediction module is further used to: when a sub-image block with an unavailable motion vector appears in the related block, use the representative motion vector of the related block as the motion vector of the sub-image block with an unavailable motion vector, and predict the corresponding sub-image block in the current image block.

[0569] In one example, the prediction module is further used to: when a sub-image block with an unavailable motion vector appears in the related block and the representative motion vector of the related block is unavailable, abandon the prediction of the corresponding sub-image blocks in the current image block based on the motion vectors of each sub-image block in the related block.

[0570] In one example, the prediction module is further configured to: when the sub-image block in the relevant block is unavailable or the sub-image block in the relevant block adopts intra-frame coding mode, determine a sub-image block in the relevant block having an unavailable motion vector.

[0571] In one example, the prediction module is further configured to: when determining to adopt one of the candidates other than the specific candidate in the second candidate list of the motion vector, perform an affine transformation on the adopted candidate according to an affine transformation model;

[0572] The sub-image block in the current image block is predicted according to the candidate after the affine transformation.

[0573] In one example, in at least one candidate other than the specific candidate in the second candidate list of motion vectors, each candidate includes motion vectors of a group of control points.

[0574] In one example, when the affine transformation model includes a four-parameter affine transformation model, each of the at least one candidate includes motion vectors of two control points;

[0575] When the affine transformation model includes a six-parameter affine transformation model, each of the at least one candidate includes motion vectors of three control points.

[0576] In one example, the prediction module is further configured to: determine, from neighboring blocks of the current image block, in a specific scanning order, a control point motion vector group of the neighboring blocks predicted using an affine transformation mode;

[0577] Each determined control point motion vector group of the neighboring block is added as a candidate to the first motion vector candidate list.

[0578] In one example, determining, from neighboring blocks of the current image block in a specific scanning order, a control point motion vector group of neighboring blocks predicted using an affine transformation mode includes:

[0579] Determine a control point motion vector group of a first adjacent block in a left adjacent block of the current image block according to a first scanning order;

[0580] Determine a control point motion vector group of a second adjacent block in an upper adjacent block of the current image block according to a second scanning order;

[0581] The control point motion vector group of the first neighboring block and the control point motion vector group of the second neighboring block are added to the first motion vector candidate list.

[0582] In one example, the construction module is further configured to: construct motion vectors of some control points of the current image block according to neighboring blocks of the control points;

[0583] The motion vectors of some control points of the current image block are added to the first motion vector candidate list.

[0584] In one example, constructing motion vectors of some control points of the current image block based on neighboring blocks of the control points includes:

[0585] For each control point in the partial control points, specific neighboring blocks of the control point are scanned in sequence according to a third scanning order, and a motion vector of the specific neighboring block that meets a preset condition is used as the motion vector of the control point.

[0586] In one example, the construction module is further configured to: when the motion vectors of some control points point to different reference frames, abandon adding the motion vectors of some control points of the current image block into the first motion vector candidate list.

[0587] In one example, the construction module is further configured to: when the number of candidates in the first motion vector candidate list is greater than a preset value, abandon adding the motion vectors of some control points of the current image block to the first motion vector candidate list.

[0588] In one example, the construction module is further configured to: construct a second candidate list of motion vectors, wherein the candidate added to the second candidate list of motion vectors is a motion vector of an image block;

[0589] When it is determined to adopt a candidate in the second candidate list of motion vectors, the motion vector of the current image block is determined according to the motion vector of the candidate.

[0590] In one example, determining the motion vector of the current image block according to the motion vector of the candidate includes:

[0591] The confirmed candidate is used as the motion vector of the current image block, or the confirmed candidate is scaled and used as the motion vector of the current image block.

[0592] In one example, constructing the second candidate list of motion vectors includes:

[0593] The M candidates added to the second motion vector candidate list are determined according to the motion vectors of the M neighboring blocks of the current image block on the current image.

[0594] In one example, the construction module is further configured to: sequentially take the motion vectors of the preset M neighboring blocks as M candidates in a preset order, and add them to the second motion vector candidate list;

[0595] The N adjacent blocks refer to the N adjacent blocks first determined according to the preset order.

[0596] In one example, the construction module is further configured to: when the motion vectors of one or more neighboring blocks among the M neighboring blocks are unavailable, abandon the candidate to be added to the second candidate list of motion vectors determined based on the motion vectors of the one or more neighboring blocks.

[0597] In one example, M is less than or equal to 4.

[0598] An embodiment of the present application further provides a computer storage medium having a computer program stored thereon. When the computer program is executed by a computer, the computer executes the method provided in the above method embodiment.

[0599] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to perform the method provided in the above method embodiment.

[0600] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any other combination. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0601] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0602] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0603] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0604] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0605] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A video image processing method, characterized in that: include: When a specific neighboring block of the current image block meets a preset condition, determining a related block of the current image block according to a motion vector of the specific neighboring block; Adding a representative motion vector of a related block of the current image block to a motion vector candidate list, wherein the representative motion vector is used to indicate a sub-block-based temporal motion vector motion technique (ATMVP); Scanning neighboring blocks of the current image block in a specific scanning order, and determining a control point motion vector group of the neighboring blocks for prediction using an affine transformation mode; and The determined control point motion vector group of the neighboring block is added as other candidates to the motion vector candidate list.

2. The method according to claim 1, characterized in that The method further comprises: Dividing the current image block into a plurality of sub-image blocks; and The motion vector of the sub-image block of the current image block is predicted according to the control point motion vector group of the neighboring blocks.

3. The method according to claim 1 or 2, characterized in that Scan the neighboring blocks of the current image block in a specific scanning order, and determine that the control point motion vector group of the neighboring blocks predicted using the affine transformation mode includes: Scanning the left neighboring blocks of the current image block in a first scanning order, stopping scanning when a first left neighboring block predicted using an affine transformation mode is scanned, and determining a control point motion vector group of the first left neighboring block; and Scanning upper neighboring blocks of the current image block in a second scanning order, stopping scanning when a first upper neighboring block predicted using an affine transformation mode is scanned, and determining a control point motion vector group of the first upper neighboring block; The step of adding the determined control point motion vector group of the neighboring block as other candidates to the first candidate list of motion vectors of the current image block includes: The control point motion vector group of the first left neighboring block and the control point motion vector group of the first upper neighboring block are added as other candidates to the first candidate list of motion vectors of the current image block.

4. The method according to claim 1 or 2, characterized in that When the affine transformation model adopted by the neighboring block includes a four-parameter affine transformation model, the control point motion vector group of the neighboring block includes motion vectors of two control points; When the affine transformation model adopted by the neighboring block includes a six-parameter affine transformation model, the control point motion vector group of the neighboring block includes motion vectors of three control points.

5. The method according to claim 1, wherein The method further comprises: The current image block is predicted and encoded using the first motion vector candidate list, and a bit stream obtained after encoding is output.

6. A video image processing device, characterized in that: include: a memory for storing instructions; as well as A processor is configured to execute instructions stored in the memory, wherein execution of the instructions stored in the memory causes the processor to perform the following operations: When a specific neighboring block of the current image block meets a preset condition, determining a related block of the current image block according to a motion vector of the specific neighboring block; Adding a representative motion vector of a related block of the current image block to a motion vector candidate list, wherein the representative motion vector is used to indicate a sub-block-based temporal motion vector motion technique (ATMVP); Scanning neighboring blocks of the current image block in a specific scanning order, and determining a control point motion vector group of the neighboring blocks for prediction using an affine transformation mode; and The determined control point motion vector group of the neighboring block is added as other candidates to the motion vector candidate list.

7. A video image processing method, characterized in that: include: When a specific neighboring block of the current image block meets a preset condition, determining a related block of the current image block according to a motion vector of the specific neighboring block; Adding a representative motion vector of a related block of the current image block to a motion vector candidate list, wherein the representative motion vector is used to indicate a sub-block-based temporal motion vector motion technique (ATMVP); Scanning neighboring blocks of the current image block in a specific scanning order, and determining motion vectors of one or more control points of the current image block according to motion vectors of the neighboring blocks; Determine a constructed control point motion vector according to the motion vectors of one or more control points of the current image block; as well as The determined construction control point motion vector is added as another candidate to the motion vector candidate list.

8. The method according to claim 7, characterized in that The method further comprises: Dividing the current image block into a plurality of sub-image blocks; and The motion vector of the sub-image block of the current image block is predicted according to the constructed control point motion vector.

9. The method according to claim 7 or 8, characterized in that Scanning neighboring blocks of the current image block in a specific scanning order and determining motion vectors of one or more control points of the current image block according to motion vectors of the neighboring blocks includes: Scanning a plurality of first neighboring blocks among neighboring blocks of the current image block in a first scanning order, stopping the first scanning when a first available first neighboring block is scanned, and determining a motion vector of a first control point of the current image block according to a motion vector of the first neighboring block; Scanning a plurality of second neighboring blocks among the neighboring blocks of the current image block in a second scanning order, stopping scanning when a first available second neighboring block is scanned, and determining a motion vector of a second control point of the current image block according to a motion vector of the second neighboring block; scanning a plurality of third neighboring blocks among the neighboring blocks of the current image block in a third scanning order, stopping the scanning when a first available third neighboring block is scanned, and determining a motion vector of a third control point of the current image block according to a motion vector of the third neighboring block; determining a motion vector of a fourth control point of the current image block according to a time domain vector of a fourth neighboring block of the current image block; The step of determining and constructing a control point motion vector according to the motion vectors of one or more control points of the current image block includes: The motion vectors of the first to fourth control points are combined into a plurality of combined motion vectors as constructed control point motion vectors.

10. The method according to claim 9, characterized in that Also includes: When the motion vectors of the control points point to different reference frames respectively, the combined motion vector formed by combining the motion vectors of the control points is abandoned and added to the first motion vector candidate list as a constructed control point motion vector.

11. A video image processing method, characterized in that: include: When a specific neighboring block of the current image block meets a preset condition, determining a related block of the current image block according to a motion vector of the specific neighboring block; Dividing the current image block into a plurality of sub-image blocks; determining, according to the motion vector of the specific neighboring block, a related sub-image block of the sub-image block in a reference image of the current image block; determining a motion vector of the sub-image block according to the motion vector of the related sub-image block; wherein, when the reference image of the current image block is a specific reference image and the reference image of the related block is a specific reference image, the reference motion vector of the sub-image block of the current image block is determined according to the unscaled motion vector of the corresponding related sub-image block; The current image block is encoded / decoded according to the motion vector of the sub-image block.

12. The method according to claim 11, characterized in that The preset condition is that the reference frame of the specific neighboring block is the same as the collocated frame of the current image; and / or When the specific neighboring block does not meet the preset condition, the related block of the current image block is determined according to the motion vector of the default value.

13. A non-transitory computer-readable storage medium, characterized in that A code stream formed by a computer program is stored thereon, and when the computer program is executed by a computer, the computer executes the video image processing method according to claim 1, 7 or 11.

14. A video image processing device, characterized in that: include: a memory for storing instructions; as well as A processor is configured to execute instructions stored in the memory, wherein execution of the instructions stored in the memory causes the processor to perform the following operations: When a specific neighboring block of the current image block meets a preset condition, determining a related block of the current image block according to a motion vector of the specific neighboring block; Dividing the current image block into a plurality of sub-image blocks; determining, according to the motion vector of the specific neighboring block, a related sub-image block of the sub-image block in a reference image of the current image block; determining a motion vector of the sub-image block according to the motion vector of the related sub-image block; wherein, when the reference image of the current image block is a specific reference image and the reference image of the related block is a specific reference image, the reference motion vector of the sub-image block of the current image block is determined according to the unscaled motion vector of the corresponding related sub-image block; The current image block is encoded / decoded according to the motion vector of the sub-image block.

15. A method for generating a bit stream, characterized in that: include: When a specific neighboring block of the current image block meets a preset condition, determining a related block of the current image block according to a motion vector of the specific neighboring block; Dividing the current image block into a plurality of sub-image blocks; determining, according to the motion vector of the specific neighboring block, a related sub-image block of the sub-image block in a reference image of the current image block; determining a motion vector of the sub-image block according to the motion vector of the related sub-image block; wherein, when the reference image of the current image block is a specific reference image and the reference image of the related block is a specific reference image, the reference motion vector of the sub-image block of the current image block is determined according to the unscaled motion vector of the corresponding related sub-image block; The current image block is encoded according to the motion vector of the sub-image block to generate a bit stream.

Images