Video image processing method and device
By efficiently scanning and dividing the adjacent blocks of the current image block in video image processing, the redundant operation of ATMVP technology is simplified, complexity is reduced and the accuracy of motion vectors is improved, and candidate motion vector scanning in the prior art is solved Problem of too many times.
Patent Information
- Application Number
- CN202210376602.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-10-30
- Filing Date
- 2019-03-13
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2039-03-13
AI Technical Summary
Existing advanced/optional time domain motion vector prediction technology (ATMVP) has problems with redundant operations and high complexity when building candidate motion vector lists, especially when obtaining the reference motion vector of the current image block, the number of scans has passed. many.
By scanning N adjacent blocks of the preset M adjacent blocks of the current image block sequentially, the target adjacent block is determined, and the motion vector prediction is performed based on the motion vector of the target block and the reference image, and the candidate motion vector is reduced. Number of scans.
It simplifies the complexity of ATMVP technology, reduces redundant operations, maintains the performance gain of the original technology, and improves the accuracy of motion vectors.
Smart Images

Figure CN114938452B_ABST
Abstract
Description
[0001] This application is a divisional application of the Chinese patent application "Video Image Processing Method and Device" with an application date of March 13, 2019 and application number 201980002813.5.
[0002] This application claims priority to PCT applications No. PCT / CN2018 / 081652, PCT / CN2018 / 095710, PCT / CN2018 / 103693, PCT / CN2018 / 107436 and PCT / CN2018 / 112805, the entire contents of which are incorporated herein by reference.
[0003] Copyright Notice
[0004] The disclosure of this patent document contains material that is subject to copyright protection. The copyright is reserved by the copyright owner. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it exists in the official records and files of the Patent and Trademark Office. Technical Field
[0005] The present application relates to the field of video coding and decoding, and specifically to a video image processing method and device. Background Art
[0006] At present, the main video coding standards all use block-based motion compensation technology in the inter-frame prediction part. Its main principle is to find a most similar block in the coded image for the current image block. This process is called motion compensation. For example, for a frame of image, it is first divided into coding areas of equal size (Coding Tree Unit, CTU), for example, the size is 64×64 or 128×128. Each CTU can be further divided into square or rectangular coding units (Coding Unit, CU). Each CU searches for the most similar block in the reference frame (generally a reconstructed frame near the current frame in the time domain) as the prediction block of the current CU. The relative displacement between the current block (i.e., the current CU) and the similar block (i.e., the prediction block of the current CU) is called a motion vector (Motion Vector, MV). The process of finding the most similar block in the reference frame as the prediction block of the current block is motion compensation.
[0007] In current technology, the motion vector candidate list of the current CU is usually constructed according to two methods. The motion vector candidate list is also called the merge candidate list. The motion vector candidate list includes candidate motion vectors in the spatial domain, which usually fills the motion vector (or motion information) of the encoded neighboring blocks of the current CU into the motion vector candidate list. The motion vector candidate list also includes candidate motion vectors in the temporal domain. The temporal motion vector prediction (Temproal Motion Vector Prediction, TMVP) uses the motion vector (or motion information) of the corresponding CU (i.e., the co-located CU) of the current CU in the adjacent encoded image. The best candidate motion vector is selected from the merge candidate list as the motion vector of the current CU; the prediction block of the current CU is determined based on the motion vector of the current CU.
[0008] Advanced / Alternative temporal motion vector prediction (ATMVP) is a motion vector prediction mechanism. The basic idea of ATMVP is to perform motion compensation by obtaining motion information of multiple sub-blocks in the current CU. ATMVP introduces motion information of multiple sub-blocks in the current CU as candidates in constructing a candidate list (such as a merge candidate list or an AMVP (Advanced Motion Vector Prediction) candidate list). The implementation of ATMVP can be roughly divided into two steps. The first step is to determine a temporal vector by scanning the candidate motion vector list of the current CU or the motion vector of the adjacent image block of the current CU; the second step is to divide the current CU into N×N (N defaults to 4) sub-blocks (sub-CU), determine the corresponding block of each sub-block in the reference frame according to the temporal vector obtained in the first step, and determine the motion vector of each sub-block according to the motion vector of the corresponding block of each sub-block in the reference frame.
[0009] In the first step of the current ATMVP technology, there is still room for improvement in the process of determining a time domain vector by scanning the candidate motion vector list of the current CU or the motion vector of the adjacent image block of the current CU. The second step of the current ATMVP technology sets the size of the sub-CU adaptively at the frame level, and the default size is 4×4. When certain preset conditions are met, the size of the sub-CU will be set to 8×8. There are some problems in the sub-CU size setting that do not match the current motion information storage granularity (8×8). There are redundant operations in some cases between the ATMVP technology and the TMVP technology, and there is room for improvement in the process of constructing the candidate motion vector list. Summary of the invention
[0010] The present application provides a video image processing method and device, which can reduce the complexity of ATMVP technology while maintaining the performance gain of existing ATMVP technology.
[0011] In a first aspect, a video image processing method is provided, the method comprising:
[0012] Scanning N neighboring blocks of the preset M neighboring blocks of the current image block in sequence, and determining a target neighboring block according to the scanning result, where N is less than M;
[0013] Determining a related block of the current image block according to a motion vector of the target neighboring block, the current image block, and a reference image of the current image block;
[0014] Dividing the current image block and the related block into a plurality of sub-image blocks in the same manner, wherein each sub-image block in the current image block corresponds to each sub-image block in the related block in a one-to-one manner;
[0015] The corresponding sub-image blocks in the current image block are predicted respectively according to the motion vectors of the sub-image blocks in the related blocks.
[0016] In the solution provided by the present application, in the process of obtaining the reference motion vector of the current image block, only N (N is less than M) candidate motion vectors among the M candidate motion vectors that have been obtained are scanned in sequence, which can reduce the number of scans of the candidate motion vectors in the process of obtaining the reference motion vector of the current image block compared to the prior art. It should be understood that applying the solution provided by the present application to the first step of the existing ATMVP technology can simplify the redundant operations existing therein.
[0017] In a second aspect, a video image processing method is provided, the method comprising:
[0018] Determine M neighboring blocks of the current image block according to the M candidates in the second candidate list of the motion vector of the current image block;
[0019] Scanning N neighboring blocks among the M neighboring blocks in sequence, and determining a target neighboring block according to the scanning result, where N is less than M;
[0020] Determining a related block of the current image block according to a motion vector of the target neighboring block, the current image block, and a reference image of the current image block;
[0021] Determine a specific candidate in a first candidate list of motion vectors of the current image block according to a related block of the current image block;
[0022] When it is determined to adopt the specific candidate, the current image block and the related block are divided into a plurality of sub-image blocks in the same manner, and each sub-image block in the current image block corresponds to each sub-image block in the related block one by one;
[0023] The corresponding sub-image blocks in the current image block are predicted respectively according to the motion vectors of the sub-image blocks in the related blocks.
[0024] According to a third aspect, a video image processing device is provided, the device comprising:
[0025] A construction module is used to sequentially scan N neighboring blocks among the preset M neighboring blocks of the current image block, determine a target neighboring block according to the scanning result, where N is less than M; determine a related block of the current image block according to a motion vector of the target neighboring block, the current image block, and a reference image of the current image block; and divide the current image block and the related block into a plurality of sub-image blocks in the same manner, where each sub-image block in the current image block corresponds to each sub-image block in the related block in a one-to-one manner;
[0026] The prediction module is used to predict the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.
[0027] In a fourth aspect, a video image processing device is provided, the device comprising:
[0028] A construction module is used to determine M neighboring blocks of the current image block according to M candidates in a second candidate list of motion vectors of the current image block; sequentially scan N neighboring blocks among the M neighboring blocks, and determine a target neighboring block according to the scanning result, where N is less than M; determine a related block of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block; determine a specific candidate in a first candidate list of motion vectors of the current image block according to the related blocks of the current image block; when it is determined to adopt the specific candidate, divide the current image block and the related block into a plurality of sub-image blocks in the same manner, and each sub-image block in the current image block corresponds one-to-one to each sub-image block in the related block;
[0029] The prediction module is used to predict the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.
[0030] In a fifth aspect, a video image processing device is provided, which includes a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to execute the method in the first aspect or any possible implementation of the first aspect.
[0031] In a sixth aspect, a video image processing device is provided, which includes a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to execute the method in the second aspect or any possible implementation of the second aspect.
[0032] According to a seventh aspect, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer implements the method in the first aspect or any possible implementation manner of the first aspect.
[0033] According to an eighth aspect, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer implements the method in the second aspect or any possible implementation manner of the second aspect.
[0034] According to a ninth aspect, a computer program product comprising instructions is provided, wherein when the instructions are executed by a computer, the computer implements the method according to the first aspect or any possible implementation manner of the first aspect.
[0035] In a tenth aspect, a computer program product comprising instructions is provided, wherein when the instructions are executed by a computer, the computer implements the method in the second aspect or any possible implementation manner of the second aspect.
[0036] In an eleventh aspect, a video image processing method is provided, the method comprising:
[0037] Determine a base motion vector list, wherein the base motion vector list includes at least one bi-prediction base motion vector group, wherein the bi-prediction base motion vector group includes a first base motion vector and a second base motion vector;
[0038] Determine two motion vector offsets from a preset offset set, wherein the two motion vector offsets correspond to the first base motion vector and the second base motion vector respectively;
[0039] Determine a motion vector of a current image block according to the first basic motion vector, the second basic motion vector and the two motion vector offsets;
[0040] The current image block is predicted according to the motion vector of the current image block.
[0041] In a twelfth aspect, a video image processing method is provided, the method comprising:
[0042] Determine a basic motion vector list, wherein the basic motion vector list includes a basic motion vector group;
[0043] When at least one basic motion vector in the basic motion vector group points to a specific reference image, determining the motion vector of the current image block according to the basic motion vector group and the motion vector offset is abandoned.
[0044] In a thirteenth aspect, a video image processing device is provided, the device comprising:
[0045] A construction module is configured to determine a basic motion vector list, wherein the basic motion vector list includes at least one bi-prediction basic motion vector group, wherein the bi-prediction basic motion vector group includes a first basic motion vector and a second basic motion vector; determine two motion vector offsets from a preset offset set, wherein the two motion vector offsets correspond to the first basic motion vector and the second basic motion vector, respectively; and determine a motion vector of a current image block according to the first basic motion vector, the second basic motion vector, and the two motion vector offsets;
[0046] A prediction module is used to predict the current image block according to the motion vector of the current image block.
[0047] In a fourteenth aspect, a video image processing device is provided, the device comprising:
[0048] A determination module, configured to determine a basic motion vector list, wherein the basic motion vector list includes a basic motion vector group;
[0049] The processing module is configured to abandon determining the motion vector of the current image block according to the basic motion vector group and the motion vector offset when at least one basic motion vector in the basic motion vector group points to a specific reference image.
[0050] In the fifteenth aspect, a video image processing device is provided, which includes a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to execute the method in the eleventh aspect or any possible implementation of the eleventh aspect.
[0051] In the sixteenth aspect, a video image processing device is provided, which includes a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to execute the method in the twelfth aspect or any possible implementation of the twelfth aspect.
[0052] In the seventeenth aspect, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer implements the method in the eleventh aspect or any possible implementation of the eleventh aspect.
[0053] In the eighteenth aspect, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer implements the method in the twelfth aspect or any possible implementation of the twelfth aspect.
[0054] In the nineteenth aspect, a computer program product comprising instructions is provided, wherein when the instructions are executed by a computer, the computer implements the method in the eleventh aspect or any possible implementation manner of the eleventh aspect.
[0055] In the twentieth aspect, a computer program product comprising instructions is provided, wherein when the instructions are executed by a computer, the computer implements the method in the twelfth aspect or any possible implementation manner of the twelfth aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 It is a schematic flow chart of the video image processing method provided in an embodiment of the present application.
[0057] Figure 2 It is a schematic diagram of obtaining the candidate motion vector of the current block through the neighboring blocks of the current image block.
[0058] Figure 3 It is a schematic diagram of scaling the candidate motion vectors.
[0059] Figure 4 It is another schematic flow chart of the video image processing method provided in an embodiment of the present application.
[0060] Figure 5 It is a schematic block diagram of a video image processing device provided in an embodiment of the present application.
[0061] Figure 6 It is another schematic block diagram of the video image processing device provided in an embodiment of the present application.
[0062] Figure 7 It is a schematic flow chart of the video image processing method provided in an embodiment of the present application.
[0063] Figure 8 It is another schematic flow chart of the video image processing method provided in an embodiment of the present application.
[0064] Fig. 9 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.
[0065] Fig.10 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.
[0066] Fig.11 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.
[0067] Fig.12 It is a schematic diagram of obtaining candidates for the first candidate list of motion vectors.
[0068] Fig.13 This is another schematic diagram of constructing candidates for the first candidate list of motion vectors.
[0069] Fig.14 and Fig.15 It is a schematic flow chart of the video image processing method provided in an embodiment of the present application.
[0070] Fig.16 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.
[0071] Fig.17 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.
[0072] Fig.18 It is another schematic flow chart of the video image processing method provided in an embodiment of the present application.
[0073] Fig.19 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application.
[0074] Fig. 20 This is another schematic block diagram of the video image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0075] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0076] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.
[0077] In video coding and decoding, the prediction step is used to reduce redundant information in an image. A prediction block refers to a basic unit used for prediction in a frame of an image. In some standards, the prediction block is also called a prediction unit (PU). Before encoding / compressing a frame of an image, the image is divided into multiple image blocks. Further, each of the multiple image blocks can be divided into multiple image blocks again, and so on. In different coding methods, the number of levels of segmentation can be different, and the operation methods undertaken are also different. In different coding standards, the names of image blocks on the same level may be different. For example, in some video standards, each of the multiple image blocks into which a frame of an image is first divided is called a coding tree unit (CTU); each coding tree unit can contain a coding unit (CU) or be divided into multiple coding units again; a coding unit can be divided into one, two, four or other number of prediction units according to the prediction method. In some video standards, the coding tree unit is also called the largest coding unit (LCU).
[0078] Prediction refers to finding image data similar to the prediction block, also called the reference block of the prediction block. The difference between the prediction block and the reference block of the prediction block is encoded / compressed to reduce redundant information in encoding / compression. The difference between the prediction block and the reference block can be the residual obtained by subtracting the corresponding pixel values of the prediction block from those of the reference block. Prediction includes intra-frame prediction and inter-frame prediction. Intra-frame prediction refers to finding the reference block of the prediction block within the frame where the prediction block is located, and inter-frame prediction refers to finding the reference block of the prediction block in frames other than the frame where the prediction block is located.
[0079] In some existing video standards, a prediction unit is the smallest unit in an image, and a prediction unit will not be further divided into multiple image blocks. The "image block" or "current image block" mentioned below refers to a prediction unit (or a coding unit), and an image block can be further divided into multiple sub-image blocks, each of which can be further predicted.
[0080] In this solution, before predicting the current image block, a motion vector candidate list is constructed, and the current image block is predicted based on the candidate motion vector selected in the motion vector candidate list. The motion vector candidate list has multiple types of modes. The following first illustrates the multiple types of modes of the motion vector candidate list.
[0081] In the first mode, as a first example, at the encoding end, after the motion vector candidate list is constructed, the encoding of the current image block can be completed through the following steps.
[0082] 1) Select the best motion vector (denoted as MV1) from the motion vector candidate list, use the selected MV1 as the motion vector of the current image block, and obtain the index of the MV1 in the motion vector candidate list.
[0083] 2) According to the motion vector MV1 of the current image block, the predicted image block of the current image block is determined from the reference image (ie, the reference frame), that is, the position of the predicted image block of the current image block in the reference frame is determined.
[0084] 3) Obtain the residual between the current image block and the predicted image block.
[0085] 4) Sending the index of the motion vector MV1 of the current image block in the motion vector candidate list and the residual obtained in step 3) to the decoding end.
[0086] As an example, at the decoding end, the current image block may be decoded through the following steps.
[0087] 1) Receive the residual and the index of the motion vector of the current image block in the motion vector candidate list from the encoder.
[0088] 2) Obtaining a motion vector candidate list by the method according to the embodiment of the present application. The motion vector candidate list obtained by the decoding end is consistent with the motion vector candidate list obtained by the encoding end.
[0089] 3) According to the index, obtain the motion vector MV1 of the current image block from the motion vector candidate list.
[0090] 4) According to the motion vector MV1, the predicted image block of the current image block is obtained, and then combined with the residual, the current image block is decoded.
[0091] That is, in the first type of mode, the motion vector of the current image block is equal to the predicted MV (Motion vector prediction, MVP). In some standards, the first type of mode is also called Merge mode.
[0092] In the second mode, unlike the first mode, after the encoder selects the best motion vector MV1 from the motion vector candidate list, it also uses the MV1 as the search starting point for motion search, and records the displacement between the final searched position and the search starting point as the motion vector difference (MVD). Then, based on the motion vector MV1+MVD of the current image block, the predicted image block of the current image block is determined from the reference image. The encoder also sends the MVD to the decoder. In some standards, the second mode is also called AMVP mode (i.e., normal inter-frame prediction mode).
[0093] The motion vector candidate lists in different types of modes can be constructed in the same or different ways. The motion vector candidate lists constructed in the same way can be applied to only one type of mode or different types of construction modes, which is not limited here.
[0094] In this solution, two construction methods of motion vector candidate lists are provided. For the convenience of description, the motion vector candidate lists of the two construction methods are referred to as the first motion vector candidate list and the second motion vector candidate list hereinafter. One difference between the two lists is that at least one candidate in the first motion vector candidate list includes the motion vector of the sub-image block, and each candidate in the second motion vector candidate list includes the motion vector of the image block. As mentioned above, the image block and the current image block here are the same type of concepts, both referring to a prediction unit (or a coding unit), and the sub-image block refers to a plurality of sub-image blocks obtained by segmenting the image block. When using the candidate in the first motion vector candidate list for prediction, the reference block of the current image block is determined according to the candidate, and then the residual between the image block and the reference block is calculated. When using the candidate in the second motion vector candidate list for prediction, if the candidate used is the motion vector of the sub-image block, the reference block of each sub-image block in the current image block is determined according to the candidate, and then the residual between each sub-image block in the current image block and its reference block is calculated, and the residual of each sub-image block is spliced into the residual of the current image block.
[0095] Among them, when determining the candidates in the first candidate list of motion vectors and / or the second candidate list of motion vectors, one of the candidates can be determined according to the ATMVP technology. In one example, when constructing the first candidate list of motion vectors, the motion vector determined according to the ATMVP technology can be added to the list as the first candidate. In one example, when constructing the second candidate list of motion vectors, after adding the motion vectors of a preset number of spatial neighboring blocks at a preset position of the current image block to the second candidate list of motion vectors, the motion vector determined according to the ATMVP technology can be added to the list as a candidate. Of course, the order in which the candidates of the two candidate lists are added can be other orders, and there is no limitation on this.
[0096] The following first uses an example of how to determine one of the candidates based on the ATMVP technology in combination with the construction and use of the second candidate list of motion vectors.
[0097] In describing the construction method of the second candidate list of motion vectors, the motion vector is explained here for ease of understanding. The motion vector of an image block can contain two pieces of information: 1) the image pointed to by the motion vector; 2) the displacement. The meaning represented by the motion vector of an image block is the image block that has the displacement with the image block in the image pointed to by the motion vector. For an encoded / decoded image block, the meaning contained in its motion vector includes: the reference image of the encoded / decoded image block, and the displacement of the reference block of the encoded / decoded image block relative to the encoded / decoded image block. It should be noted that the reference block of an image block mentioned in this article refers to the image block representing the residual used to calculate the image block.
[0098] Figure 1 A schematic flow chart of a video image processing method provided in an embodiment of the present application. The method comprises the following steps.
[0099] S110 , determining M candidate motion vectors to be added to a second candidate list of motion vectors for a current image block.
[0100] The current image block is an image block to be encoded (or decoded). The image frame where the current image block is located is called the current frame. For example, the current image block is a coding unit (CU).
[0101] For example, the second motion vector candidate list of the current image block may be a Merge candidate list or an AMVP candidate list. For example, the second motion vector candidate list may be a normal motion vector candidate list (Normal Merge List) in the Merge candidate list. It should be understood that the second motion vector candidate list may also have other names.
[0102] The M candidate motion vectors may be determined based on the motion vectors of the M neighboring blocks of the current image block in the current frame. The neighboring blocks may be image blocks that are adjacent to or have a certain position spacing with the current image block in the current frame. It should be understood that the M neighboring blocks are image blocks that have been encoded (or decoded) in the current frame.
[0103] As an example, Figure 2 As shown, the M neighboring blocks of the current image block are Figure 2 The image blocks shown in 4 positions A1 (left) → B1 (upper) → B0 (upper right) → A0 (lower left) around the current image block are shown in . According to the motion vectors of the image blocks at these 4 positions, M (ie, M is equal to 4) candidate motion vectors of the current image block are determined.
[0104] In addition, when an unavailable neighboring block appears in the M neighboring blocks, or a neighboring block using intra-frame coding mode appears in the M neighboring blocks, the motion vector of the unavailable neighboring block or the neighboring block using intra-frame coding mode is unavailable. Then, the motion vector of the unavailable neighboring block is not used as a candidate motion vector, and the unavailable motion vector is abandoned to be added to the second candidate list of motion vectors of the current image block.
[0105] As a possible implementation, after completing step S110, the M candidate motion vectors have been added to the second candidate list of motion vectors. In step S120, the second candidate list of motion vectors may be directly scanned.
[0106] S120, sequentially scan N candidate motion vectors among the M candidate motion vectors, and determine a reference motion vector according to the scanning result, where N is less than M. Wherein, M and N are both natural numbers.
[0107] Regardless of whether all the M candidate motion vectors are added to the second candidate motion vector list, or some of the M candidate motion vectors are unavailable, resulting in only some of the M candidate motion vectors being added to the second candidate motion vector list, the N candidate motion vectors among the M candidate motion vectors are fixedly scanned in sequence. Fixedly scanning the N candidate motion vectors among the M candidate motion vectors in sequence may refer to fixedly scanning the candidate motion vectors among the N candidate motion vectors that have been added to the candidate motion vector list; or, may refer to fixedly scanning the N candidate motion vectors among the M candidate motion vectors that have been added to the candidate motion vector list.
[0108] The process of determining the reference motion vector according to the scanning results of the N candidate motion vectors may be to judge the N candidate motion vectors in sequence based on preset conditions, and determine the reference motion vector according to the judgment results.
[0109] As an example, the preset conditions include: the image block can obtain or not adopt the intra-frame prediction coding mode, and the reference frame pointed to by the candidate motion vector is the same as the reference frame of the current image block.
[0110] Among them, the reference image of the current image block is the reference image that is closest to the image where the current image block is located in time; or, the reference image of the current image block is the reference image preset by the codec end; or, the reference image of the current image block is the reference image specified in the video parameter set, sequence header, sequence parameter set, image header, image parameter set, and slice header.
[0111] For example, the reference image of the current image block is the collocated frame of the current image block, which is a frame set in the slice-level information header for obtaining motion information for prediction. In some application scenarios, the collocated frame is also called a collocated picture.
[0112] It should be understood that, depending on the evolution of future technology, the preset condition may be given other different definitions, and the corresponding solutions also fall within the scope of protection of this application.
[0113] The process of determining the reference motion vector according to the scanning results of N candidate motion vectors will be described in detail below.
[0114] In step S120, only N motion vectors among the M candidate motion vectors obtained in step S110 are scanned, which can reduce the number of scans.
[0115] Optionally, in step S120, the first N candidate motion vectors among the M candidate motion vectors may be scanned in sequence.
[0116] Optionally, in step S120, the last N candidate motion vectors among the M candidate motion vectors may be scanned in sequence; or, the middle N candidate motion vectors among the M candidate motion vectors may be scanned in sequence. This application does not limit this.
[0117] As an example, in step S120, some of the M candidate motion vectors are scanned in sequence.
[0118] As another example, in step S120, part of the candidate motion vectors currently added to the second candidate list of motion vectors are scanned in sequence.
[0119] S130: Determine candidate motion vectors to be added to the second candidate list of motion vectors according to the reference motion vector, the current image block, and the reference image of the current image block.
[0120] The second candidate list of motion vectors of the current image block includes the M candidate motion vectors determined in step S110 and the candidate motion vector determined in step S130. In one example, after determining to continue adding the M+1th candidate motion vector to the second candidate list of motion vectors according to S130, other candidate motion vectors to continue adding to the second candidate list of motion vectors are determined according to other methods, which are not limited here.
[0121] After constructing the second candidate list of motion vectors, Figure 1 As shown, the method may further include: S140, determining the motion vector of the current image block according to the second candidate list of motion vectors obtained in step S130.
[0122] It should be understood that the solution provided in the present application can be applied to the ATMVP technology. In the first step of the existing ATMVP technology, the time domain vector of the current image block is obtained by scanning all the spatial domain candidate motion vectors currently added to the second candidate list of motion vectors. For example, the second candidate list of motion vectors is usually filled with 4 spatial domain candidate motion vectors, and the following situation may occur: 4 candidate motion vectors need to be scanned to obtain the time domain vector of the current image block.
[0123] In the embodiment of the present application, in the process of obtaining the reference motion vector of the current image block, only N (N is less than M) candidate motion vectors among the M candidate motion vectors that have been obtained are scanned in sequence, which can reduce the number of scans of the candidate motion vectors in the process of obtaining the reference motion vector of the current image block compared to the prior art. It should be understood that applying the solution provided by the present application to the first step of the existing ATMVP technology can simplify the redundant operations existing therein.
[0124] The applicant selected the official test sequence as the test sequence on the latest reference software VTM-2.0 of Versatile Video Coding, and tested the solution provided in this application with RA configuration and LDB configuration. The test results showed that the performance gain of ATMVP technology can be maintained after reducing the number of scans.
[0125] Therefore, the solution provided in this application can reduce the complexity of ATMVP technology while maintaining the performance gain of existing ATMVP technology.
[0126] It should be understood that the second candidate list of motion vectors formed by the construction scheme provided in the present application can be applied to the encoding end and the decoding end. In other words, the execution subject of the method provided in the present application can be the encoding end or the decoding end.
[0127] As an example, the second candidate list of motion vectors formed by the construction scheme provided in the present application can be applied to the first type of mode (eg, Merge mode) mentioned above.
[0128] Optionally, in this embodiment, in step S110, four candidate motion vectors for adding to the second candidate list of motion vectors of the current image block are determined according to the motion vectors of the four neighboring blocks of the current image block in the current frame, that is, M is equal to 4. In step S120, N candidate motion vectors among the four candidate motion vectors are scanned, and N is less than 4.
[0129] For example, N is equal to 1. For example, in step S120, only the first candidate motion vector in the second candidate list of motion vectors is scanned. For another example, N is equal to 2 or 3.
[0130] The following will describe the manner of determining the reference motion vector of the current image block according to the scanning results of the N candidate motion vectors in step S120.
[0131] In step S120, it is determined whether N candidate motion vectors among the M candidate motion vectors meet the preset condition one by one, and a reference motion vector is determined according to the determination result. In this paper, the preset condition is defined as the reference frame pointed to by the candidate motion vector is the same as the reference image of the current image block.
[0132] Optionally, in step S120, N candidate motion vectors are scanned in sequence. When the first candidate motion vector that meets the preset conditions is scanned, that is, when the first reference frame is scanned and the candidate motion vector that is the same as the co-located frame of the current frame is scanned, the scanning is stopped, and the reference motion vector is determined based on the scanned first candidate motion vector that meets the preset conditions.
[0133] It should be understood that when the first candidate motion vector meeting the preset condition is scanned, the number of scans may be equal to N or may be less than N. For example, when the first scanned candidate motion vector meets the preset condition, the scanning is stopped and the candidate motion vector is used as the reference motion vector of the current image block.
[0134] Optionally, in step S120, when no candidate motion vector meeting the preset condition is scanned among the N candidate motion vectors, that is, when the reference frames pointed to by the N candidate motion vectors are all different from the co-located frames of the current image block, a default value is used as the value of the reference motion vector.
[0135] For example, the default value is (0, 0), that is, the reference motion vector is (0, 0). It should be understood that the default value may have other definitions according to actual conditions.
[0136] Optionally, in step S120, when no candidate motion vector meeting the preset conditions is scanned among the N candidate motion vectors, that is, when the reference frames pointed to by the N candidate motion vectors are all different from the co-located frames of the current image block, the specific candidate motion vector in the second candidate list of motion vectors is scaled, and the reference motion vector is determined based on the specific candidate motion vector after the scaling process.
[0137] The specific candidate motion vector may be the first motion vector or the last motion vector obtained in a scanning order among the N candidate motion vectors.
[0138] The specific candidate motion vector may also be a motion vector obtained in other scanning orders among the N candidate motion vectors.
[0139] When the preset condition is defined as that the reference frame pointed to by the candidate motion vector is the same as the reference frame of the current image block, the specific candidate motion vector in the second candidate list of motion vectors is scaled, and the reference motion vector is determined based on the scaled specific candidate motion vector, including: scaling the specific candidate motion vector in the second candidate list of motion vectors so that the reference frame pointed to by the scaled specific candidate motion vector is the same as the reference image of the current image block; and using the scaled specific candidate motion vector as the reference motion vector.
[0140] like Figure 3 As shown, curr_pjc represents the image where the current image block is located, col_pic represents the collocated picture of the current image block, and neigh_ref_pic represents the reference frame pointed to by the specific candidate motion vector. In one implementation, the scaling ratio of the specific motion vector is determined based on the time distance between the reference image neigh_ref_pic pointed to by the specific candidate motion vector and the image curr_pic where the image block corresponding to the specific motion vector is located, and the time distance between the reference image col_pic of the current image block and the image curr_pic where the current image block is located.
[0141] It should be understood that the difference in the degree of motion between image frames is small. In the case of intense motion between the current frame and its co-located frame, if the motion vector (0, 0) is used as the basis for locating the corresponding block of the current block, the motion between frames is not taken into account. Instead, it is directly assumed that the absolute coordinates of the current block in the co-located frame have not changed. In fact, there is a high probability that the coordinates of the current block in the co-located frame are different from those in the current frame, and therefore a large deviation will occur.
[0142] In the embodiment of the present application, when no candidate motion vector whose reference frame is the same as the collocated frame of the current frame is scanned among the N candidate motion vectors, one of the N candidate motion vectors is scaled so that its reference frame is the same as the collocated frame of the current frame, and then the scaled candidate motion vector is used as the motion vector of the current image block. In this way, the accuracy of the motion vector of the current image block can be improved.
[0143] Optionally, when N is an integer less than M and greater than 1, the specific candidate motion vector in this embodiment may be a candidate motion vector whose reference frame and the co-located frame of the current image block are closest in time domain among the N candidate motion vectors.
[0144] It should be understood that selecting a candidate motion vector whose reference frame is closest to the co-located frame of the current frame among the N candidate motion vectors for scaling processing can reduce the time consumption of scaling processing, thereby improving the efficiency of obtaining the motion vector of the current image block.
[0145] Optionally, when N is an integer less than M and greater than 1, the specific candidate motion vector in this embodiment may also be any candidate motion vector among the N candidate motion vectors.
[0146] It should be understood that when N is equal to 1, the specific candidate motion vector in this embodiment is the scanned candidate motion vector.
[0147] Optionally, as an embodiment, N is equal to 1, and in step S120, a reference motion vector of the current image block is obtained by scanning a candidate motion vector in the second candidate list of the motion vector. When the reference frame pointed to by the scanned candidate motion vector is different from the collocated frame of the current frame where the current image block is located, the candidate motion vector is scaled so that the reference frame of the candidate motion vector after the scaling process is the same as the collocated frame of the current frame; the candidate motion vector after the scaling process is used as the reference motion vector of the current image block. When the reference frame of the scanned candidate motion vector is the same as the collocated frame of the current frame, the candidate motion vector is used as the motion vector of the current image block.
[0148] In this embodiment, by scanning a candidate motion vector in the candidate motion vector list to obtain the motion vector of the current image block, the number of times the candidate motion vector is scanned in the process of obtaining the motion vector of the current image block is effectively reduced; when the reference frame of the scanned candidate motion vector is different from the collocated frame of the current frame, the candidate motion vector is scaled so that its reference frame is the same as the collocated frame of the current frame, and then the scaled candidate motion vector is used as the motion vector of the current image block, which can improve the accuracy of the motion vector of the current image block. Therefore, compared with the prior art, the solution provided by the embodiment of the present application can both simplify the process of determining the motion vector of the current image block and improve the accuracy of the motion vector of the current image block.
[0149] It should be understood that when the definition of the preset condition changes, the process of scaling the specific candidate motion vector in the second candidate list of motion vectors must also change accordingly, that is, to ensure that the specific candidate motion vector after scaling meets the preset condition.
[0150] The following describes the process of determining the candidate motion vectors to be added to the second candidate list of motion vectors according to the reference motion vector, the current image block and the reference image of the current image block in step S130.
[0151] Optionally, as an implementation method, determining candidate motion vectors to be continued to be added to the second candidate list of motion vectors based on a reference motion vector, a current image block, and a reference image of the current image block includes: dividing the current image block into multiple sub-image blocks; determining relevant blocks of the sub-image blocks in the reference image of the current image block based on the reference motion vector; and determining candidate motion vectors to be continued to be added to the second candidate list of motion vectors based on the motion vectors of the relevant blocks.
[0152] In one example, the motion vector of the associated block of each sub-image block in the current image block is added as a candidate to the second motion vector candidate list. When the candidate is used for prediction, the sub-image block is predicted according to the motion vector of the associated block of each sub-image block in the current image block.
[0153] In one example, the representative motion vector of the relevant block of the current image block is added as a candidate to the second candidate list of motion vectors, and the candidate is marked as determined according to the ATMVP technology. When the candidate is used for prediction, the relevant block of the current image block is determined according to the mark and the candidate, and the current image block and the relevant block are divided into multiple sub-image blocks in the same way, and each sub-image block in the current image block corresponds to each sub-image block in the relevant block one by one; the corresponding sub-image blocks in the current image block are predicted according to the motion vectors of each sub-image block in the relevant block. Optionally, when a sub-image block with an unavailable motion vector appears in the relevant block, the representative motion vector of the relevant block is used to replace the unavailable motion vector, and the corresponding sub-image block in the current image block is predicted. Optionally, when a sub-image block with an unavailable motion vector appears in the relevant block, and the representative motion vectors of the relevant block are all unavailable, the candidate determined according to the ATMVP technology is abandoned from being added to the second candidate list of motion vectors. In one example, when a sub-image block in a related block is unavailable, or a sub-image block in a related block adopts an intra-frame coding mode, it is determined that a sub-image block with an unavailable motion vector appears in the related block.
[0154] Optionally, the representative motion vector of the relevant block of the current image block may refer to a motion vector of a center position of the relevant block, or other motion vectors representing the relevant block, which is not limited here.
[0155] In some video coding standards, the related block may be referred to as a collocated block or a corresponding block.
[0156] For example, the current image block is a CU, and the sub-image blocks obtained after dividing it can be called sub-CU. Optionally, the size of the sub-image block and / or the size of the related block of the sub-image block is fixed to be greater than or equal to 64 pixels. Optionally, the size of the sub-image block and / or the size of the related block of the sub-image block are both fixed to 8×8 pixels.
[0157] In the current ATMVP technology, the size of the sub-image block is set adaptively at the frame level. The size of the sub-image block defaults to 4×4. When certain conditions are met, the size of the sub-image block is set to 8×8. For example, at the encoding end, when encoding the current image block, the average block size of each sub-image block in the CU when the previous coded image block of the same time domain layer is coded in ATMVP mode is calculated. When the average block size is greater than the threshold, the size of the sub-image block of the current image block is set to 8×8, otherwise the default value of 4×4 is used. At present, in the new generation of video coding standards (Versatile Video Coding, VVC), motion vectors are stored in a size of 8×8. It should be understood that when the size of the sub-image block is set to 4×4, the size of the motion vector of the sub-image block (also 4×4) does not meet the storage granularity of the motion vector in the current standard. In addition, in the current ATMVP technology, when encoding the current image block, it is also necessary to store information on the size of the sub-image block of the previous coded image block of the same time domain layer.
[0158] In an embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8. On the one hand, it can adapt to the storage granularity of the motion vector specified in the video standard VVC. On the other hand, there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.
[0159] At present, the TMVP technology scales the MV of the co-located CU in the adjacent encoded image at the lower right corner or center of the current CU to obtain the time domain candidate motion vector of the current CU. In other words, TMVP obtains the MVP by traversing the coding blocks at two fixed positions in the reference image, and the order is TB→TC, and the first traversed MV is directly used as the MVP of TMVP. In an embodiment of the present application, the sub-image block size in the ATMVP technology is set to 8×8, and it will use an MV in the existing merge candidate list (merge list) of each embodiment of the present application to locate the relevant block, and the MV of the located relevant block is used as the MVP of ATMVP.
[0160] In some versions of the video standard VVC, the merge list is constructed by first constructing the merge candidate list of ATMVP and then constructing the merge candidate list of TMVP. In some other versions of the video standard VVC, the merge candidate list of TMVP is constructed during the merge list construction process, and the merge candidate list of ATMVP is constructed during the affine merge list construction process. ATMVP and TMVP construct two different lists respectively, but logically there is no need to add the same or the same group of MVs to the two merge candidate lists. When the width and height of the current CU are both equal to 8, there may be some redundancy in the merge candidate lists constructed by TMVP and ATMVP respectively, that is, the two technologies may export the same set of time domain candidate motion information for the current CU.
[0161] In some embodiments of the present application, when the width and height of the current CU are both equal to 8, it is set not to perform the TMVP operation. In other words, when the size of the sub-image block and / or the size of the related block of the sub-image block is 8×8 pixels, it is set not to perform the TMVP operation. This can avoid including the same or the same group of MVs in the merge candidate lists constructed by ATMVP and TMVP respectively, and some redundant operations can be skipped, effectively saving encoding and decoding time and improving encoding efficiency.
[0162] In some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, it is set not to perform the TMVP operation. In other words, when at least one of the width and height of the sub-image block and / or the related blocks of the sub-image block is less than 8 pixels, it is set not to perform the TMVP operation. This is because: the hardware design of the encoder and / or decoder requires that the time to complete encoding or decoding of the processing area of the same size is consistent as much as possible. However, for an area containing more small blocks, the pipeline time required for encoding or decoding will be much longer than that of other areas. Further, it can be set not to perform the TMVP operation when the width and height of the sub-image block and / or the related blocks of the sub-image block are both less than 8 pixels. Therefore, saving the pipeline time for encoding or decoding of small blocks is very meaningful for parallel processing of hardware. In addition, the current coding technology has an increasingly high utilization rate of time domain correlation, and many time domain prediction technologies have been adopted, such as ATMVP technology. Therefore, for small blocks, the performance impact of skipping the TMVP operation can be ignored, which can effectively save encoding and decoding time and improve coding efficiency.
[0163] It should be understood that, under the premise of ensuring that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block can also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.
[0164] Optionally, as another implementation method, determining candidate motion vectors to be continued to be added to the second candidate list of motion vectors based on the reference motion vector, the current image block and the reference image of the current image block includes: determining relevant blocks of the current image block in the reference image of the current image block based on the reference motion vector; and determining candidate motion vectors to be continued to be added to the second candidate list of motion vectors based on the motion vectors of the relevant blocks.
[0165] In the encoding / decoding technology, an image that has been encoded / decoded is generally used as a reference image to be encoded / decoded currently. In some embodiments, a reference image may also be constructed to improve the similarity between the reference image and the image to be encoded / decoded currently.
[0166] For example, there is a specific encoding / decoding scene in the video content, in which the background basically does not change, and only the foreground in the video changes or moves. For example, video surveillance belongs to this type of scene. In the video surveillance scene, the surveillance camera is usually fixed or only moves slowly, and it can be considered that the background basically does not change. In contrast, objects such as people or cars captured in the video surveillance lens often move or change, and it can be considered that the foreground changes frequently. In this type of scene, a specific reference image can be created, and the specific reference image only contains high-quality background information. The specific reference image can include multiple image blocks, any of which is taken from a decoded image, and different image blocks in the specific reference image may be taken from different decoded images. When performing inter-frame prediction, the background part of the current image to be encoded / decoded can refer to the specific reference image, thereby reducing the residual information of the inter-frame prediction, thereby improving the encoding / decoding efficiency.
[0167] The above is a specific example of a specific reference image. In some implementations, the specific reference image has at least one of the following properties: a composite reference, a long-term reference image, and an image that is not output. Among them, the image that is not output refers to an image that is not output for display; generally speaking, the image that is not output exists as a reference image for other images. For example, the specific reference image can be a constructed long-term reference image, or can be a composite frame that is not output, or can be a long-term reference image that is not output, and so on. In some implementations, the composite frame is also called a composite reference frame.
[0168] In some implementations, the non-specific reference image may be a reference image that does not have at least one of the following properties: a construction frame, a long-term reference image, and an image that is not output. For example, the non-specific reference image may include a reference image other than the construction frame, or a reference image other than the long-term reference image, or a reference image other than the image that is not output, or a reference image other than the constructed long-term reference image, or a reference image other than the construction frame that is not output, or a reference image other than the long-term reference image that is not output, and the like.
[0169] In some implementations, when an image in a video can be used as a reference image, a long-term reference image and a short-term reference image can be distinguished. Among them, the short-term reference image is a concept corresponding to the long-term reference image. The short-term reference image exists in the reference image buffer for a period of time. After a decoded reference image after the short-term reference image is moved in and out of the reference image buffer, the short-term reference image will be moved out of the reference image buffer. The reference image buffer may also be called a reference image list cache, a reference image list, a reference frame list cache or a reference frame list, etc., which are collectively referred to as a reference image buffer in this article.
[0170] A long-term reference image (or part of the data in the long-term reference image) can always exist in the reference image buffer. The long-term reference image (or part of the data in the long-term reference image) is not affected by the movement of the decoded reference image in and out of the reference image buffer. The long-term reference image (or part of the data in the long-term reference image) will only be moved out of the reference image buffer when the decoding end issues an update instruction operation.
[0171] Short-term reference images and long-term reference images may be called differently in different standards. For example, in standards such as H.264 / Advanced Video Coding (AVC) or H.265 / HEVC, short-term reference images are called short-term reference frames, and long-term reference images are called long-term reference frames. For example, in audio video coding standard (AVS) 1-P2, AVS2-P2, Institute of Electrical and Electronics Engineers (IEEE) 1857.9-P4 and other standards, long-term reference images are called background pictures. For example, in standards such as VP8 and VP9, long-term reference images are called golden frames.
[0172] It should be understood that the use of specific terms in the embodiments of the present application does not necessarily mean that they must be applied to specific scenarios. For example, calling a long-term reference image a long-term reference frame does not mean that it must be used in technologies corresponding to standards such as H.264 / AVC or H.265 / HEVC.
[0173] The long-term reference image mentioned above can be constructed from image blocks taken from multiple decoded images, or can be obtained by updating an existing reference frame (e.g., a pre-stored reference frame) using multiple decoded images. Of course, the constructed specific reference image can also be a short-term reference image. Alternatively, the long-term reference image may not be a constructed reference image.
[0174] In the above implementation, the specific reference image may include a long-term reference image, and the non-specific reference image may include a short-term reference image.
[0175] Optionally, the type of reference frame can be identified by a special field in the codestream structure.
[0176] Optionally, when it is determined that the reference image is a long-term reference image, the reference image is determined to be a specific reference image; or, when it is determined that the reference image is a frame that is not output, the reference image is determined to be a specific reference image; or, when it is determined that the reference image is a construction frame, the reference image is determined to be a specific reference image; or, when it is determined that the reference image is a frame that is not output and further determined to be a construction frame, the reference image is determined to be a specific reference image.
[0177] Optionally, various types of reference images may have corresponding identifiers. In this case, the decoding end may determine whether the reference image is a specific reference image according to the identifier of the reference image.
[0178] In some implementations, when it is determined that the reference image has an identification of a long-term reference image, the reference image is determined to be a specific reference image.
[0179] In some implementations, when it is determined that the reference image has a flag that is not to be output, the reference image is determined to be a specific reference image.
[0180] In some implementations, when it is determined that the reference image has an identifier of a constructed frame, the reference image is determined to be a specific reference image.
[0181] In some implementations, when it is determined that the reference image has at least two of the following three identifiers, the reference image is determined to be a specific reference image: an identifier of a long-term reference image, an identifier that is not output, and an identifier of a construction frame or a synthetic reference frame. For example, when it is determined that the reference image has an identifier that is not output and it is determined that the reference image has an identifier of a construction frame, the reference image is determined to be a specific reference image.
[0182] Specifically, an image may have an identification indicating whether it is an output frame. When an image is indicated as not being output, it indicates that the frame is a reference image. Further, it is determined whether the frame has an identification of a constructed frame. If so, the reference image is determined to be a specific reference image. If an image is indicated to be output, it is not necessary to determine whether it is a constructed frame, and it is directly determined that the frame is not a specific reference image. Alternatively, if an image is indicated as not being output but has an identification that it is not a constructed frame, it can be determined that the frame is not a specific reference image.
[0183] Optionally, when it is determined that the reference image satisfies one of the following conditions by parsing parameters from a picture header, a picture parameter set (PPS), and a slice header, the reference image is determined to be a specific reference image:
[0184] The reference image is a long-term reference image;
[0185] The reference image is a construction reference image;
[0186] The reference image is an image that is not output;
[0187] When the reference image is an image that is not output, it is further determined that the reference image is a constructed reference image.
[0188] In some implementations of the embodiments of the present application, in the process of determining the motion vector of the current image block, it is involved to use the motion vector of a certain image block on other images to determine the motion vector of the image block. For the convenience of description, the image block is called the first image block, and the certain image block on the other image to be used is called the time domain reference block or related block of the first image block. It can be understood that the first image block and the time domain reference block (or related block) of the first image block are located on different images. Then, in the process of determining the motion vector of the first image block by using the motion vector of the time domain reference block (or related block), it may be necessary to scale the motion vector of the time domain reference block (or related block). For the convenience of description, the term "related block" is uniformly adopted in this article.
[0189] For example, when the ATMVP technology is applied in constructing the AMVP candidate list, when determining the motion vector of the related block of the current image block according to the ATMVP technology, it is necessary to scale the motion vector of the related block, and then determine the motion vector of the current image block according to the scaled motion vector. Generally speaking, the scaling ratio of the motion vector of the related block is determined based on the time distance between the reference image pointed to by the motion vector of the related block and the image where the related block is located, and the time distance between the reference image of the current image block and the image where the current image block is located.
[0190] In one example, the motion vector of the relevant block is called MV 2, and the reference frame index value of the reference image pointed to by the motion vector MV 2 is x. The reference frame index value x is the difference between the sequence number (for example, POC) of the reference image pointed to by MV 2 and the sequence number of the image where the relevant block is located. The reference frame index value of the reference image of the first image block is called y. The reference frame index value y is the difference between the sequence number of the reference image of the first image block and the sequence number of the image where the first image block is located. Then, the scaling ratio of the motion vector MV 2 is y / x. Optionally, the product of the motion vector MV 2 and y / x can be used as the motion vector of the first image block.
[0191] However, when the motion vector MV 2 of the relevant block points to a specific reference image, or when the reference image of the first image block is a specific reference image, since the temporal distance between the specific reference image and the image where the first image block is located is not clearly defined, scaling the motion vector MV 2 of the relevant block is meaningless.
[0192] Optionally, in this embodiment, when the motion vector of the current image block is determined according to the motion vector of the related block, specifically: when the motion vector of the related block points to a specific reference image, or the reference image of the current image block is a specific reference image, the motion vector of the current image block is determined according to the motion vector of the processed related block, wherein the motion vector of the processed related block is the same as the motion vector of the related block before processing.
[0193] For example, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0194] Optionally, in this embodiment, when the motion vector of the current image block is determined according to the motion vector of the related block, specifically: when the motion vector of the related block points to a specific reference image, or the reference image of the current image block is a specific reference image, the determination of the motion vector of the current image block according to the motion vector of the related block is abandoned.
[0195] In some embodiments, step S120 includes: if no candidate motion vector meeting the preset condition is scanned among the N candidate motion vectors, scaling the specific candidate motion vector in the second candidate list of motion vectors, and determining the reference motion vector according to the specific candidate motion vector after scaling. In this case, optionally, the method also includes: when the specific candidate motion vector points to a specific reference image, or the reference image of the current image block is a specific reference image, determining the candidate motion vector to be added to the second candidate list of motion vectors according to the processed specific candidate motion vector, wherein the processed specific candidate motion vector is the same as the specific candidate motion vector before processing.
[0196] The processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0197] In some embodiments, step S120 includes: if no candidate motion vector meeting the preset condition is scanned among the N candidate motion vectors, scaling the specific candidate motion vector in the second candidate list of motion vectors, and determining the reference motion vector according to the specific candidate motion vector after scaling. In this case, optionally, the method further includes: when the specific candidate motion vector points to a specific reference image, or the reference image of the current image block is a specific reference image, abandoning the determination of the candidate motion vector to be added to the second candidate list of motion vectors according to the specific candidate motion vector.
[0198] As can be seen from the above, in the embodiment of the present application, in the process of obtaining the reference motion vector of the current image block, only N (N is less than M) candidate motion vectors among the M candidate motion vectors that have been obtained are scanned in sequence, which can reduce the number of scans of the candidate motion vectors in the process of obtaining the reference motion vector of the current image block compared to the prior art. It should be understood that applying the solution provided by the present application to the existing ATMVP technology can simplify the redundant operations existing therein.
[0199] When no candidate motion vector whose reference frame is the same as the collocated frame of the current frame is scanned among the N candidate motion vectors, one of the N candidate motion vectors is scaled so that its reference frame is the same as the collocated frame of the current frame, and then the scaled candidate motion vector is used as the motion vector of the current image block, which can improve the accuracy of the motion vector of the current image block.
[0200] Dividing the current image block into sub-image blocks of 8×8 size can, on the one hand, adapt to the storage granularity of motion vectors specified in the video standard VVC, and on the other hand, there is no need to store information on the size of the sub-blocks of the previous encoded image block, thus saving storage space.
[0201] In some embodiments of the present application, when the size of a sub-image block and / or the size of a related block of the sub-image block is 8×8 pixels, setting not to perform a TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0202] In some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, the TMVP operation is set not to be performed. In other words, when at least one of the width and height of the sub-image block and / or the related block of the sub-image block is less than 8 pixels, the TMVP operation is set not to be performed. For the above situation, the performance impact caused by skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0203] like Figure 4 As shown, the embodiment of the present application also provides a video image processing method, which includes the following steps:
[0204] S410 , obtaining M candidate motion vectors for adding to a second candidate list of motion vectors of a current image block.
[0205] Step S410 corresponds to step S110 described above. Please refer to the above for the specific description, which will not be repeated here.
[0206] S420 , sequentially scan at least part of the M candidate motion vectors, and determine a reference motion vector of the current image block according to the scanning result.
[0207] As an optional implementation, some of the M candidate motion vectors are scanned in sequence, and the reference motion vector of the current image block is determined according to the scanning result. In this implementation, step S420 may correspond to step S120 described above, and the specific description is as above.
[0208] As another optional implementation manner, all candidate motion vectors among the M candidate motion vectors are scanned in sequence, and a reference motion vector of the current image block is determined according to the scanning result.
[0209] It should be understood that, in step S420, the specific method of determining the reference motion vector of the current image block according to the scanning result can refer to the relevant description in the above embodiment, which will not be repeated here.
[0210] S430, dividing the current image block into a plurality of sub-image blocks, wherein the size of the sub-image blocks is fixed to be greater than or equal to 64 pixels.
[0211] For example, the current image block is a CU, and the sub-image blocks obtained by dividing the current image block may be referred to as sub-CUs.
[0212] S440 , determining a related block of the sub-image block in the reference image of the current image block according to the reference motion vector.
[0213] The reference image of the current image block may be a co-located frame of the current image block.
[0214] S450: Determine candidate motion vectors to be added to the second candidate list of motion vectors according to the motion vector of the relevant block.
[0215] In the current ATMVP technology, the size of the sub-image block is set adaptively at the frame level. The size of the sub-image block defaults to 4×4. When certain conditions are met, the size of the sub-image block is set to 8×8. For example, at the encoding end, when encoding the current image block, the average block size of each sub-image block in the CU when the previous coded image block of the same time domain layer is encoded in ATMVP mode is calculated. When the average block size is greater than the threshold, the size of the sub-image block of the current image block is set to 8×8, otherwise the default value of 4×4 is used. In other words, in the prior art, when encoding the current image block, it is also necessary to store information about the size of the sub-image block of the previous coded image block of the same time domain layer.
[0216] In the embodiment of the present application, the size of the sub-image block of the current image block is fixed to be greater than or equal to 64 pixels, and there is no need to store information on the size of the sub-image block of the previous encoded image block, thereby saving storage space.
[0217] Optionally, in this embodiment, the size of the sub-image block and / or the size of the associated block of the sub-image block are both fixed to 8×8 pixels.
[0218] In the current ATMVP technology, the size of the sub-image block is set adaptively at the frame level. The size of the sub-image block defaults to 4×4. When certain conditions are met, the size of the sub-image block is set to 8×8. For example, at the encoding end, when encoding the current image block, the average block size of each sub-image block in the CU when the previous coded image block of the same time domain layer is coded in ATMVP mode is calculated. When the average block size is greater than the threshold, the size of the sub-image block of the current image block is set to 8×8, otherwise the default value of 4×4 is used. At present, in the new generation of video coding standards (Versatile Video Coding, VVC), motion vectors are stored in a size of 8×8. It should be understood that when the size of the sub-image block is set to 4×4, the size of the motion vector of the sub-image block (also 4×4) does not meet the storage granularity of the motion vector in the current standard. In addition, in the current ATMVP technology, when encoding the current image block, it is also necessary to store information on the size of the sub-image block of the previous coded image block of the same time domain layer.
[0219] In an embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8. On the one hand, it can adapt to the storage granularity of the motion vector specified in the video standard VVC. On the other hand, there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.
[0220] In some embodiments of the present application, when the size of a sub-image block and / or the size of a related block of the sub-image block is 8×8 pixels, setting not to perform a TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0221] In some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, the TMVP operation is set not to be performed. In other words, when at least one of the width and height of the sub-image block and / or the related block of the sub-image block is less than 8 pixels, the TMVP operation is set not to be performed. For the above situation, the performance impact caused by skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0222] It should be understood that, under the premise of ensuring that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block can also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.
[0223] Optionally, in step S420, at least part of the candidate motion vectors are scanned in sequence, and when the first candidate motion vector meeting the preset condition is scanned, the scanning is stopped, and a reference motion vector is determined based on the scanned first candidate motion vector meeting the preset condition.
[0224] Determining the reference motion vector according to the scanned first candidate motion vector that meets the preset condition may include: taking the first candidate motion vector that meets the preset condition as the target neighboring block.
[0225] Optionally, the preset condition includes: a reference image of the candidate motion vector is the same as a reference image of the current image block.
[0226] Optionally, step S450 includes: when the motion vector of the relevant block points to a specific reference image, or the reference image of the current image block is a specific reference image, determining the candidate motion vectors to be added to the second candidate list of motion vectors based on the motion vector of the processed relevant block, wherein the motion vector of the processed relevant block is the same as the motion vector of the relevant block before processing.
[0227] For example, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0228] Optionally, step S450 includes: when the motion vector of the relevant block points to a specific reference image, or the reference image of the current image block is a specific reference image, abandoning the candidate motion vector to be added to the second candidate list of motion vectors determined according to the motion vector of the relevant block.
[0229] therefore, Figure 4 In the embodiment shown, the current image block is divided into sub-image blocks of 8×8 in size. On the one hand, this can adapt to the storage granularity of motion vectors specified in the video standard VVC. On the other hand, there is no need to store information about the size of the sub-blocks of the previous encoded image block, thereby saving storage space.
[0230] In some embodiments of the present application, when the size of a sub-image block and / or the size of a related block of the sub-image block is 8×8 pixels, setting not to perform a TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0231] In some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, the TMVP operation is set not to be performed. In other words, when at least one of the width and height of the sub-image block and / or the related block of the sub-image block is less than 8 pixels, the TMVP operation is set not to be performed. For the above situation, the performance impact caused by skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0232] The above describes how to determine the candidate to be added to the second candidate list of motion vectors according to the ATMVP technology. In some implementations, other candidates may be added to the second candidate list of motion vectors, which is not limited here.
[0233] Combination of the above Figure 1 and Figure 4 The method embodiment of the present application is described, and the device embodiment corresponding to the above method embodiment will be described below. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the content not described in detail can be referred to the above method embodiment, and for the sake of brevity, it will not be repeated here.
[0234] Figure 5 Schematic block diagram of a video image processing device 500 provided in an embodiment of the present application. The device 500 is used to perform the following Figure 1 The method embodiment shown in the figure. The device 500 includes the following units.
[0235] An acquisition unit 510 is used to acquire M candidate motion vectors for adding to a second candidate list of motion vectors of a current image block;
[0236] A determination unit 520, configured to sequentially scan N candidate motion vectors among the M candidate motion vectors, and determine a reference motion vector according to the scanning result, where N is less than M;
[0237] The determination unit 520 is further configured to determine, according to the reference motion vector, the current image block, and the reference image of the current image block, a candidate motion vector to be added to the second candidate list of motion vectors;
[0238] The determination unit 520 is further configured to determine the motion vector of the current image block according to the second motion vector candidate list.
[0239] In the first step of the existing ATMVP technology, the time domain vector of the current image block is obtained by scanning all candidate motion vectors currently added to the second candidate list of motion vectors. For example, the second candidate list of motion vectors is usually filled with 4 candidate motion vectors, and the following situation may occur: 4 candidate motion vectors need to be scanned to obtain the time domain vector of the current image block.
[0240] In the embodiment of the present application, in the process of obtaining the reference motion vector of the current image block, only N (N is less than M) candidate motion vectors among the M candidate motion vectors that have been obtained are scanned in sequence, which can reduce the number of scans of the candidate motion vectors in the process of obtaining the reference motion vector of the current image block compared to the prior art. It should be understood that applying the solution provided by the present application to the first step of the existing ATMVP technology can simplify the redundant operations existing therein.
[0241] The applicant selected the official test sequence as the test sequence on the latest reference software VTM-2.0 of Versatile Video Coding, and tested the solution provided in this application with RA configuration and LDB configuration. The test results showed that the performance gain of ATMVP technology can be maintained after reducing the number of scans.
[0242] Therefore, the solution provided in this application can reduce the complexity of ATMVP technology while maintaining the performance gain of existing ATMVP technology.
[0243] Optionally, as an embodiment, the acquisition unit 510 is used to acquire M candidate motion vectors for adding to the second candidate list of motion vectors of the current image block according to the motion vectors of M neighboring blocks of the current image block in the current frame.
[0244] Optionally, as an embodiment, the neighboring block is an image block that is adjacent to the current image block or has a certain positional spacing with the current image block in the current frame.
[0245] Optionally, as an embodiment, the determination unit 520 is configured to sequentially scan the first N candidate motion vectors among the M candidate motion vectors.
[0246] Optionally, as an embodiment, M is equal to 4 and N is less than 4.
[0247] Optionally, as an embodiment, N is equal to 1 or 2.
[0248] Optionally, as an embodiment, the determination unit 520 is configured to, based on a preset condition, sequentially scan N candidate motion vectors among the M candidate motion vectors, and determine a reference motion vector according to the scanning result.
[0249] Optionally, as an embodiment, the preset condition includes: a candidate motion vector pointing to a reference frame that is the same as a reference image of the current image block.
[0250] Optionally, as an embodiment, the determination unit 520 is used to scan N candidate motion vectors in sequence, stop scanning when the first candidate motion vector that meets the preset conditions is scanned, and determine the reference motion vector based on the scanned first candidate motion vector that meets the preset conditions.
[0251] Optionally, as an embodiment, the determination unit 520 is used to, when no candidate motion vector meeting a preset condition is scanned among N candidate motion vectors, scale a specific candidate motion vector in the second candidate list of motion vectors, and determine a reference motion vector based on the scaled specific candidate motion vector.
[0252] Optionally, as an embodiment, the specific candidate motion vector is the first motion vector or the last motion vector obtained in a scanning order among the N candidate motion vectors.
[0253] Optionally, as an embodiment, the determination unit 520 is used to scale a specific candidate motion vector in the second candidate list of motion vectors so that the reference frame pointed to by the scaled specific candidate motion vector is the same as the reference image of the current image block; and the scaled specific candidate motion vector is used as a reference motion vector.
[0254] Optionally, as an embodiment, the determination unit 520 is configured to use a default value as a reference motion vector when no candidate motion vector meeting a preset condition is scanned among the N candidate motion vectors.
[0255] Optionally, as an embodiment, the default value is a motion vector (0, 0).
[0256] Optionally, as an embodiment, the determination unit 520 is used to divide the current image block into multiple sub-image blocks; determine the relevant blocks of the sub-image blocks in the reference image of the current image block according to the reference motion vector; and determine the candidate motion vector to be continued to be added to the second candidate list of motion vectors according to the motion vectors of the relevant blocks.
[0257] Optionally, as an embodiment, the size of the sub-image block and / or the size of the associated block of the sub-image block is fixed to be greater than or equal to 64 pixels.
[0258] Optionally, as an embodiment, the current image block is a coding unit CU.
[0259] Optionally, as an embodiment, the determination unit 520 is used to determine a related block of the current image block in the reference image of the current image block according to the reference motion vector; and determine a candidate motion vector to be added to the second candidate list of motion vectors according to the motion vector of the related block.
[0260] Optionally, as an embodiment, the determination unit 520 is used to determine, when the motion vector of the relevant block points to a specific reference image, or the reference image of the current image block is a specific reference image, to continue to add candidate motion vectors to the second candidate list of motion vectors based on the motion vector of the processed relevant block, wherein the motion vector of the processed relevant block is the same as the motion vector of the relevant block before processing.
[0261] Optionally, as an embodiment, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0262] Optionally, as an embodiment, the determination unit 520 is used to abandon determining the candidate motion vector to be added to the second candidate list of motion vectors based on the motion vector of the relevant block when the motion vector of the relevant block points to a specific reference image or the reference image of the current image block is a specific reference image.
[0263] Optionally, as an embodiment, the determination unit 520 is used to determine, when a specific candidate motion vector points to a specific reference image, or the reference image of the current image block is a specific reference image, the candidate motion vector to be continued to be added to the second candidate list of motion vectors according to the processed specific candidate motion vector, wherein the processed specific candidate motion vector is the same as the specific candidate motion vector before processing.
[0264] Optionally, as an embodiment, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0265] Optionally, as an embodiment, the determination unit 520 is used to abandon determining the candidate motion vector to be added to the second candidate list of motion vectors based on the specific candidate motion vector when the specific candidate motion vector points to a specific reference image or the reference image of the current image block is a specific reference image.
[0266] Optionally, as an embodiment, the second motion vector candidate list is a Merge candidate list.
[0267] Optionally, as an embodiment, the reference image of the current image block is a co-located frame of the current image block.
[0268] Optionally, as an embodiment, the size of the sub-image block and / or the size of the related block of the sub-image block are both fixed to 8×8 pixels.
[0269] Optionally, in some embodiments of the present application, when the size of the sub-image block and / or the size of the related blocks of the sub-image block is 8×8 pixels, setting not to perform the TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0270] Optionally, in some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, it is set not to perform the TMVP operation. In other words, when at least one of the width and height of the sub-image block and / or the related block of the sub-image block is less than 8 pixels, it is set not to perform the TMVP operation. For the above situation, the performance impact of skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0271] It should be understood that both the acquisition unit 510 and the determination unit 520 in this embodiment can be implemented by a processor.
[0272] like Figure 6 As shown, the embodiment of the present application also provides a video image processing device 600. The device 600 is used to perform the following Figure 4 The method embodiment shown in the figure. The device 600 includes the following units.
[0273] A determination unit 610 is configured to obtain M candidate motion vectors for adding to a second candidate list of motion vectors of a current image block;
[0274] A determination unit 620, configured to sequentially scan at least part of the M candidate motion vectors, and determine a reference motion vector of a current image block according to the scanning result;
[0275] A dividing unit 630, configured to divide the current image block into a plurality of sub-image blocks, wherein the size of the sub-image blocks is fixed to be greater than or equal to 64 pixels;
[0276] The determination unit 620 is further configured to determine a related block of the sub-image block in the reference image of the current image block according to the reference motion vector;
[0277] The determination unit 620 is further configured to determine, according to the motion vector of the relevant block, a candidate motion vector to be added to the second candidate list of motion vectors.
[0278] In the current ATMVP technology, the size of the sub-image block is set adaptively at the frame level. The size of the sub-image block defaults to 4×4. When certain conditions are met, the size of the sub-image block is set to 8×8. For example, at the encoding end, when encoding the current image block, the average block size of each sub-image block in the CU when the previous coded image block of the same time domain layer is encoded in ATMVP mode is calculated. When the average block size is greater than the threshold, the size of the sub-image block of the current image block is set to 8×8, otherwise the default value of 4×4 is used. In other words, in the prior art, when encoding the current image block, it is also necessary to store information about the size of the sub-image block of the previous coded image block of the same time domain layer.
[0279] In the embodiment of the present application, the size of the sub-image block of the current image block is fixed to be greater than or equal to 64 pixels, and there is no need to store information on the size of the sub-image block of the previous encoded image block, thereby saving storage space.
[0280] Optionally, as an embodiment, the size of the sub-image block and / or the size of the related block of the sub-image block are both fixed to 8×8 pixels.
[0281] At present, in the new generation of video coding standards (Versatile Video Coding, VVC), motion vectors are stored in a size of 8×8. In the embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8, which can adapt to the storage granularity of the motion vector specified in the video standard VVC on the one hand, and on the other hand, there is no need to store the size information of the sub-image block of the last encoded image block, so storage space can be saved.
[0282] In some embodiments of the present application, when the size of a sub-image block and / or the size of a related block of the sub-image block is 8×8 pixels, setting not to perform a TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0283] In some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, the TMVP operation is set not to be performed. In other words, when at least one of the width and height of the sub-image block and / or the related block of the sub-image block is less than 8 pixels, the TMVP operation is set not to be performed. For the above situation, the performance impact caused by skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0284] Optionally, as an embodiment, the determination unit 620 is used to scan at least part of the candidate motion vectors in sequence, stop scanning when the first candidate motion vector that meets the preset conditions is scanned, and determine the reference motion vector based on the scanned first candidate motion vector that meets the preset conditions.
[0285] Optionally, as an embodiment, the determination unit 620 is configured to use the first candidate motion vector that meets a preset condition as the target neighboring block.
[0286] Optionally, as an embodiment, the preset condition includes: a reference image of the candidate motion vector is the same as a reference image of the current image block.
[0287] It should be understood that the acquisition unit 610, the determination unit 620 and the division unit 630 in this embodiment can all be implemented by a processor.
[0288] In the above description, the motion vector of an image block contains two pieces of information: 1) the image to which the motion vector points; 2) the displacement. In some application scenarios, the motion vector of an image block only contains the information of "displacement". The image block also provides index information for indicating the reference image of the image block. For an encoded / decoded image block, the meaning contained in its motion vector includes: the displacement of the reference block of the encoded / decoded image block on the reference image relative to the image block that has the same position as the encoded / decoded image block and is located in the reference image. When determining the reference block of the encoded / decoded image block, it is necessary to determine the reference block of the encoded / decoded image block through the index information of the reference image of the encoded / decoded image block and the motion vector of the encoded / decoded image block. Then, in Figure 1 In the video image processing method shown, step S120 does not scan the candidate motion vectors in the second candidate list of motion vectors, but directly scans the image block corresponding to the candidate motion vector. The video image processing method is provided below for the new definition of the motion vector (that is, including "displacement" information but not including "the image pointed to"). It should be noted that the methods for determining the candidate motion vector based on the ATMVP technology provided for the two different meanings of "motion vector" are basically the same, and the above explanation is also applicable to the video image processing method provided below. The difference is mainly in the construction of the second candidate list of motion vectors. When determining the candidates to be added to the second candidate list of motion vectors according to the ATMVP technology, the video image processing method described above scans the motion vectors added to the second candidate list of motion vectors, while the video image processing method provided below scans the image blocks corresponding to the motion vectors added to the second candidate list of motion vectors.
[0289] like Figure 7 As shown, an embodiment of the present application provides a video image processing method, which includes the following steps.
[0290] S710, determining M neighboring blocks of the current image block.
[0291] The current image block is an image block to be encoded (or decoded). For example, the current image block is a coding unit (CU).
[0292] The image frame where the current image block is located is called the current frame.
[0293] The neighboring blocks are image blocks that are adjacent to the current image block or have a certain positional distance with the current image block on the current image.
[0294] The M neighboring blocks are image blocks that have been encoded (or decoded) in the current frame.
[0295] As an example, Figure 2 As shown, press Figure 2 The order of the image blocks located at the four positions A1 (left) → B1 (upper) → B0 (upper right) → A0 (lower left) around the current image block as shown in the figure is used to determine the four neighboring blocks of the current image block in sequence.
[0296] S720 , sequentially scan N neighboring blocks among the M neighboring blocks, and determine a target neighboring block according to the scanning result, where N is less than M.
[0297] The process of determining the target neighboring block according to the scanning results of the N neighboring blocks may be to judge the N neighboring blocks in sequence based on preset conditions, and determine the target neighboring block according to the judgment results.
[0298] As an example, the preset condition is defined as that the reference image of the neighboring block is the same as the reference image of the current image block.
[0299] Among them, the reference image of the current image block is the reference image that is closest to the image where the current image block is located in time; or, the reference image of the current image block is the reference image preset by the codec end; or, the reference image of the current image block is the reference image specified in the video parameter set, sequence header, sequence parameter set, image header, image parameter set, and slice header.
[0300] For example, the reference image of the current image block is the co-located frame of the current image block, and the co-located frame is a frame set in the slice-level information header and used to obtain motion information for prediction.
[0301] It should be understood that, depending on the evolution of future technology, the preset condition may be given other different definitions, and the corresponding solutions also fall within the scope of protection of this application.
[0302] The process of determining the target neighboring block according to the scanning results of N neighboring blocks will be described in detail below.
[0303] In step S720, only N neighboring blocks among the M neighboring blocks obtained in step S710 are scanned, which can reduce the number of scans.
[0304] Optionally, in step S720, the first N neighboring blocks among the M neighboring blocks may be scanned in sequence.
[0305] When the M neighboring blocks of the current image block are determined in sequence according to a preset order in step S710, the first N neighboring blocks obtained in step S720 refer to the N neighboring blocks that are first determined according to the preset order.
[0306] Optionally, in step S720, the last N neighboring blocks among the M neighboring blocks may be scanned in sequence; or the middle N neighboring blocks among the M neighboring blocks may be scanned in sequence. This application does not limit this.
[0307] S730 , determining a related block of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block.
[0308] S740, encoding / decoding the current image block according to the motion vector of the related block.
[0309] Optionally, step S740 includes: determining a reference block of the current image block according to a motion vector of a related block and a reference image.
[0310] For example, step S740 includes: constructing a candidate block list of the current image block, wherein the candidate blocks in the candidate block list include M adjacent blocks and related blocks; and encoding and decoding the current image block according to the reference blocks of the candidate blocks in the candidate block list.
[0311] In one example, the candidate block list is a merge candidate list of the current image block. In one example, the candidate block list is an AMVP candidate list of the current image block.
[0312] At the encoding end, the index of the candidate block of the current block is written into the bitstream. At the decoding end, after obtaining the index, the candidate block corresponding to the index is found from the candidate block list, and the reference block of the current image block is determined based on the reference block of the candidate block, or the motion vector of the current image block is determined based on the motion vector of the candidate block.
[0313] For example, the reference block of the candidate block is directly used to determine the reference block of the current image block, or the motion vector of the candidate block is directly used to determine the motion vector of the current image block. For another example, the encoder also writes the MVD of the current block into the bitstream. After the decoder obtains the MVD, it adds the motion vector of the candidate block to the MVD as the motion vector of the current block, and then determines the reference block of the current block based on the motion vector and the reference image of the current block.
[0314] In an embodiment of the present application, in the process of acquiring the target neighboring block of the current image block, only N (N is less than M) neighboring blocks among the M neighboring blocks that have been acquired are scanned in sequence. Compared with the prior art, the number of scans of candidate neighboring blocks in the process of acquiring the target neighboring block of the current image block can be reduced, thereby reducing complexity.
[0315] Optionally, in this embodiment, in step S710, four neighboring blocks of the current image block in the current frame are determined, that is, M is equal to 4. In step S720, N neighboring blocks of the four neighboring blocks are scanned, where N is less than 4.
[0316] For example, N is equal to 1. For example, in step S720, only the first neighboring block among the four neighboring blocks is scanned.
[0317] For another example, N is equal to 2 or 3.
[0318] The following will describe the manner of determining the target neighboring block according to the scanning results of the N neighboring blocks in step S720.
[0319] Optionally, in step S720, N neighboring blocks are scanned in sequence, and when the first neighboring block meeting the preset condition is scanned, the scanning is stopped, and the target neighboring block is determined according to the first scanned neighboring block meeting the preset condition.
[0320] For example, the preset condition is defined as that the reference image of the neighboring block is the same as the reference image of the current image block.
[0321] It should be understood that in the future evolving technology, the preset condition may be given other definitions.
[0322] In the following, description is given by taking the case where the preset condition is defined as the reference image of the neighboring block being the same as the reference image of the current image block as an example.
[0323] For example, the first neighboring block that meets the preset conditions is used as the target neighboring block.
[0324] Optionally, when no neighboring block meeting the preset condition is scanned among the N neighboring blocks in step S720, the method further includes: scaling the motion vector of a specific neighboring block among the M neighboring blocks, and encoding / decoding the current image block according to the scaled motion vector.
[0325] For example, the reference block of the current image block is determined according to the motion vector after scaling and the reference image of the current image block.
[0326] Optionally, the specific neighboring block is the first neighboring block or the last neighboring block obtained in a scanning order among the N neighboring blocks.
[0327] The specific neighboring block may also be a neighboring block obtained in other scanning orders among the N neighboring blocks.
[0328] Optionally, encoding / decoding the current image block according to the scaled motion vector includes: scaling the motion vector of a specific neighboring block so that the reference frame pointed to by the scaled motion vector is the same as the reference image of the current image block; and using the image block pointed to by the scaled motion vector in the reference image of the current image block as the reference block of the current image block.
[0329] Optionally, in step S720, when no neighboring block meeting a preset condition is scanned among the N neighboring blocks, a default block is used as a candidate reference block for the current image block.
[0330] For example, the default block is an image block pointed to by the motion vector (0, 0).
[0331] The following describes the process of determining the related blocks of the current image block according to the motion vector of the target neighboring block, the current image block and the reference image of the current image block in step S730.
[0332] Optionally, as an implementation method, determining the relevant blocks of the current image block according to the motion vector of the target neighboring block, the current image block and the reference image of the current image block includes: dividing the current image block into multiple sub-image blocks; determining the relevant blocks of the sub-image blocks in the reference image of the current image block according to the motion vector of the target neighboring block, the relevant blocks of the current image block including the relevant blocks of the sub-image blocks.
[0333] The related block can be called collocated block or corresponding block.
[0334] For example, the current image block is a CU, and the sub-image blocks obtained by dividing the current image block may be referred to as sub-CUs.
[0335] Optionally, the size of the sub-image block and / or the size of a block associated with the sub-image block is fixed to be greater than or equal to 64 pixels.
[0336] Optionally, the size of the sub-image block and / or the size of the associated block of the sub-image block are both fixed to 8×8 pixels.
[0337] In the current ATMVP technology, the size of the sub-image block is set adaptively at the frame level. The size of the sub-image block defaults to 4×4. When certain conditions are met, the size of the sub-image block is set to 8×8. For example, at the encoding end, when encoding the current image block, the average block size of each sub-image block in the CU when the previous coded image block of the same time domain layer is coded in ATMVP mode is calculated. When the average block size is greater than the threshold, the size of the sub-image block of the current image block is set to 8×8, otherwise the default value of 4×4 is used. At present, in the new generation of video coding standards (Versatile Video Coding, VVC), motion vectors are stored in a size of 8×8. It should be understood that when the size of the sub-image block is set to 4×4, the size of the motion vector of the sub-image block (also 4×4) does not meet the storage granularity of the motion vector in the current standard. In addition, in the current ATMVP technology, when encoding the current image block, it is also necessary to store information on the size of the sub-image block of the previous coded image block of the same time domain layer.
[0338] In an embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8. On the one hand, it can adapt to the storage granularity of the motion vector specified in the video standard VVC. On the other hand, there is no need to store information about the size of the sub-image block of the previous encoded image block, thereby saving storage space.
[0339] In some embodiments of the present application, when the size of a sub-image block and / or the size of a related block of the sub-image block is 8×8 pixels, setting not to perform a TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0340] In some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, the TMVP operation is set not to be performed. In other words, when at least one of the width and height of the sub-image block and / or the related block of the sub-image block is less than 8 pixels, the TMVP operation is set not to be performed. For the above situation, the performance impact caused by skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0341] It should be understood that, under the premise of ensuring that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block can also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.
[0342] Optionally, as another implementation manner, determining the relevant blocks of the current image block according to the motion vector of the target neighboring block, the current image block and the reference image of the current image block includes: determining the relevant blocks of the current image block in the reference image of the current image block according to the motion vector of the target neighboring block.
[0343] Optionally, step S740 includes: when the reference image of the relevant block is a specific reference image, or the reference image of the current image block is a specific reference image, determining a candidate reference block of the current image block according to the processed motion vector of the relevant block and the reference image of the current image block; wherein the motion vector of the processed relevant block is the same as the motion vector of the relevant block before processing.
[0344] For example, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0345] Optionally, step S740 includes: when the reference image of the relevant block is a specific reference image, or the reference image of the current block is a specific reference image, abandoning determining the candidate reference block of the current image block according to the motion vector of the relevant block.
[0346] In some embodiments, step S720 includes: when the motion vector of a specific neighboring block points to a specific reference image, or the reference image of the current image block is a specific reference image, determining the reference block of the current image block based on the motion vector of the processed related block and the reference image of the current image block; wherein the motion vector of the processed related block is the same as the motion vector of the related block before processing.
[0347] The processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0348] From the above, it can be seen that in the embodiment of the present application, in the process of obtaining the target neighboring block of the current image block, only N (N is less than M) neighboring blocks among the M neighboring blocks that have been obtained are scanned in sequence. Compared with the prior art, the number of scans of candidate neighboring blocks in the process of obtaining the target neighboring block of the current image block can be reduced, thereby reducing complexity.
[0349] When no neighboring block whose reference frame is the same as the collocated frame of the current frame is scanned among the N neighboring blocks, the motion vector of one of the N neighboring blocks is scaled so that its reference frame is the same as the collocated frame of the current frame, and then the scaled motion vector is used as the motion vector of the current image block, which can improve the accuracy of the motion vector of the current image block.
[0350] Dividing the current image block into sub-image blocks of 8×8 size can, on the one hand, adapt to the storage granularity of motion vectors specified in the video standard VVC, and on the other hand, there is no need to store information on the size of the sub-blocks of the previous encoded image block, thus saving storage space.
[0351] In some embodiments of the present application, when the size of a sub-image block and / or the size of a related block of the sub-image block is 8×8 pixels, setting not to perform a TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0352] In some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, the TMVP operation is set not to be performed. In other words, when at least one of the width and height of the sub-image block and / or the related block of the sub-image block is less than 8 pixels, the TMVP operation is set not to be performed. For the above situation, the performance impact caused by skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0353] In some implementations of the embodiments of the present application, in the process of determining the motion vector of the current image block, it is involved to use the motion vector of a certain image block on other images to determine the motion vector of the image block. For the convenience of description, the image block is called the first image block, and the certain image block on the other image to be used is called the time domain reference block or related block of the first image block. It can be understood that the first image block and the time domain reference block (or related block) of the first image block are located on different images. Then, in the process of determining the motion vector of the first image block using the motion vector of the time domain reference block (or related block), it may be necessary to scale the motion vector of the time domain reference block (or related block). For the convenience of description, the term "related block" is uniformly used in this article.
[0354] For example, when the ATMVP technology is applied in constructing the AMVP candidate list, after determining the relevant block of the current image block according to the ATMVP technology, when determining the motion vector of the current image block according to the motion vector of the relevant block, it is necessary to scale the motion vector of the relevant block, and then determine the motion vector of the current image block according to the scaled motion vector. Generally speaking, the scaling ratio of the motion vector of the relevant block is determined based on the time distance between the reference image pointed to by the motion vector of the relevant block and the image where the relevant block is located, and the time distance between the reference image of the current image block and the image where the current image block is located.
[0355] In one example, the motion vector of the relevant block is called MV 2, and the reference frame index value of the reference image pointed to by the motion vector MV 2 is x. The reference frame index value x is the difference between the sequence number (for example, POC) of the reference image pointed to by MV 2 and the sequence number of the image where the relevant block is located. The reference frame index value of the reference image of the first image block is called y. The reference frame index value y is the difference between the sequence number of the reference image of the first image block and the sequence number of the image where the first image block is located. Then, the scaling ratio of the motion vector MV 2 is y / x. Optionally, the product of the motion vector MV 2 and y / x can be used as the motion vector of the first image block.
[0356] However, when the motion vector MV 2 of the relevant block points to a specific reference image, or when the reference image of the first image block is a specific reference image, since the temporal distance between the specific reference image and the image where the first image block is located is not clearly defined, scaling the motion vector MV 2 of the relevant block is meaningless.
[0357] Optionally, in this embodiment, when the motion vector of the current image block is determined according to the motion vector of the related block, specifically: when the motion vector of the related block points to a specific reference image, or the reference image of the current image block is a specific reference image, the motion vector of the current image block is determined according to the motion vector of the processed related block, wherein the motion vector of the processed related block is the same as the motion vector of the related block before processing.
[0358] For example, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0359] Optionally, in this embodiment, when the motion vector of the current image block is determined according to the motion vector of the related block, specifically: when the motion vector of the related block points to a specific reference image, or the reference image of the current image block is a specific reference image, the determination of the motion vector of the current image block according to the motion vector of the related block is abandoned.
[0360] like Figure 8 As shown, an embodiment of the present application also provides a video image processing method, which includes the following steps.
[0361] S810, determining M neighboring blocks of the current image block.
[0362] Step S810 may correspond to step S710 in the above embodiment.
[0363] S820, sequentially scan at least part of the M neighboring blocks, and determine a target neighboring block according to the scanning result.
[0364] Optionally, some of the M neighboring blocks are scanned in sequence, and the target neighboring block is determined according to the scanning result.
[0365] Optionally, all the neighboring blocks among the M neighboring blocks are scanned in sequence, and the target neighboring block is determined according to the scanning result.
[0366] S830, dividing the current image block into a plurality of sub-image blocks, wherein the size of the sub-image blocks is fixed to be greater than or equal to 64 pixels.
[0367] S840, determining a related block of the current image block in a reference image of the current image block according to the motion vector of the target neighboring block and the sub-image block.
[0368] Optionally, the reference image of the current image block is a reference image that is closest in time distance to the image where the current image block is located.
[0369] Optionally, the reference image of the current image block is a reference image preset by the codec end.
[0370] Optionally, the reference image of the current image block is a reference image specified in a video parameter set, a sequence header, a sequence parameter set, a picture header, a picture parameter set, or a slice header.
[0371] S850, encoding / decoding the current image block according to the motion vector of the related block.
[0372] In the embodiment of the present application, the size of the sub-image block of the current image block is fixed to be greater than or equal to 64 pixels, and there is no need to store information on the size of the sub-image block of the previous encoded image block, thereby saving storage space.
[0373] Optionally, in this embodiment, the size of the sub-image block and / or the size of the temporal reference block of the sub-image block are both fixed to 8×8 pixels.
[0374] At present, in the new generation of video coding standards (Versatile Video Coding, VVC), motion vectors are stored in a size of 8×8. In the embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8, which can adapt to the storage granularity of the motion vector specified in the video standard VVC on the one hand, and on the other hand, there is no need to store the size information of the sub-image block of the last encoded image block, so storage space can be saved.
[0375] In some embodiments of the present application, when the size of a sub-image block and / or the size of a related block of the sub-image block is 8×8 pixels, setting not to perform a TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0376] In some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, the TMVP operation is set not to be performed. In other words, when at least one of the width and height of the sub-image block and / or the related block of the sub-image block is less than 8 pixels, the TMVP operation is set not to be performed. For the above situation, the performance impact caused by skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0377] It should be understood that, under the premise of ensuring that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block may also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.
[0378] Optionally, step S820 includes: scanning at least part of the neighboring blocks in sequence, stopping the scanning when the first neighboring block meeting the preset condition is scanned, and determining the target neighboring block according to the scanned first neighboring block meeting the preset condition.
[0379] For example, the first neighboring block that meets the preset conditions is used as the target neighboring block.
[0380] For example, the preset condition is defined as: the reference image of the neighboring block is the same as the reference image of the current image block.
[0381] Optionally, step S840 includes: determining relevant blocks of the sub-image block in a reference image of the current image block according to the motion vector of the target neighboring block and the sub-image block, wherein the relevant blocks of the current image block include relevant blocks of the sub-image block.
[0382] Combination of the above Figure 7 and Figure 8 The method embodiment of the present application is described below. Figure 7 and Figure 8 The device embodiment corresponding to the method embodiment shown. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the contents not described in detail can be referred to the previous method embodiment, and will not be repeated here for the sake of brevity.
[0383] Fig. 9 Schematic block diagram of a video image processing device 900 provided in an embodiment of the present application. The device 900 is used to perform the following Figure 7 The method embodiment shown in the figure. The device 900 includes the following units.
[0384] An acquisition unit 910 is used to acquire M neighboring blocks of a current image block;
[0385] A determination unit 920 is configured to sequentially scan N neighboring blocks among the M neighboring blocks and determine a target neighboring block according to the scanning result, where N is less than M.
[0386] The determination unit 920 is further configured to determine a related block of the current image block according to a motion vector of a target neighboring block, the current image block, and a reference image of the current image block;
[0387] The encoding / decoding unit 930 is used to encode / decode the current image block according to the motion vector of the related block.
[0388] In an embodiment of the present application, in the process of acquiring the target neighboring block of the current image block, only N (N is less than M) neighboring blocks among the M neighboring blocks that have been acquired are scanned in sequence. Compared with the prior art, the number of scans of candidate neighboring blocks in the process of acquiring the target neighboring block of the current image block can be reduced, thereby reducing complexity.
[0389] Optionally, as an embodiment, M is equal to 4 and N is less than 4.
[0390] Optionally, as an embodiment, N is equal to 1 or 2.
[0391] Optionally, as an embodiment, the determination unit 920 is configured to sequentially scan the first N neighboring blocks among the M neighboring blocks.
[0392] Optionally, as an embodiment, the acquisition unit 910 is used to sequentially acquire M neighboring blocks of the current image block in a preset order; the first N neighboring blocks refer to the N neighboring blocks first determined in the preset order.
[0393] Optionally, as an embodiment, the determination unit 920 is used to scan N neighboring blocks in sequence, stop scanning when the first neighboring block that meets the preset conditions is scanned, and determine the target neighboring block based on the first scanned neighboring block that meets the preset conditions.
[0394] Optionally, as an embodiment, the determining unit 920 is configured to take the first neighboring block that meets a preset condition as the target neighboring block.
[0395] Optionally, as an embodiment, the preset condition includes: a reference image of a neighboring block is the same as a reference image of a current image block.
[0396] Optionally, as an embodiment, the encoding / decoding unit 930 is configured to determine a reference block of the current image block according to a motion vector of a related block and a reference image.
[0397] Optionally, as an embodiment, the encoding / decoding unit 930 is used to construct a candidate block list for the current image block, the candidate blocks in the candidate block list include M adjacent blocks and related blocks; and encode and decode the current image block according to the reference blocks of the candidate blocks in the candidate block list.
[0398] Optionally, as an embodiment, the encoding / decoding unit 930 is also used to, when no neighboring block meeting a preset condition is scanned among the N neighboring blocks, scale the motion vector of a specific neighboring block among the M neighboring blocks, and encode / decode the current image block according to the scaled motion vector.
[0399] Optionally, as an embodiment, the encoding / decoding unit 930 is configured to determine a reference block of the current image block according to the scaled motion vector and the reference image of the current image block.
[0400] Optionally, as an embodiment, the specific neighboring block is the first neighboring block or the last neighboring block obtained in a scanning order among the N neighboring blocks.
[0401] Optionally, as an embodiment, the encoding / decoding unit 930 is used to scale the motion vector of a specific neighboring block so that the reference frame pointed to by the scaled motion vector is the same as the reference image of the current image block; and the image block pointed to by the scaled motion vector in the reference image of the current image block is used as the reference block of the current image block.
[0402] Optionally, as an embodiment, the determining unit 920 is configured to use a default block as a reference block for the current image block when no adjacent block meeting a preset condition is scanned among the N adjacent blocks.
[0403] Optionally, as an embodiment, the default block is an image block pointed to by the motion vector (0, 0).
[0404] Optionally, as an embodiment, the determining unit 920 is configured to:
[0405] Divide the current image block into a plurality of sub-image blocks;
[0406] According to the motion vector of the target neighboring block, a related block of the sub-image block is determined in the reference image of the current image block, and the related block of the current image block includes the related block of the sub-image block.
[0407] Optionally, as an embodiment, the size of the sub-image block and / or the size of the associated block of the sub-image block is fixed to be greater than or equal to 64 pixels.
[0408] Optionally, in some embodiments of the present application, when the size of the sub-image block and / or the size of the related blocks of the sub-image block is 8×8 pixels, setting not to perform the TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0409] Optionally, in some embodiments of the present application, when the width and / or height of the current CU is less than 8 pixels, it is set not to perform the TMVP operation. In other words, when at least one of the width and height of the sub-image block and / or the related block of the sub-image block is less than 8 pixels, it is set not to perform the TMVP operation. For the above situation, the performance impact of skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0410] Optionally, as an embodiment, the current image block is a coding unit CU.
[0411] Optionally, as an embodiment, the determining unit 920 is configured to determine a related block of the current image block in a reference image of the current image block according to a motion vector of a target neighboring block.
[0412] Optionally, as an embodiment, the neighboring block is an image block that is adjacent to the current image block or has a certain positional distance with the current image block on the current image.
[0413] Optionally, as an embodiment, the encoding / decoding unit 930 is configured to, when the reference image of the relevant block is a specific reference image, or the reference image of the current image block is a specific reference image, determine the reference block of the current image block according to the processed motion vector of the relevant block and the reference image of the current image block;
[0414] The motion vector of the related block after processing is the same as the motion vector of the related block before processing.
[0415] Optionally, as an embodiment, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0416] Optionally, as an embodiment, the encoding / decoding unit 930 is configured to abandon determining the reference block of the current image block according to the motion vector of the relevant block when the reference image of the relevant block is a specific reference image or the reference image of the current block is a specific reference image.
[0417] Optionally, as an embodiment, the determination unit 920 is used to: when the motion vector of a specific neighboring block points to a specific reference image, or the reference image of the current image block is a specific reference image, determine the reference block of the current image block based on the motion vector of the processed related block and the reference image of the current image block; wherein the motion vector of the processed related block is the same as the motion vector of the related block before processing.
[0418] Optionally, as an embodiment, the processed motion vector of the relevant block includes: a motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or a motion vector of the relevant block with the scaling step skipped.
[0419] It should be understood that the acquisition unit 910, the determination unit 920 and the encoding / decoding unit 930 in this embodiment can all be implemented by a processor.
[0420] like Fig.10 As shown, the embodiment of the present application also provides a video image processing device 1000. The device 1000 is used to perform the following Figure 8 The method embodiment shown. The device 1000 includes the following units.
[0421] An acquisition unit 1010 is used to acquire M neighboring blocks of a current image block;
[0422] A determining unit 1020 is configured to sequentially scan at least some of the M neighboring blocks and determine a target neighboring block according to the scanning result;
[0423] A dividing unit 1030 is used to divide the current image block into a plurality of sub-image blocks, wherein the size of the sub-image blocks is fixed to be greater than or equal to 64 pixels;
[0424] The determining unit 1020 is further configured to determine a related block of the current image block in a reference image of the current image block according to the motion vector of the target neighboring block and the sub-image block;
[0425] The encoding / decoding unit 1040 is configured to encode / decode the current image block according to the motion vector of the related block.
[0426] In the embodiment of the present application, the size of the sub-image block of the current image block is fixed to be greater than or equal to 64 pixels, and there is no need to store information on the size of the sub-image block of the previous encoded image block, thereby saving storage space.
[0427] Optionally, as an embodiment, the size of the sub-image block and / or the size of the temporal reference block of the sub-image block are both fixed to 8×8 pixels.
[0428] At present, in the new generation of video coding standards (Versatile Video Coding, VVC), motion vectors are stored in a size of 8×8. In the embodiment of the present application, the size of the sub-image block of the current image block is set to 8×8, which can adapt to the storage granularity of the motion vector specified in the video standard VVC on the one hand, and on the other hand, there is no need to store the size information of the sub-image block of the last encoded image block, so storage space can be saved.
[0429] In some embodiments of the present application, when the size of the sub-image block in the ATMVP technology and / or the size of the related blocks of the sub-image block is 8×8 pixels, setting not to perform the TMVP operation can skip some redundant operations, effectively save encoding and decoding time, and improve encoding efficiency.
[0430] In some embodiments of the present application, when at least one of the width and height of the current CU is less than 8, TMVP operation is set not to be performed, and the performance impact of skipping the TMVP operation can be ignored, thereby effectively saving encoding and decoding time and improving encoding efficiency.
[0431] It should be understood that, under the premise of ensuring that the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to be equal to 64 pixels, the size of the sub-image block and / or the size of the block associated with the sub-image block can also be other sizes, for example, the size of the sub-image block and / or the size of the block associated with the sub-image block is A×B, A≤64, B≤64, and A and B are both integers of 4. For example, the size of the sub-image block and / or the size of the block associated with the sub-image block is 4×16 pixels, or 16×4 pixels.
[0432] Optionally, as an embodiment, at least part of the M neighboring blocks are scanned in sequence, and a target neighboring block is determined based on the scanning results, including: scanning at least part of the neighboring blocks in sequence, and when the first neighboring block that meets a preset condition is scanned, stopping the scanning, and determining the target neighboring block based on the scanned first neighboring block that meets the preset condition.
[0433] Optionally, as an embodiment, the determining unit 1020 is configured to take the first neighboring block that meets a preset condition as the target neighboring block.
[0434] Optionally, as an embodiment, the preset condition includes: a reference image of a neighboring block is the same as a reference image of a current image block.
[0435] Optionally, as an embodiment, the determination unit 1020 is used to determine the relevant blocks of the sub-image block in the reference image of the current image block according to the motion vector of the target neighboring block and the sub-image block, wherein the relevant blocks of the current image block include the relevant blocks of the sub-image block.
[0436] It should be understood that the acquisition unit 1010, the determination unit 1020, the division unit 1030 and the encoding / decoding unit 1040 in this embodiment can all be implemented by a processor.
[0437] like Fig.11 As shown, the embodiment of the present application also provides a video image processing device 1100. The device 1100 can be used to execute the method embodiment described above. The device 1100 includes a processor 1110 and a memory 1120, the memory 1120 is used to store instructions, the processor 1110 is used to execute the instructions stored in the memory 1120, and the execution of the instructions stored in the memory 1120 enables the processor 1110 to execute the method according to the method embodiment described above.
[0438] Alternatively, if Fig.11 As shown, the apparatus 1100 may further include a communication interface 1130 for communicating with an external device. For example, the processor 1110 is used to control the communication interface 1130 to receive and / or send signals.
[0439] The apparatuses 500, 600, 900, 1000 and 1100 provided in the present application can be applied to an encoder or a decoder.
[0440] The second motion vector candidate list is explained above, and the first motion vector candidate list will be explained below.
[0441] In the motion compensation prediction stage, the previous mainstream video coding standards only applied the translation motion model. In the real world, there are too many forms of motion, such as zooming in / out, rotation, perspective motion and other irregular motions. In order to improve the efficiency of inter-frame prediction, the affine motion compensation model can be introduced in the encoding and decoding technology. Affine motion compensation describes the affine motion field of the image block through the MV of a set of control points. In an example, the affine motion compensation model adopts a four-parameter Affine model, and the group of control points includes two control points (for example, the upper left corner and the upper right corner of the image block). In an example, the affine motion compensation model adopts a six-parameter Affine model, and the group of control points includes three control points (for example, the upper left corner, the upper right corner and the lower left corner of the image block).
[0442] In one implementation, when constructing the first candidate list of motion vectors, the added candidates may be a set of MVs of control points, or referred to as control point motion vector prediction (CPMVP). Optionally, the first candidate list of motion vectors may be used in Merge mode, specifically, may be referred to as Affine Merge mode; correspondingly, the first candidate list of motion vectors may be referred to as affine mergecandidate list. In Affine Merge mode, the prediction in the first candidate list of motion vectors is directly used as the CPMV (Control point motion vector) of the current image block, i.e., there is no need to perform an affine motion estimation process.
[0443] In one implementation, the candidates determined according to the ATMVP technique may be added to the first candidate list of motion vectors.
[0444] In one example, the control point motion vector group of the related blocks of the current image block is added as a candidate to the first candidate list of motion vectors. When the candidate in the first list of motion vectors is used for prediction, the current image block is predicted according to the control point motion vector group of the related blocks of the current image block.
[0445] In one example, as described above, the representative motion vector of the relevant block of the current image block is added as a candidate to the first candidate list of motion vectors. Further, optionally, the candidate is also marked as determined according to the ATMVP technology. When the candidate in the first candidate list of motion vectors is used for prediction, the relevant block of the current image block is determined according to the mark and the candidate, and the current image block and the relevant block are divided into multiple sub-image blocks in the same way, and each sub-image block in the current image block corresponds to each sub-image block in the relevant block one by one; the motion vectors of the corresponding sub-image blocks in the current image block are predicted according to the motion vectors of each sub-image block in the relevant block.
[0446] Wherein, optionally, when a sub-image block with an unavailable motion vector appears in the relevant block, the representative motion vector of the relevant block is used to replace the unavailable motion vector, and the corresponding sub-image block in the current image block is predicted. Optionally, when the representative motion vectors of the relevant blocks are all unavailable, the candidate determined according to the ATMVP technology is abandoned to be added to the second candidate list of the motion vector. In one example, when the sub-image block in the relevant block is unavailable, or the sub-image block in the relevant block adopts the intra-frame coding mode, the sub-image block with an unavailable motion vector in the relevant block is determined.
[0447] Wherein, optionally, each candidate in the first candidate list of motion vectors includes a motion vector of a group of control points; when the representative motion vector of the relevant block of the current image block is added to the first candidate list of motion vectors, in order to ensure the consistency of the data format, the representative motion vector of the relevant block can be inserted as the motion vector of each control point in the candidate (that is, the motion vector of each control point in the candidate is assigned to the representative motion vector of the relevant block).
[0448] Optionally, the representative motion vector of the relevant block of the current image block may refer to a motion vector at the center position of the relevant block, or other motion vectors representing the relevant block, which is not limited here.
[0449] According to the above description of the second candidate list of motion vectors, when determining the candidates according to the ATMVP technology, it is necessary to determine the relevant blocks of the current image block. In this solution, when determining the candidates to be added to the first candidate list of motion vectors according to the ATMVP technology, there are two methods for determining the relevant blocks of the current image block:
[0450] Method 1: Scan N neighboring blocks among the preset M neighboring blocks of the current image block in sequence, determine the target neighboring block according to the scanning result, N is less than M, and M is less than or equal to 4; determine the related block of the current image block according to the motion vector of the target neighboring block, the current image block and the reference image of the current image block.
[0451] Method 2: Determine the M neighboring blocks of the current image block based on the M candidates in the second candidate list of the motion vector of the current image block; sequentially scan N neighboring blocks among the M neighboring blocks, and determine the target neighboring block based on the scanning results, where N is less than M and M is less than or equal to 4; determine the related blocks of the current image block based on the motion vector of the target neighboring block, the current image block, and the reference image of the current image block. The M candidates in the second candidate list of the motion vector may refer to the M neighboring blocks of the current image block.
[0452] Among them, for the description of the two steps of "determining the target neighboring block according to the scanning result" and "determining the related block of the current image block according to the motion vector of the target neighboring block, the current image block and the reference image of the current image block" in Method 1 and Method 2, please refer to the explanation above and will not be repeated here.
[0453] In one implementation, a method for determining candidates to be added to a first candidate list of motion vectors includes: determining, from neighboring blocks of a current image block, a control point motion vector group of neighboring blocks predicted using an affine transformation mode in a specific scanning order; and adding each determined control point motion vector group of the neighboring blocks as a candidate to the first candidate list of motion vectors.
[0454] In one example, the neighboring block predicted by the affine change mode means that the motion vector of the neighboring block is determined according to the candidates in the affine merge candidate list. That is, the candidates come from the affine motion model of the spatial neighboring blocks of the current image block using the affine mode; that is, the CPMV of the spatial neighboring blocks using the affine mode is used as the CPMVP of the current block.
[0455] In one example, the control point motion vector group may include the motion vectors of two control points of the neighboring block (e.g., the upper left corner and the upper right corner of the neighboring block), or include the motion vectors of three control points of the neighboring block (e.g., the upper left corner, the upper right corner and the lower left corner of the image block), depending on whether the four-parameter Affine model or the six-parameter Affine model is adopted.
[0456] In one example, determining a control point motion vector group of a neighboring block predicted using an affine transformation mode in a specific scanning order includes:
[0457] Determine a control point motion vector group of a first neighboring block in a left neighboring block of the current image block according to a first scanning order;
[0458] Determine a control point motion vector group of a second adjacent block in an upper adjacent block of the current image block according to a second scanning order;
[0459] The control point motion vector group of the first neighboring block and the control point motion vector group of the second neighboring block are added to the first motion vector candidate list.
[0460] For example, Fig.12 As shown, Fig.12 This is a schematic diagram of obtaining candidates for the first candidate list of motion vectors through neighboring blocks of the current image block. On the left side of the current image block, scan in sequence according to the scanning order of image block A->image block D->image block E, and add the control point motion vector group of the first image block that meets the preset conditions as a candidate to the first candidate list of motion vectors. On the upper side of the current image block, scan in sequence according to the scanning order of image block B->image block C, and add the control point motion vector group of the first image block that meets the preset conditions as a candidate to the first candidate list of motion vectors. Optionally, in this scanning order, if no image block that meets the threshold condition is found, then give up determining the candidate in this scanning order.
[0461] In one implementation, a method for determining a candidate to be added to a first candidate list of motion vectors includes:
[0462] constructing motion vectors of some control points of the current image block according to neighboring blocks of the some control points;
[0463] The motion vectors of some control points of the current image block are added to the first motion vector candidate list.
[0464] That is, in this implementation, the candidate is added to the first candidate list of motion vectors by constructing the candidate. In one example, before adding the candidate to the first candidate list of motion vectors by constructing the candidate, it is first determined whether the number of candidates in the first candidate list of motion vectors has reached a preset value (e.g., 5). If the preset value has not been reached, the candidate is added to the first candidate list of motion vectors by constructing the candidate.
[0465] In one example, the constructed candidate is a combination of motion information of neighboring blocks of some control points of the current image block and is added to the first candidate list of motion vectors as CPMVP.
[0466] like Fig.13 As shown, Fig.13 This is a schematic diagram of constructing the candidates of the first candidate list of motion vectors through the neighboring blocks of the current image block. The current image block has a total of four control points, namely CP1, CP2, CP3, and CP4. Among them, image blocks A0 and A1 are the spatial neighboring blocks of CP1; image blocks A2, B2, and B3 are the spatial neighboring blocks of CP2; image blocks B0 and B1 are the spatial neighboring blocks of CP2, and T is the temporal neighboring block of CP4. The coordinates of control points CP1, CP2, CP3, and CP4 are: (0, 0), (W, 0), (H, 0), and (W, H), respectively, where W and H represent the width and height of the current CU, respectively. The priority for obtaining the motion information of the neighboring blocks of each control point is:
[0467] For CP1, the acquisition priority is: B2->B3->A2. When B2 is available, use the MV of B2 as the MV of control point CP1; when B2 is not available, use the MV of B3 as the MV of control point CP1; if both B2 and B3 are not available, use the MV of A1 as the MV of control point CP1; if B2, B3 and A1 are not available, the motion information of control point CP1 is not available.
[0468] Similarly, for CP2, the acquisition priority is: B1->B0; for CP3, the acquisition priority is: A1->A0; for CP4, the MV of T is directly used as the MV of control point CP4.
[0469] The constructed MV is inserted only when the MVs of the current CU's control points (six-parameter model: CP0, CP1 and CP2; four-parameter model: CP0 and CP1) are available, otherwise jump directly to the next step. After obtaining the MVs of all control points (if any), the MVs of the control points are combined in different ways to obtain multiple affine candidates. The combination methods are as follows:
[0470] If the four-parameter affine model is used, one or more candidates can be obtained by combining two of the MVs of the four control points, and two combinations are selected: {CP1, CP2} and {CP1, CP3}. The combination {CP1, CP3} requires converting the MVs of the two selected control points into the MVs of the upper left and upper right control points of the current CU (CP1 and CP2) according to the four-parameter model.
[0471] If the six-parameter affine model is used, one or more candidates can be obtained by combining three of the MVs of the four control points, and four combinations are selected: {CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}. Among them, the combination methods {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} need to convert the MVs of the three selected control points into the MVs of the upper left corner, upper right corner, and lower left corner control points of the current CU (CP1, CP2, and CP3) according to the six-parameter model.
[0472] In one example, if the reference frames used by different combinations of MVs (2 or 3) are different, the candidate generated by the combination is considered unusable.
[0473] In one implementation, the method for determining the candidates to be added to the first candidate list of motion vectors includes: using a default vector for filling. Optionally, the default vector may be a zero vector or other vector. Optionally, after determining the candidates to be added to the first candidate list of motion vectors by other methods, it is determined whether the number of candidates currently added to the first candidate list has reached a preset value; if not, the default vector is used to fill the first candidate list until the number of candidates in the first candidate list reaches the preset value.
[0474] When the candidate in the first candidate list of motion vectors is used to predict the current image block, if the candidate used is at least one candidate other than the candidate determined by the ATMVP technology, the motion vector of the sub-image block in the current image block is derived according to the candidate through the affine motion model. When the candidate used is the candidate determined by the ATMVP technology, according to the above description, the reference block of each sub-image block in the current image block is determined according to the motion vector of each sub-image block in the relevant block, the reference blocks of each sub-image block are spliced into the reference block of the current image block, and the residual of the current image block is calculated according to the reference block.
[0475] Combine the following Fig.14 and Fig.15 A video image processing method provided in an embodiment of the present application is described by way of example. Fig.14 As shown, the method includes the following steps.
[0476] S1410 , sequentially scan N neighboring blocks among the preset M neighboring blocks of the current image block, and determine a target neighboring block according to the scanning result, where N is less than M. Optionally, M is less than or equal to four.
[0477] S1420: Determine a related block of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block.
[0478] S1430: Divide the current image block and the related block into a plurality of sub-image blocks in the same manner, wherein each sub-image block in the current image block corresponds to each sub-image block in the related block in a one-to-one manner.
[0479] S1440: Predict the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.
[0480] right Fig.14 The explanation of the video image processing method shown can be found above and will not be repeated here.
[0481] like Fig.15 As shown, the method includes the following steps.
[0482] S1510 , determining M neighboring blocks of the current image block according to M candidates in a second candidate list of motion vectors of the current image block.
[0483] S1520 , sequentially scan N neighboring blocks among the M neighboring blocks, and determine a target neighboring block according to the scanning result, where N is less than M. Optionally, M is less than or equal to 4.
[0484] S1530: Determine a related block of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block.
[0485] S1540: Determine a specific candidate in a first candidate list of motion vectors of the current image block according to a related block of the current image block, wherein the specific candidate may be the candidate determined according to the ATMVP technology mentioned above.
[0486] S1550, when it is determined to adopt the specific candidate, the current image block and the related block are divided into a plurality of sub-image blocks in the same manner, and each sub-image block in the current image block corresponds to each sub-image block in the related block one by one.
[0487] S1560: Predict the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.
[0488] right Fig.15 The explanation of the video image processing method shown can be found above and will not be repeated here.
[0489] Fig.16 1 is a schematic block diagram of a video image processing device 1600 provided in an embodiment of the present application. The device 1600 is used to perform the following Fig.14 The method embodiment shown in FIG. 1600 includes the following units.
[0490] The construction module 1610 sequentially scans N neighboring blocks among the preset M neighboring blocks of the current image block, determines a target neighboring block according to the scanning result, where N is less than M; determines a related block of the current image block according to a motion vector of the target neighboring block, the current image block, and a reference image of the current image block; divides the current image block and the related block into a plurality of sub-image blocks in the same manner, where each sub-image block in the current image block corresponds to each sub-image block in the related block one by one;
[0491] The prediction module 1620 predicts the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.
[0492] In one example, N is equal to 1 or 2.
[0493] In one example, the prediction module is further used to: before predicting the corresponding sub-image blocks in the current image block according to the motion vectors of each sub-image block in the related block, add the representative motion vector of the related block as a candidate to a first candidate list of motion vectors;
[0494] When it is determined to adopt the candidate, the prediction module predicts the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the relevant block.
[0495] In one example, predicting the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related block respectively includes:
[0496] The motion vectors of the sub-image blocks in the related blocks are respectively used as the motion vectors of the corresponding sub-image blocks in the current image block.
[0497] In one example, the representative motion vector of the relevant block is added as the first candidate to the first candidate list of motion vectors.
[0498] In one example, the representative motion vector of the relevant block includes a motion vector at a center position of the relevant block.
[0499] In one example, the prediction module is further used to: when a sub-image block with an unavailable motion vector appears in the related block, use the representative motion vector of the related block as the motion vector of the sub-image block with an unavailable motion vector, and predict the corresponding sub-image block in the current image block.
[0500] In one example, the prediction module is further used to: when a sub-image block with an unavailable motion vector appears in the related block and the representative motion vector of the related block is unavailable, abandon the prediction of the corresponding sub-image blocks in the current image block according to the motion vectors of each sub-image block in the related block.
[0501] In one example, the prediction module is further used to: when the sub-image block in the related block is not available, or the sub-image block in the related block adopts intra-frame coding mode, determine the sub-image block in the related block where the motion vector is not available.
[0502] In one example, the construction module is further used to: determine other candidates, and add the other candidates to the first candidate list of motion vectors, wherein at least one of the other candidates includes a motion vector of a sub-image block.
[0503] In one example, the construction module is further used to: when it is determined to adopt one of the other candidates, determine the motion vector of the sub-image block in the current image block according to the adopted candidate.
[0504] In one example, the at least one candidate includes motion vectors of a set of control points.
[0505] In one example, the prediction module is also used to:
[0506] When it is determined to adopt a candidate among the at least one candidate, performing an affine transformation on the adopted candidate according to an affine transformation model;
[0507] The sub-image block in the current image block is predicted according to the affine transformed candidate.
[0508] In one example, when the affine transformation model includes a four-parameter affine transformation model, each of the at least one candidate includes motion vectors of two control points;
[0509] When the affine transformation model includes a six-parameter affine transformation model, each of the at least one candidate includes motion vectors of three control points.
[0510] In one example, the construction module is further used to: determine, from the neighboring blocks of the current image block, in a specific scanning order, a control point motion vector group of the neighboring blocks predicted using an affine transformation mode;
[0511] Each determined control point motion vector group of the neighboring block is added as a candidate to the first motion vector candidate list.
[0512] In one example, determining, from the neighboring blocks of the current image block in a specific scanning order, a control point motion vector group of the neighboring blocks predicted using an affine transformation mode includes:
[0513] Determine a control point motion vector group of a first neighboring block in a left neighboring block of the current image block according to a first scanning order;
[0514] Determine a control point motion vector group of a second adjacent block in an upper adjacent block of the current image block according to a second scanning order;
[0515] The control point motion vector group of the first neighboring block and the control point motion vector group of the second neighboring block are added to the first motion vector candidate list.
[0516] In one example, the construction module is further used to: construct motion vectors of some control points of the current image block according to neighboring blocks of the some control points;
[0517] The motion vectors of some control points of the current image block are added to the first motion vector candidate list.
[0518] In one example, constructing the motion vectors of some control points according to neighboring blocks of some control points of the current image block includes:
[0519] For each control point in the partial control points, specific neighboring blocks of the control point are scanned in sequence according to a third scanning order, and a motion vector of the specific neighboring block that meets a preset condition is used as the motion vector of the control point.
[0520] In one example, the building blocks are also used to:
[0521] When the motion vectors of some control points point to different reference frames respectively, adding the motion vectors of some control points of the current image block into the first motion vector candidate list is abandoned.
[0522] In one example, when the number of candidates in the first motion vector candidate list is greater than a preset value, adding the motion vectors of some control points of the current image block to the first motion vector candidate list is abandoned.
[0523] In one example, the building blocks are also used to:
[0524] Constructing a second candidate list of motion vectors, wherein the candidate added to the second candidate list of motion vectors is a motion vector of an image block;
[0525] When it is confirmed to adopt a candidate in the second candidate list of motion vectors, the motion vector of the current image block is determined according to the motion vector of the candidate.
[0526] In one example, determining the motion vector of the current image block according to the motion vector of the candidate includes:
[0527] The confirmed candidate is used as the motion vector of the current image block, or the confirmed candidate is scaled and used as the motion vector of the current image block.
[0528] In one example, constructing the second motion vector candidate list includes:
[0529] The candidate to be added to the second candidate list of motion vectors is determined according to the motion vectors of a plurality of neighboring blocks of the current image block on the current image.
[0530] In one example, the plurality of neighboring blocks of the current image block on the current image include the preset M neighboring blocks.
[0531] In one example, the building blocks are also used to:
[0532] Sequentially taking the preset motion vectors of the M neighboring blocks as M candidates in a preset order, and adding them into the second motion vector candidate list;
[0533] The N adjacent blocks refer to the N adjacent blocks that are first determined according to the preset order.
[0534] In one example, the building blocks are also used to:
[0535] When the motion vectors of one or more neighboring blocks among the M neighboring blocks are not available, the candidates to be added to the second motion vector candidate list determined according to the motion vectors of the one or more neighboring blocks are abandoned.
[0536] In one example, the step of sequentially scanning N neighboring blocks among the M neighboring blocks and determining a target neighboring block according to the scanning result includes:
[0537] The N adjacent blocks are scanned in sequence, and when a first adjacent block meeting a preset condition is scanned, the scanning is stopped, and a target adjacent block is determined according to the first adjacent block meeting the preset condition.
[0538] In one example, determining the target adjacent block according to the scanned first adjacent block that meets the preset condition includes:
[0539] The first neighboring block that meets the preset condition is used as the target neighboring block.
[0540] In one example, the preset conditions include:
[0541] The reference image of the neighboring block is the same as the reference image of the current image block.
[0542] In one example, the construction module is also used to scale the motion vector of a specific neighboring block among the M neighboring blocks when no neighboring block that meets the preset condition is scanned among the N neighboring blocks, and the prediction module is also used to predict the current image block based on the scaled motion vector.
[0543] In one example, predicting the current image block according to the scaled motion vector includes:
[0544] A reference block of the current image block is determined according to the scaled motion vector and the reference image of the current image block.
[0545] In one example, the specific neighboring block is the first neighboring block or the last neighboring block obtained in a scanning order among the N neighboring blocks.
[0546] In one example, scaling the motion vector of a specific neighboring block among the M neighboring blocks, and predicting the current image block according to the scaled motion vector includes:
[0547] Scaling the motion vector of the specific neighboring block so that the reference frame pointed to by the motion vector after the scaling process is the same as the reference image of the current image block;
[0548] The image block pointed to by the motion vector after the scaling process in the reference image of the current image block is used as the reference block of the current image block.
[0549] In one example, when no adjacent block meeting the preset condition is scanned among the N adjacent blocks, a default block is used as a reference block of the current image block.
[0550] In one example, the default block is an image block pointed to by a motion vector (0, 0).
[0551] In one example, the size of the sub-image block and / or the size of a block associated with the sub-image block is fixed to be greater than or equal to 64 pixels.
[0552] In one example, the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to 8×8 pixels, or 16×4 pixels, or 4×16 pixels.
[0553] In one example, when the size of the sub-image block in the ATMVP technology and / or the size of the related blocks of the sub-image block is 8×8 pixels or at least one of its width and height is less than 8 pixels, setting not to perform the TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0554] In an example, the current image block is a coding unit CU.
[0555] In one example, determining the related blocks of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block includes:
[0556] According to the motion vector of the target neighboring block, a related block of the current image block is determined in a reference image of the current image block.
[0557] In one example, the neighboring block is an image block that is adjacent to the current image block or has a certain positional distance from the current image block on the current image.
[0558] In one example, predicting the current image block according to the motion vector of the related block includes:
[0559] When the reference image of the relevant block is a specific reference image, or the reference image of the current image block is a specific reference image, determining a reference block of the current image block according to the processed motion vector of the relevant block and the reference image of the current image block;
[0560] The motion vector of the related block after the processing is the same as the motion vector of the related block before the processing.
[0561] In one example, the processed motion vector of the relevant block includes:
[0562] A motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or
[0563] The motion vector of the relevant block skips the scaling step.
[0564] In one example, predicting the current image block according to the motion vector of the related block includes:
[0565] When the reference image of the related block is a specific reference image, or the reference image of the current block is a specific reference image, determining the reference block of the current image block according to the motion vector of the related block is abandoned.
[0566] In one example, the building blocks are also used to:
[0567] When the motion vector of the specific neighboring block points to a specific reference image, or the reference image of the current image block is a specific reference image, determining the reference block of the current image block according to the processed motion vector of the related block and the reference image of the current image block;
[0568] The motion vector of the related block after the processing is the same as the motion vector of the related block before the processing.
[0569] In one example, the processed motion vector of the relevant block includes:
[0570] A motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or
[0571] The motion vector of the relevant block skips the scaling step.
[0572] In one example, M is less than or equal to 4.
[0573] Fig.17 1 is a schematic block diagram of a video image processing device 1700 provided in an embodiment of the present application. The device 1700 is used to perform the following steps: Fig.15 The method embodiment shown in FIG. 1700 includes the following units.
[0574] A construction module 1710 is configured to determine M neighboring blocks of the current image block according to M candidates in a second candidate list of motion vectors of the current image block; sequentially scan N neighboring blocks among the M neighboring blocks, and determine a target neighboring block according to the scanning result, where N is less than M; determine a related block of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block; determine a specific candidate in a first candidate list of motion vectors of the current image block according to the related blocks of the current image block; when it is determined to adopt the specific candidate, divide the current image block and the related block into a plurality of sub-image blocks in the same manner, and each sub-image block in the current image block corresponds one-to-one to each sub-image block in the related block;
[0575] The prediction module 1720 is used to predict the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related blocks.
[0576] In one example, at least one candidate in the first candidate list of motion vectors includes a motion vector of a sub-image block, and each candidate in the second candidate list of motion vectors includes a motion vector of an image block.
[0577] In one example, N is equal to 1 or 2.
[0578] In one example, the M candidates include motion vectors of M neighboring blocks of the current image block on the current image.
[0579] In one example, the step of sequentially scanning N neighboring blocks among the M neighboring blocks and determining a target neighboring block according to the scanning result includes:
[0580] The N adjacent blocks are scanned in sequence, and when a first adjacent block meeting a preset condition is scanned, the scanning is stopped, and a target adjacent block is determined according to the first adjacent block meeting the preset condition.
[0581] In one example, determining the target adjacent block according to the scanned first adjacent block that meets the preset condition includes:
[0582] The first neighboring block that meets the preset condition is used as the target neighboring block.
[0583] In one example, the preset conditions include:
[0584] The reference image of the neighboring block is the same as the reference image of the current image block.
[0585] In one example, the construction module is also used to scale the motion vector of a specific neighboring block among the M neighboring blocks when no neighboring block that meets the preset condition is scanned among the N neighboring blocks, and the prediction module is also used to predict the current image block based on the scaled motion vector.
[0586] In one example, predicting the current image block according to the scaled motion vector includes:
[0587] A reference block of the current image block is determined according to the scaled motion vector and the reference image of the current image block.
[0588] In one example, the specific neighboring block is the first neighboring block or the last neighboring block obtained in a scanning order among the N neighboring blocks.
[0589] In one example, scaling the motion vector of a specific neighboring block among the M neighboring blocks, and predicting the current image block according to the scaled motion vector includes:
[0590] Scaling the motion vector of the specific neighboring block so that the reference frame pointed to by the motion vector after the scaling process is the same as the reference image of the current image block;
[0591] The image block pointed to by the motion vector after the scaling process in the reference image of the current image block is used as the reference block of the current image block.
[0592] In one example, when no adjacent block meeting the preset condition is scanned among the N adjacent blocks, a default block is used as a reference block of the current image block.
[0593] In one example, the default block is an image block pointed to by a motion vector (0, 0).
[0594] In one example, the size of the sub-image block and / or the size of a block associated with the sub-image block is fixed to be greater than or equal to 64 pixels.
[0595] In one example, the size of the sub-image block and / or the size of the block associated with the sub-image block is fixed to 8×8 pixels, or 16×4 pixels, or 4×16 pixels.
[0596] In one example, when the size of the sub-image block in the ATMVP technology and / or the size of the related blocks of the sub-image block is 8×8 pixels or at least one of its width and height is less than 8 pixels, setting not to perform the TMVP operation can skip some redundant operations, effectively saving encoding and decoding time and improving encoding efficiency.
[0597] In an example, the current image block is a coding unit CU.
[0598] In one example, determining the related blocks of the current image block according to the motion vector of the target neighboring block, the current image block, and a reference image of the current image block includes:
[0599] According to the motion vector of the target neighboring block, a related block of the current image block is determined in a reference image of the current image block.
[0600] In one example, the neighboring block is an image block that is adjacent to the current image block or has a certain positional distance from the current image block on the current image.
[0601] In one example, predicting the current image block according to the motion vector of the related block includes:
[0602] When the reference image of the relevant block is a specific reference image, or the reference image of the current image block is a specific reference image, determining a reference block of the current image block according to the processed motion vector of the relevant block and the reference image of the current image block;
[0603] The motion vector of the related block after the processing is the same as the motion vector of the related block before the processing.
[0604] In one example, the processed motion vector of the relevant block includes:
[0605] A motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or
[0606] The motion vector of the relevant block skips the scaling step.
[0607] In one example, predicting the current image block according to the motion vector of the related block includes:
[0608] When the reference image of the related block is a specific reference image, or the reference image of the current block is a specific reference image, determining the reference block of the current image block according to the motion vector of the related block is abandoned.
[0609] In one example, the construction module is further used to: when the motion vector of the specific neighboring block points to a specific reference image, or the reference image of the current image block is a specific reference image, determine the reference block of the current image block according to the processed motion vector of the related block and the reference image of the current image block;
[0610] The motion vector of the related block after the processing is the same as the motion vector of the related block before the processing.
[0611] In one example, the processed motion vector of the relevant block includes:
[0612] A motion vector obtained by scaling the motion vector of the relevant block according to a scaling ratio of 1, or
[0613] The motion vector of the relevant block skips the scaling step.
[0614] In one example, predicting the corresponding sub-image blocks in the current image block according to the motion vectors of the sub-image blocks in the related block respectively includes:
[0615] The motion vectors of the sub-image blocks in the related blocks are respectively used as the motion vectors of the corresponding sub-image blocks in the current image block.
[0616] In one example, determining a specific candidate in a first candidate list of motion vectors of the current image block according to a related block of the current image block includes:
[0617] The representative motion vector of the block related to the current image block is added to the first motion vector candidate list as the specific candidate.
[0618] In one example, the representative motion vector of the relevant block is added as the first candidate to the first candidate list of motion vectors.
[0619] In one example, the representative motion vector of the relevant block includes a motion vector at a center position of the relevant block.
[0620] In one example, the prediction module is further used to: when a sub-image block with an unavailable motion vector appears in the related block, use the representative motion vector of the related block as the motion vector of the sub-image block with an unavailable motion vector, and predict the corresponding sub-image block in the current image block.
[0621] In one example, the prediction module is further used to: when a sub-image block with an unavailable motion vector appears in the related block and the representative motion vector of the related block is unavailable, abandon the prediction of the corresponding sub-image blocks in the current image block according to the motion vectors of each sub-image block in the related block.
[0622] In one example, the prediction module is further used to: when the sub-image block in the related block is not available, or the sub-image block in the related block adopts intra-frame coding mode, determine the sub-image block in the related block where the motion vector is not available.
[0623] In one example, the prediction module is further used to: when determining to adopt one of the candidates other than the specific candidate in the second candidate list of the motion vector, perform affine transformation on the adopted candidate according to an affine transformation model;
[0624] The sub-image block in the current image block is predicted according to the affine transformed candidate.
[0625] In one example, in at least one candidate other than the specific candidate in the second candidate list of motion vectors, each candidate includes a motion vector of a group of control points.
[0626] In one example, when the affine transformation model includes a four-parameter affine transformation model, each of the at least one candidate includes motion vectors of two control points;
[0627] When the affine transformation model includes a six-parameter affine transformation model, each of the at least one candidate includes motion vectors of three control points.
[0628] In one example, the prediction module is further used to: determine, from the neighboring blocks of the current image block, in a specific scanning order, a control point motion vector group of the neighboring blocks predicted using an affine transformation mode;
[0629] Each determined control point motion vector group of the neighboring block is added as a candidate to the first motion vector candidate list.
[0630] In one example, determining, from the neighboring blocks of the current image block in a specific scanning order, a control point motion vector group of the neighboring blocks predicted using an affine transformation mode includes:
[0631] Determine a control point motion vector group of a first neighboring block in a left neighboring block of the current image block according to a first scanning order;
[0632] Determine a control point motion vector group of a second adjacent block in an upper adjacent block of the current image block according to a second scanning order;
[0633] The control point motion vector group of the first neighboring block and the control point motion vector group of the second neighboring block are added to the first motion vector candidate list.
[0634] In one example, the construction module is further used to: construct motion vectors of some control points of the current image block according to neighboring blocks of the some control points;
[0635] The motion vectors of some control points of the current image block are added to the first motion vector candidate list.
[0636] In one example, constructing the motion vectors of some control points according to neighboring blocks of some control points of the current image block includes:
[0637] For each control point in the partial control points, specific neighboring blocks of the control point are scanned in sequence according to a third scanning order, and a motion vector of the specific neighboring block that meets a preset condition is used as the motion vector of the control point.
[0638] In one example, the construction module is further used for: when the motion vectors of some control points point to different reference frames respectively, abandoning adding the motion vectors of some control points of the current image block into the first motion vector candidate list.
[0639] In one example, the construction module is further used to: when the number of candidates in the first motion vector candidate list is greater than a preset value, abandon adding the motion vectors of some control points of the current image block to the first motion vector candidate list.
[0640] In one example, the construction module is further used to: construct a second candidate list of motion vectors, wherein the candidate added to the second candidate list of motion vectors is a motion vector of an image block;
[0641] When it is confirmed to adopt a candidate in the second candidate list of motion vectors, the motion vector of the current image block is determined according to the motion vector of the candidate.
[0642] In one example, determining the motion vector of the current image block according to the motion vector of the candidate includes:
[0643] The confirmed candidate is used as the motion vector of the current image block, or the confirmed candidate is scaled and used as the motion vector of the current image block.
[0644] In one example, constructing the second motion vector candidate list includes:
[0645] The M candidates added to the second candidate list of motion vectors are determined according to the motion vectors of the M neighboring blocks of the current image block on the current image.
[0646] In one example, the construction module is further used to: sequentially take the motion vectors of the preset M neighboring blocks as M candidates in a preset order, and add them into the second motion vector candidate list;
[0647] The N adjacent blocks refer to the N adjacent blocks that are first determined according to the preset order.
[0648] In one example, the construction module is further used to: when the motion vectors of one or more neighboring blocks among the M neighboring blocks are not available, abandon the candidates to be added to the second candidate list of motion vectors determined according to the motion vectors of the one or more neighboring blocks.
[0649] In one example, M is less than or equal to 4.
[0650] like Fig.18 As shown, the embodiment of the present application also provides a video image processing method 1800, which includes the following steps:
[0651] S1810, determining a basic motion vector list, the basic motion vector list including at least one bi-prediction basic motion vector group, the bi-prediction basic motion vector group including a first basic motion vector and a second basic motion vector;
[0652] S1820, determining two motion vector offsets from a preset offset set, the two motion vector offsets respectively corresponding to the first basic motion vector and the second basic motion vector;
[0653] S1830, determining a motion vector of a current image block according to the first basic motion vector, the second basic motion vector, and the two motion vector offsets;
[0654] S1840: Predict the current image block according to the motion vector of the current image block.
[0655] The video image processing method of the embodiment of the present application offsets the basic motion vector in the dual-prediction basic motion vector group based on a preset offset set, and can obtain a more accurate motion vector of the current image block through a limited number of calculations, so that the predicted residual is smaller, thereby improving the encoding efficiency.
[0656] In some embodiments, the video image processing method of the embodiment of the present application can be used to improve the Merge with Motion Vector Difference (MMVD) technology, also known as the Ultimate motion vector expression (UMVE) technology. In particular, the MMVD technology is used to construct a merge candidate list, also known as a motion vector candidate list.
[0657] In some embodiments, before S1810, the video image processing method 1800 may further include the following steps: obtaining a merge candidate list, the merge candidate list including P groups of merged motion vector candidates, where P is an integer greater than or equal to 1. S1810 determines the basic motion vector list, which may include: determining the basic motion vector list according to the merge candidate list. For example, when P is greater than or equal to 2, two groups of merged motion vector candidates in the merge candidate list are taken to form the basic motion vector list. Optionally, the two groups of merged motion vector candidates may be the first two groups of merged motion vector candidates in the merge candidate list, or may be the other two groups of merged motion vector candidates when the first two groups of merged motion vector candidates do not meet the conditions. Alternatively, the two groups of merged motion vector candidates may be any two groups of merged motion vector candidates that meet the conditions in the merge candidate list, and the embodiments of the present application are not limited to this. For example, when P is less than 2, the basic motion vector list is filled with motion vector (0, 0).
[0658] Specifically, the MMVD technology may first construct a base motion vector list (base MVP list) using the merged motion vector candidates in the existing merge candidate list (or the various motion vector candidate lists obtained in the different types of modes described above). For example, the merged motion vector candidates in the existing merge candidate list are traversed. If the number of groups of merged motion vector candidates in the existing merge list is greater than 2, the first 2 groups of merged motion vector candidates in the merge list are taken to form the base MVP list of MMVD; otherwise, MV (0, 0) is used to fill in the base MVP list of MMVD. In the embodiment of the present application, other default motion vectors may also be used to fill in the base motion vector list, such as (1, 1), (2, 2), etc., which is not limited in the embodiment of the present application.
[0659] It should be understood that each of the two groups of basic MVs in the base MVPlist can be a unidirectionally predicted basic motion vector or a dual-prediction basic motion vector group. Of course, the base MVP list can include more or fewer groups of basic MVs, and this application does not limit this. This article only discusses the case of a dual-prediction basic motion vector group. The two basic motion vectors included in the dual-prediction basic motion vector group can be a forward basic motion vector and a backward basic motion vector; they can also be two basic motion vectors with the same direction, such as two forward basic motion vectors or two backward basic motion vectors.
[0660] Optionally, in some embodiments, S1820 determines two motion vector offsets from a preset offset set, which may include: determining multiple groups of motion vector offset combinations including two motion vector offsets from the offset set; S1830 determines the motion vector of the current image block according to the first basic motion vector, the second basic motion vector and the two motion vector offsets, which may include: determining a motion vector offset combination that makes the rate-distortion loss meet a preset condition from the multiple groups of motion vector offset combinations, and determining the motion vector of the current image block according to the first basic motion vector, the second basic motion vector and the motion vector offset combination that makes the rate-distortion loss meet the preset condition. Optionally, the rate-distortion loss meeting the preset condition may be that the rate-distortion loss is less than a preset threshold, or the rate-distortion loss is minimized, such as the smallest prediction residual, etc., which is not limited in the embodiments of the present application.
[0661] In the embodiment of the present application, the MMVD technology can offset the basic motion vector prediction value according to certain rules, generate a new motion vector prediction candidate as the MMVD motion vector prediction value, and put it into the MMVD motion vector candidate list. Optionally, in some embodiments, the motion vector offset (offset) in the offset set can have 8 options (2 1 , 2 2 , ..., 2 8 ), that is, the preset offset set is {2, 4, 8, 16, 32, 64, 128, 256}. For example, the number of base motion vectors (also called base motion vector candidates) in the base MVP list is 2, and the motion vector offset (offset) has 8 options (2 1 , 2 2 , ..., 2 8 ). For the two components MV_x and MV_y of a set of basic MVs in the base MVP list of MMVD, the motion vector offset can be added or subtracted (two options). For example, MMVD motion vector prediction value = base motion vector prediction value + motion vector offset offset. Therefore, the motion vector of the current image block of MMVD has a total of 2x8x2x2=64 refinement modes. The embodiment of the present application may also select a part of the 64 refinement modes, or derive more refinement modes from the 64 refinement modes, without being limited to 64 refinement modes. For example, the number of refinement modes may be 32 or 128, and so on, which is not limited to the embodiment of the present application. From these motion vectors, a motion vector that makes the rate-distortion loss meet the preset conditions is determined as the motion vector of the current image block for encoding and / or decoding.
[0662] From another perspective, determining the motion vector that makes the rate-distortion loss meet the preset conditions can also be equivalently considered as determining the motion vector offset combination that makes the rate-distortion loss meet the preset conditions from the multiple groups of motion vector offset combinations, and determining the motion vector of the current image block according to the first basic motion vector, the second basic motion vector and the motion vector offset combination that makes the rate-distortion loss meet the preset conditions.
[0663] It should be understood that the two motion vector offsets in a motion vector offset combination may be the same or different.
[0664] Optionally, the multiple groups of motion vector offset combinations including two motion vector offsets may be formed by traversing motion vector offsets in a preset offset set.
[0665] Alternatively, one of the motion vector offsets in the multiple sets of motion vector offset combinations may be a motion vector offset in the offset set calculated by a preset algorithm, and the calculated motion vector offset is used as a fixed value, and the motion vector offset in the traversal offset set is used as another motion vector offset in the multiple sets of motion vector offset combinations to form multiple sets of motion vector offset combinations. Instead of performing a scaling operation (scaling operation with a scaling ratio of 1), a simple traversal is performed to find a suitable motion vector offset combination, which can reduce the amount of calculation as a whole and improve encoding / decoding efficiency.
[0666] Alternatively, one of the motion vector offsets in the multiple motion vector offset combinations may be a motion vector offset in the offset set calculated by a preset algorithm. The calculated motion vector offset is taken as a fixed value, and another motion vector offset in the multiple motion vector offset combinations is obtained by scaling the fixed value to form multiple motion vector offset combinations.
[0667] Alternatively, multiple groups of motion vector offset combinations including two motion vector offsets may also be formed in other ways, which is not limited in the embodiments of the present application.
[0668] Optionally, in some embodiments, when both the first basic motion vector and the second basic motion vector point to a non-specific reference image, the selected motion vector offset can be scaled to obtain a new motion vector offset. The first basic motion vector and the second basic motion vector are then adjusted by the new motion vector offset. According to the first basic motion vector, the second basic motion vector and the two new motion vector offsets, that is, according to the adjusted motion vector of the current image block, the motion vector is determined; and the current image block is predicted according to the motion vector of the current image block. In other words, in dual prediction, when determining the motion vector prediction value of the current image block according to the MMVD technology, if the distance between the current image and the reference images of the two basic motion vectors is different, the motion vector offset needs to be scaled, and then the scaled motion vector offset is added (or subtracted) from the basic motion vector prediction to determine the motion vector of the current image block.
[0669] When both the first basic motion vector and the second basic motion vector point to a non-specific reference image, the two motion vector offsets are used to adjust the first basic motion vector and the second basic motion vector according to the two motion vector offsets. Optionally, the ratio of the distance from the current image to the reference image of the first basic motion vector to the distance from the current image to the reference image of the second basic motion vector is equal to the ratio of the motion vector offset used by the first basic motion vector to the motion vector offset used by the second basic motion vector. In other words, the scaling ratio is determined by the distance between the image where the current image block (e.g., the first image block) is located and the reference images of the current image block in two reference directions, and these two reference images are the reference images of the current basic motion vector. This implementation method can further reduce the number of motion vector offset combinations to be tried, and can further improve the encoding / decoding efficiency.
[0670] It should be understood that the adjustment may be to add the two motion vector offsets to the first base motion vector and the second base motion vector respectively; or the adjustment may be to subtract the two motion vector offsets from the first base motion vector and the second base motion vector respectively.
[0671] For example, in the encoding or decoding process, there are 8 optional motion vector offsets {2, 4, 8, 16, 32, 64, 128, 256}. Let the motion vector offset selected in the current process be X. Let the frame number of the current image be P2, and the frame number of the reference image of the first basic motion vector and the frame number of the reference image of the second basic motion vector be P0 and P1 respectively. If P2-P0=P2-P1, then add the same size of motion vector offset X to the two basic motion vectors; if P2-P0=2*(P2-P1), then add the motion vector offset 2*X to the first basic motion vector, and add the motion vector offset X to the second basic motion vector; similarly, if P2-P1=2*(P2-P0), then add the motion vector offset X to the first basic motion vector, and add the motion vector offset 2*X to the second basic motion vector.
[0672] In addition, if the ratio of the distance between the current image and the reference image of the first basic motion vector to the distance between the current image and the reference image of the second basic motion vector is not a multiple of 2, a suitable ratio (a multiple of 2) is selected so that the ratio of the motion vector offset used by the first basic motion vector to the motion vector offset used by the second basic motion vector is as close as possible to the ratio of the distance between the current image and the reference image of the first basic motion vector to the distance between the current image and the reference image of the second basic motion vector. For example, if P2-P0=3*(P2-P1), a ratio of 2 or 4 is selected. A motion vector offset of 2*X is added to the first basic motion vector, and a motion vector offset of X is added to the second basic motion vector; or, a motion vector offset of 4*X is added to the first basic motion vector, and a motion vector offset of X is added to the second basic motion vector. For another example, if P2-P0=5*(P2-P1), a ratio of 4, which is a multiple of 2 closest to 5, is selected. A motion vector offset of 4*X is added to the first basic motion vector, and a motion vector offset of X is added to the second basic motion vector.
[0673] When the first basic motion vector and / or the second basic motion vector points to a specific reference image (i.e., the reference image of the current image block is the specific reference image), since the temporal distance between the specific reference image and the image where the current image block is located is not clearly defined, scaling the motion vector offset is meaningless.
[0674] Optionally, in some embodiments, when the first basic motion vector and / or the second basic motion vector points to a specific reference image, at least one of the two motion vector offsets includes: a motion vector offset obtained by scaling the initial motion vector offset according to a scaling ratio of 1, or a motion vector offset obtained by skipping the scaling operation.
[0675] Or in other words, when the first basic motion vector and / or the second basic motion vector points to a specific reference image, the motion vector of the current image block is determined according to the processed motion vector offset and the basic motion vector group, wherein the processed motion vector offset is the same as the motion vector offset before processing. For example, at least one of the two processed motion vector offsets includes: a motion vector offset obtained by scaling the initial motion vector offset according to a scaling ratio of 1, or a motion vector offset obtained by skipping the scaling operation.
[0676] In a specific embodiment, one of the two motion vector offsets (for example, offset1) can be calculated by a preset algorithm. When the first basic motion vector points to a specific reference image, or the second basic motion vector points to a specific reference image, or the first basic motion vector and the second basic motion vector both point to a specific reference image, the other motion vector offset (for example, offset2) can be obtained by scaling offset1 with a scaling ratio of 1, or by not scaling offset1. Alternatively, the other motion vector offset (for example, offset2) can be obtained by scaling a certain initial offset with a scaling ratio of 1, or by not scaling. In other words, the initial motion vector offset can be offset1 or other initial offsets, and the embodiments of the present application are not limited to this.
[0677] In some embodiments of the present application, the method is performed by the encoding end, and the method further includes: encoding according to the prediction result and sending a code stream to the decoding end, wherein the code stream includes an index for indicating a combination of motion vector offsets that makes the rate distortion loss meet a preset condition. In this embodiment, the encoding end informs the decoding end of the index of the determined motion vector offset combination, so that the decoding end can know the two motion vector offsets through a small amount of calculation, which can simplify the decoding end. Optionally, when one of the two motion vector offsets (for example, offset1) can be calculated by a preset algorithm, the index can be the ratio of another motion vector offset (for example, offset2) to offset1.
[0678] In some embodiments of the present application, the method is performed by a decoding end, and the method further includes: receiving a code stream sent by an encoding end, the code stream including an index for indicating a combination of two motion vector offsets; the determining of two motion vector offsets from a preset offset set includes: determining the two motion vector offsets according to the index. In this embodiment, the encoding end informs the decoding end of the index of the determined motion vector offset combination, so that the decoding end can know the two motion vector offsets through a small amount of calculation, which can simplify the decoding end. Optionally, when one of the two motion vector offsets (for example, offset1) can be calculated by a preset algorithm, the index can be the ratio of another motion vector offset (for example, offset2) to offset1.
[0679] Combination of the above Fig.18 The method embodiment of the present application is described, and the device embodiment corresponding to the above method embodiment will be described below. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the content not described in detail can be referred to the above method embodiment, and for the sake of brevity, it will not be repeated here.
[0680] Fig.19 1 is a schematic block diagram of a video image processing device 1900 provided in an embodiment of the present application. The device 1900 is used to perform the following steps: Fig.18 The method embodiment shown. The apparatus 1900 includes the following modules.
[0681] A construction module 1910 is configured to determine a basic motion vector list, wherein the basic motion vector list includes at least one bi-prediction basic motion vector group, wherein the bi-prediction basic motion vector group includes a first basic motion vector and a second basic motion vector; determine two motion vector offsets from a preset offset set, wherein the two motion vector offsets correspond to the first basic motion vector and the second basic motion vector, respectively; and determine a motion vector of a current image block according to the first basic motion vector, the second basic motion vector, and the two motion vector offsets;
[0682] The prediction module 1920 is configured to predict the current image block according to the motion vector of the current image block.
[0683] The video image processing device of the embodiment of the present application offsets the basic motion vector in the dual-prediction basic motion vector group based on a preset offset set, and can obtain a more accurate motion vector of the current image block through a limited number of calculations, so that the predicted residual is smaller, thereby improving the encoding efficiency.
[0684] Optionally, in some embodiments, the construction module 1910 determines two motion vector offsets from a preset offset set, including: the construction module determines multiple groups of motion vector offset combinations including two motion vector offsets from the offset set; the construction module 1910 determines the motion vector of the current image block according to the first basic motion vector, the second basic motion vector and the two motion vector offsets, including: the construction module 1910 determines a motion vector offset combination from the multiple groups of motion vector offset combinations that makes the rate-distortion loss meet a preset condition, and determines the motion vector of the current image block according to the first basic motion vector, the second basic motion vector and the motion vector offset combination that makes the rate-distortion loss meet the preset condition.
[0685] Optionally, in some embodiments, the video image processing device 1900 is used at the encoding end, and the video image processing device 1900 also includes a sending module, which is used to: encode according to the prediction result and send a code stream to the decoding end, wherein the code stream includes an index for indicating a motion vector offset combination that makes the rate-distortion loss meet a preset condition.
[0686] Optionally, in some embodiments, the video image processing device 1900 is used at the decoding end, and the video image processing device 1900 also includes a receiving module, which is used to: receive a code stream sent by the encoding end, and the code stream includes an index for indicating a combination of two motion vector offsets; the construction module 1910 determines two motion vector offsets from a preset offset set, including: determining the two motion vector offsets according to the index.
[0687] Optionally, in some embodiments, when the first basic motion vector and / or the second basic motion vector points to a specific reference image, at least one of the two motion vector offsets includes: a motion vector offset obtained by scaling the initial motion vector offset according to a scaling ratio of 1, or a motion vector offset obtained by skipping the scaling operation.
[0688] Optionally, in some embodiments, when the first basic motion vector and the second basic motion vector both point to a non-specific reference image, the two motion vector offsets are used to: adjust the first basic motion vector and the second basic motion vector according to the two motion vector offsets.
[0689] Optionally, in some embodiments, the ratio of the distance from the current image to the reference image of the first basic motion vector to the distance from the current image to the reference image of the second basic motion vector is equal to the ratio of the motion vector offset used by the first basic motion vector to the motion vector offset used by the second basic motion vector.
[0690] Optionally, in some embodiments, the construction module 1910 is also used to: obtain a merge candidate list, the merge candidate list including P groups of merged motion vector candidates, where P is an integer greater than or equal to 1; the construction module 1910 determines a basic motion vector list, including: the construction module 1910 determines the basic motion vector list based on the merge candidate list.
[0691] Optionally, in some embodiments, the construction module 1910 determines the basic motion vector list based on the merged candidate list, including: when P is greater than or equal to 2, the construction module 1910 takes the first two groups of merged motion vector candidates in the merged candidate list to form the basic motion vector list.
[0692] Optionally, in some embodiments, the construction module 1910 determines the basic motion vector list according to the merge candidate list, including: when P is less than 2, the construction module 1910 fills the basic motion vector list with motion vector (0, 0).
[0693] Optionally, in some embodiments, the preset offset set is {2, 4, 8, 16, 32, 64, 128, 256}.
[0694] Optionally, in some embodiments, the current image block is a coding unit CU.
[0695] Optionally, in some embodiments, the current image block is a dual-prediction image block.
[0696] Fig. 20 is a schematic block diagram of a video image processing device 2000 provided in an embodiment of the present application. Fig. 20 The video image processing device 2000 shown may include a processor 2010 and a memory 2020, wherein the memory 2020 stores computer instructions, and when the processor 2010 executes the computer instructions, the video image processing device 2000 performs the following steps: determining a basic motion vector list, wherein the basic motion vector list includes at least one group of double-prediction basic motion vector groups, wherein the double-prediction basic motion vector group includes a first basic motion vector and a second basic motion vector; determining two motion vector offsets from a preset offset set, wherein the two motion vector offsets correspond to the first basic motion vector and the second basic motion vector, respectively; determining a motion vector of a current image block according to the first basic motion vector, the second basic motion vector and the two motion vector offsets; and predicting the current image block according to the motion vector of the current image block.
[0697] It should be understood that the video image processing apparatus 2000 of the embodiment of the present application may also include a network interface for transmitting a code stream, for example, receiving a code stream sent by an encoding device.
[0698] In some embodiments of the present application, the processor 2010 determines two motion vector offsets from a preset offset set, including: determining multiple groups of motion vector offset combinations including two motion vector offsets from the offset set; the processor 2010 determines the motion vector of the current image block according to the first basic motion vector, the second basic motion vector and the two motion vector offsets, including: determining a motion vector offset combination from the multiple groups of motion vector offset combinations that makes the rate-distortion loss meet a preset condition, and determining the motion vector of the current image block according to the first basic motion vector, the second basic motion vector and the motion vector offset combination that makes the rate-distortion loss meet the preset condition.
[0699] In some embodiments of the present application, the video image processing device 2000 is used at the encoding end, and the processor 2010 is further used to encode according to the prediction result and send a code stream to the decoding end, wherein the code stream includes an index for indicating a motion vector offset combination that makes the rate-distortion loss meet a preset condition.
[0700] In some embodiments of the present application, the video image processing device 2000 is used at the decoding end, and the processor 2010 is also used to receive a code stream sent by the encoding end, wherein the code stream includes an index for indicating a combination of two motion vector offsets; the processor 2010 determines two motion vector offsets from a preset offset set, including: determining the two motion vector offsets according to the index.
[0701] In some embodiments of the present application, when the first basic motion vector and / or the second basic motion vector points to a specific reference image, at least one of the two motion vector offsets includes: a motion vector offset obtained by scaling the initial motion vector offset according to a scaling ratio of 1, or a motion vector offset obtained by skipping the scaling operation.
[0702] In some embodiments of the present application, when the first basic motion vector and the second basic motion vector both point to a non-specific reference image, the two motion vector offsets are used to adjust the first basic motion vector and the second basic motion vector according to the two motion vector offsets.
[0703] In some embodiments of the present application, the ratio of the distance from the current image to the reference image of the first basic motion vector to the distance from the current image to the reference image of the second basic motion vector is equal to the ratio of the motion vector offset used by the first basic motion vector to the motion vector offset used by the second basic motion vector.
[0704] In some embodiments of the present application, the processor 2010 is also used to: obtain a merge candidate list, the merge candidate list including P groups of merged motion vector candidates, where P is an integer greater than or equal to 1; the processor 2010 determines a basic motion vector list, including: determining the basic motion vector list based on the merge candidate list.
[0705] In some embodiments of the present application, the processor 2010 determines the basic motion vector list based on the merge candidate list, including: when P is greater than or equal to 2, taking the first two groups of merged motion vector candidates in the merge candidate list to form the basic motion vector list.
[0706] In some embodiments of the present application, the processor 2010 determines the basic motion vector list according to the merge candidate list, including: when P is less than 2, filling the basic motion vector list with motion vector (0, 0).
[0707] In some embodiments of the present application, the preset offset set is {2, 4, 8, 16, 32, 64, 128, 256}.
[0708] In some embodiments of the present application, the current image block is a coding unit CU.
[0709] In some embodiments of the present application, the current image block is a dual-prediction image block.
[0710] It should be understood that Fig. 20 The video image processing device 2000 shown in FIG. Fig.19 The video image processing device 1900 shown can be used to execute the operations or processes in the above-mentioned method embodiments, and the operations and / or functions of the various modules and devices in the video image processing device 2000 or the video image processing device 1900 are respectively for realizing the corresponding processes in the above-mentioned method embodiments, which will not be repeated here for the sake of brevity.
[0711] The embodiment of the present application also provides a video image processing method, comprising: determining a basic motion vector list, wherein the basic motion vector list includes a basic motion vector group; when at least one basic motion vector in the basic motion vector group points to a specific reference image, abandoning determining the motion vector of the current image block according to the basic motion vector group and the motion vector offset. That is, when at least one basic motion vector in the basic motion vector group points to a specific reference image, abandoning putting the corresponding motion vector into the MV candidate list.
[0712] In some embodiments of the present application, a merge candidate list is obtained, the merge candidate list including P groups of merged motion vector candidates, where P is an integer greater than or equal to 1; the determining of the basic motion vector list includes: determining the basic motion vector list according to the merge candidate list.
[0713] In some embodiments of the present application, determining the basic motion vector list according to the merge candidate list includes: when P is greater than or equal to 2, taking the first two groups of merged motion vector candidates in the merge candidate list to form the basic motion vector list.
[0714] In some embodiments of the present application, determining the basic motion vector list according to the merge candidate list includes: when P is less than 2, filling the basic motion vector list with motion vector (0, 0).
[0715] In some embodiments of the present application, the current image block is a coding unit CU.
[0716] Correspondingly, the present application provides a video image processing device, including: a determination module, used to determine a basic motion vector list, the basic motion vector list including a basic motion vector group; a processing module, used to abandon determining the motion vector of the current image block based on the basic motion vector group and the motion vector offset when at least one basic motion vector in the basic motion vector group points to a specific reference image as a specific reference image.
[0717] In some embodiments of the present application, the video image processing device also includes a construction module, which is used to: obtain a merge candidate list, the merge candidate list includes P groups of merged motion vector candidates, where P is an integer greater than or equal to 1; the determination module is specifically used to: determine the basic motion vector list based on the merge candidate list.
[0718] In some embodiments of the present application, the determination module is specifically used to: when P is greater than or equal to 2, take the first two groups of merged motion vector candidates in the merge candidate list to form the basic motion vector list.
[0719] In some embodiments of the present application, the determination module is specifically used to: when P is less than 2, fill the basic motion vector list with motion vector (0, 0).
[0720] In some embodiments of the present application, the current image block is a coding unit CU.
[0721] The present application also provides a video image processing device including a processor and a memory, wherein the memory stores computer instructions, and when the processor executes the computer instructions, the video image processing device performs the following steps: determining a basic motion vector list, wherein the basic motion vector list includes a basic motion vector group; when at least one basic motion vector in the basic motion vector group points to a specific reference image, abandoning determination of the motion vector of the current image block based on the basic motion vector group and the motion vector offset.
[0722] It should be understood that the devices of each embodiment of the present application can be implemented based on a memory and a processor, each memory is used to store instructions for executing the methods of each embodiment of the present application, and the processor executes the above instructions so that the device executes the methods of each embodiment of the present application.
[0723] It should be understood that the processor mentioned in the embodiments of the present application may include a central processing unit (CPU), a network processor (NP) or a combination of a CPU and a NP. The processor may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.
[0724] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD). The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
[0725] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) is integrated in the processor.
[0726] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0727] The embodiment of the present application also provides a computer-readable storage medium on which instructions are stored. When the instructions are executed on a computer, the computer executes the steps of the above method embodiment.
[0728] An embodiment of the present application also provides a computing device, which includes the above-mentioned computer-readable storage medium.
[0729] An embodiment of the present application also provides a computer program product including instructions, characterized in that when a computer runs the instructions of the computer program product, the computer executes the steps of the above method embodiment.
[0730] The embodiment of the present application also provides a computer chip, which enables a computer to execute the steps of the above method embodiment.
[0731] The embodiments of the present application can be applied in aircraft, especially in the field of unmanned aerial vehicles.
[0732] It should be understood that the division of the circuits, sub-circuits, and sub-units in the various embodiments of the present application is only illustrative. A person skilled in the art can appreciate that the circuits, sub-circuits, and sub-units in the various examples described in the embodiments disclosed herein can be further split or combined.
[0733] The device provided by the embodiment of the present application can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, an SSD), etc.
[0734] It should be understood that the first, second and various numerical numbers involved in this document are only distinguished for the convenience of description and are not used to limit the scope of the present application.
[0735] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0736] It should be understood that in the embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0737] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0738] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0739] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0740] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0741] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0742] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A video image processing method, characterized in that: The method comprises: Determine a base motion vector list, wherein the base motion vector list includes at least one bi-prediction base motion vector group, wherein the bi-prediction base motion vector group includes a first base motion vector and a second base motion vector; Determine two motion vector offsets from a preset offset set, wherein the two motion vector offsets correspond to the first base motion vector and the second base motion vector respectively; Determine a motion vector of a current image block according to the first basic motion vector, the second basic motion vector and the two motion vector offsets; Predicting the current image block according to the motion vector of the current image block; and The current image block is encoded or decoded according to the prediction result.
2. The method according to claim 1, characterized in that The determining of two motion vector offsets from a preset offset set includes: A motion vector offset combination including the two motion vector offsets is determined from the preset offset set.
3. The method according to claim 1 or 2, characterized in that: The method is performed by the encoding end, and the method further includes: A code stream obtained by encoding according to the prediction result is output, wherein the code stream includes an index for indicating the two motion vector offsets.
4. The method according to claim 1, characterized in that: The method is performed by a decoding end, and the method further includes: An input bitstream includes an index for indicating the two motion vector offsets; The determining of two motion vector offsets from a preset offset set includes: The two motion vector offsets are determined based on the index.
5. The method according to claim 1, characterized in that When the first basic motion vector and / or the second basic motion vector points to a specific reference image, at least one of the two motion vector offsets includes: a motion vector offset obtained by scaling the initial motion vector offset according to a scaling ratio of 1, or a motion vector offset obtained by skipping the scaling operation.
6. The method according to claim 1, characterized in that When neither the first base motion vector nor the second base motion vector points to a specific reference image, at least one of the two motion vector offsets is scaled.
7. The method according to claim 6, characterized in that The ratio of the motion vector offset used by the first basic motion vector to the motion vector offset used by the second basic motion vector is determined based on the ratio of the distance from the current image to the reference image of the first basic motion vector to the distance from the current image to the reference image of the second basic motion vector.
8. The method according to claim 1, characterized in that The method further comprises: Obtain a merge candidate list, where the merge candidate list includes P groups of merged motion vector candidates, where P is an integer greater than or equal to 1; The determining of the basic motion vector list comprises: The basic motion vector list is determined according to the merge candidate list.
9. The method according to claim 8, characterized in that The step of determining the basic motion vector list according to the merge candidate list includes: When P is greater than or equal to 2, two groups of merged motion vector candidates in the merge candidate list are taken to form the basic motion vector list.
10. The method according to claim 8, characterized in that The determining the basic motion vector list according to the merge candidate list includes: When P is less than 2, the basic motion vector list is filled with motion vector (0, 0).
11. The method according to claim 1, characterized in that: The preset offset set includes {2, 4, 8, 16, 32, 64, 128, 256}.
12. The method according to claim 1, characterized in that The current image block is a coding unit CU.
13. The method according to claim 1, characterized in that The current image block is a dual-prediction image block.
14. The method according to claim 1, characterized in that The method is performed by the encoding end, and the method further includes: Output the code stream obtained by encoding according to the prediction result.
15. A video image processing device, characterized in that: include: A construction module is configured to determine a basic motion vector list, wherein the basic motion vector list includes at least one bi-prediction basic motion vector group, wherein the bi-prediction basic motion vector group includes a first basic motion vector and a second basic motion vector; determine two motion vector offsets from a preset offset set, wherein the two motion vector offsets correspond to the first basic motion vector and the second basic motion vector, respectively; and determine a motion vector of a current image block according to the first basic motion vector, the second basic motion vector, and the two motion vector offsets; A prediction module, configured to predict the current image block according to the motion vector of the current image block; The video image processing device encodes or decodes the current image block according to the prediction result of the prediction module.
16. The video image processing device according to claim 15, characterized in that: The construction module determines two motion vector offsets from a preset offset set, including: The building module determines from the offset set a motion vector offset combination comprising the two motion vector offsets.
17. The video image processing device according to claim 15 or 16, characterized in that: The video image processing device is used at the encoding end, and the video image processing device also includes an output module, which is used to: A code stream obtained by encoding according to the prediction result is output, wherein the code stream includes an index for indicating the two motion vector offsets.
18. The video image processing device according to claim 17, characterized in that: The video image processing device is used at a decoding end, and the video image processing device further comprises an input module, which is used to: An input bitstream includes an index for indicating the two motion vector offsets; The construction module determines two motion vector offsets from a preset offset set, including: The two motion vector offsets are determined based on the index.
19. The video image processing device according to claim 15, characterized in that: When the first basic motion vector and / or the second basic motion vector points to a specific reference image, at least one of the two motion vector offsets includes: a motion vector offset obtained by scaling the initial motion vector offset according to a scaling ratio of 1, or a motion vector offset obtained by skipping the scaling operation.
20. The video image processing device according to claim 15, characterized in that: When neither the first base motion vector nor the second base motion vector points to a non-specific reference image, at least one of the two motion vector offsets is scaled.
21. The video image processing device according to claim 20, characterized in that: The ratio of the motion vector offset used by the first basic motion vector to the motion vector offset used by the second basic motion vector is determined based on the ratio of the distance from the current image to the reference image of the first basic motion vector to the distance from the current image to the reference image of the second basic motion vector.
22. The video image processing device according to claim 15, characterized in that: The building blocks are also used to: Obtain a merge candidate list, where the merge candidate list includes P groups of merged motion vector candidates, where P is an integer greater than or equal to 1; The building module determines a basic motion vector list, including: The construction module determines the basic motion vector list according to the merge candidate list.
23. The video image processing device according to claim 22, characterized in that: The construction module determines the basic motion vector list according to the merge candidate list, including: When P is greater than or equal to 2, the construction module takes two groups of merged motion vector candidates in the merge candidate list to form the basic motion vector list.
24. The video image processing device according to claim 22, characterized in that: The construction module determines the basic motion vector list according to the merge candidate list, including: When P is less than 2, the construction module fills the basic motion vector list with motion vector (0, 0).
25. The video image processing device according to claim 15, characterized in that: The preset offset set includes {2, 4, 8, 16, 32, 64, 128, 256}.
26. The video image processing device according to claim 15, characterized in that: The current image block is a coding unit CU.
27. The video image processing device according to claim 15, characterized in that: The current image block is a dual-prediction image block.
28. A video image processing device, characterized in that: include: A memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, and the execution of the instructions stored in the memory enables the processor to execute the method according to any one of claims 1 to 14.
29. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the computer is caused to execute: An input bitstream includes an index for indicating two motion vector offsets; Determine a base motion vector list, wherein the base motion vector list includes at least one bi-prediction base motion vector group, wherein the bi-prediction base motion vector group includes a first base motion vector and a second base motion vector; Determine the two motion vector offsets from a preset offset set according to the index, the two motion vector offsets respectively corresponding to the first basic motion vector and the second basic motion vector; Determine a motion vector of a current image block according to the first basic motion vector, the second basic motion vector and the two motion vector offsets; as well as The current image block is predicted according to the motion vector of the current image block.
30. A video image processing method, characterized in that: The method is performed by a decoding end, and the method includes: An input bitstream includes an index for indicating two motion vector offsets; Determine a base motion vector list, wherein the base motion vector list includes at least one bi-prediction base motion vector group, wherein the bi-prediction base motion vector group includes a first base motion vector and a second base motion vector; Determine the two motion vector offsets from a preset offset set according to the index, the two motion vector offsets respectively corresponding to the first basic motion vector and the second basic motion vector; Determine a motion vector of a current image block according to the first basic motion vector, the second basic motion vector and the two motion vector offsets; Predicting the current image block according to the motion vector of the current image block; and The current image block is decoded according to the prediction result.
