METHOD FOR BUILDING MOTION INFORMATION CANDIDATE LIST, INTER PREDICTION METHOD AND APPARATUS - Patent application

Abstract

To provide a method for constructing a candidate motion information list, a prediction method, and an apparatus.SOLUTION: A construction method includes the steps of: performing a first processing process on all neighboring image blocks of a current image block until the quantity of candidate motion information sets in a candidate motion information list is equal to a first preset value or all the neighboring image blocks are scanned; scanning at least two extra to-be-selected motion information sets, and storing, in the candidate motion information list, an extra to-be-selected motion information set that satisfies a preset condition; and if the quantity of candidate motion information sets in the candidate motion information list is less than the preset quantity value after the extra to-be- selected motion information sets are scanned, filling the candidate motion information list with a first preset candidate motion information set until the quantity of candidate motion information sets in the candidate motion information list is equal to the preset quantity value.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] This application claims priority to Chinese Patent Application No. 201810990444.2, entitled "INTER PREDICTION METHOD AND APPARATUS, VIDEO ENCODER, AND VIDEO DECODER," filed on August 28, 2018, with the State Intellectual Property Office of the People's Republic of China, and to Chinese Patent Application No. 201810997139.6, entitled "METHOD FOR CONSTRUCTING CANDIDATE MOTION INFORMATION LIST, INTER PREDICTION METHOD, AND APPARATUS," filed on August 29, 2018, with the State Intellectual Property Office of the People's Republic of China, each of which is incorporated by reference in its entirety. FIELD OF THE DISCLOSURE Embodiments of the present application relate to the field of video image coding technology, and in particular to a method for constructing a motion information candidate list, an inter-prediction method and an apparatus. [Background technology]

[0002] In the non-translational motion model prediction mode, the encoder / decoder side derives the motion information of all sub-blocks of the current image block by using the same motion model based on the motion information of the control points of the current image block, and performs motion compensation based on the motion information of all sub-blocks to obtain a predicted image block. In general, the image block predicted in this manner is called an affine coding block.

[0003] Usually, the encoder / decoder side may obtain the motion information of the control point of the affine coding block by using the affine advanced motion vector prediction mode (Affine AMVP mode) or the affine merge mode (Affine Merge mode). Specifically, the encoder / decoder side builds a motion information candidate list and determines the motion information of the control point according to the motion information candidates in the list. In the process of building the motion information candidate list, before adding a new motion information candidate set to the motion information candidate list every time, the encoder / decoder side first checks whether the same motion information set already exists in the motion information candidate list. If the same motion information set exists in the motion information candidate list, the encoder / decoder side does not add the new motion information candidate set to the list.

[0004] This process is called pruning of motion information candidate list. Pruning of motion information candidate list can prevent the same motion information from being present in the motion information candidate list, and can avoid redundant rate-distortion cost calculation. However, the computational complexity of this process is also relatively high, and the delay is relatively long. Summary of the Invention [Means for solving the problem]

[0005] The embodiments of the present application provide a method for constructing a motion information candidate list, an inter-prediction method and an apparatus for solving the problems of relatively high computational complexity and relatively long delay.

[0006] In order to achieve the aforementioned objectives, the following technical solutions are used in the embodiments of this application.

[0007] According to a first aspect, a method for constructing a motion information candidate list is provided. An apparatus for constructing a motion information candidate list (hereinafter referred to as a constructing apparatus) sequentially executes a first processing process on all neighboring image blocks of a current image block in a first preset order until the amount of motion information candidate sets in the motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned. After the amount of motion information candidate sets in the motion information candidate list is equal to a first preset value or after all neighboring image blocks have been scanned, the constructing apparatus sequentially scans at least two additional selection target motion information sets in a second preset order, and stores the additional selection target motion information sets that meet a preset condition in the motion information candidate list until the amount of motion information candidate sets in the motion information candidate list is equal to a preset amount value or until all additional selection target motion information sets have been scanned. In this specification, the value of the preset amount is greater than the first preset value, and the additional selection target motion information set includes the additional selection target motion information set of at least two control points of the current image block, and the additional selection target motion information of the control point is the motion information of the inter-coded image block adjacent to the current image block. If the amount of the motion information candidate set in the motion information candidate list is less than the preset amount value after all the additional selection target motion information sets are scanned, the construction device fills the motion information candidate list with the first preset motion information candidate set until the amount of the motion information candidate set in the motion information candidate list is equal to the preset amount value. Specifically, the first processing process is as follows: For the i-th (i ∈ [1, n]) neighboring image block, when the i-th neighboring image block is an affine coding block, the construction device determines whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit.If the motion information candidate set from the coding unit to which the i-th neighboring image block belongs does not exist in the motion information candidate list, the construction device determines motion information candidates for x control points of the current image block based on the motion information of the x control points of the i-th neighboring image block, and stores the motion information candidate set including the motion information candidates for the x control points of the current image block in the motion information candidate list, where n is the amount of neighboring image blocks of the current image block, and both x and n are integers greater than or equal to 2.

[0008] It can be seen that in the first processing process, when the i-th neighboring image block is an affine coding block, the construction device only needs to determine whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit, and based on the result of the determination, determine whether to add a new motion information candidate set to the motion information candidate list, and does not need to calculate a new motion information candidate set and then determine whether the same motion information exists in the motion information candidate list. This substantially reduces the calculation complexity and delay.

[0009] Optionally, in a possible implementation form of the present application, the reference frame indexes of all control points in the additional selected motion information set that satisfies the preset condition are the same, or the reference frame indexes of all control points in the additional selected motion information set that satisfies the preset condition are the same and the additional selected motion vectors of at least two control points in the additional selected motion information set that satisfies the preset condition are different.

[0010] The constructing device only determines the reference frame indexes and the additional selection motion vectors of all the control points in the additional selection motion information set, and determines whether to store the additional selection motion information set in the motion information candidate list according to the determination result without performing motion vector scaling, which reduces the computation complexity.

[0011] Optionally, in another possible implementation form of the present application, the method used by the construction device to determine whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit is as follows: the construction device determines whether the information of the upper left corner of the coding unit to which the i-th neighboring image block belongs is the same as the information of the upper left corner of the target coding unit, and the target coding unit in this specification is the coding unit to which the neighboring image block used to calculate the motion information candidate set stored in the motion information candidate list belongs.

[0012] In the image block prediction process, the size of the coding unit is fixed.The construction device can directly determine whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit based on the information of the upper left corner of the coding unit.This determination process is simple and efficient.

[0013] Optionally, in another possible implementation form of the present application, the additional selected motion information set includes at least additional selected motion information of a first control point and additional selected motion information of a second control point, where the first control point and the second control point are neighboring control points of the current image block.

[0014] According to a second aspect, a method for constructing a motion information candidate list is provided. The constructing device sequentially executes a first processing process on all neighboring image blocks of a current image block in a first preset order until the amount of motion information candidate sets in the motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned. In addition, the constructing device further sequentially scans at least two additional selection-target motion information sets in a second preset order, and stores the additional selection-target motion information sets that satisfy a preset condition in the second motion information candidate list until the amount of motion information candidate sets in the second motion information candidate list is equal to a second preset value or until all additional selection-target motion information sets have been scanned. The additional selection-target motion information sets include additional selection-target motion information of at least two control points of the current image block, and the additional selection-target motion information of the control points is motion information of inter-coded image blocks adjacent to the current image block. If the amount of motion information candidate sets in the second motion information candidate list is less than the second preset value after all additional selection target motion information sets are scanned, the construction device fills the second motion information candidate list with the second preset motion information candidate set until the amount of motion information candidate sets in the second motion information candidate list is equal to the second preset value. Then, the construction device merges the first motion information candidate list and the second motion information candidate list according to a preset rule to obtain a third motion information candidate list. The amount of motion information candidate sets in the third motion information candidate list is equal to the preset amount value, and the sum of the first preset value and the second preset value is equal to or greater than the preset amount value.

[0015] Specifically, the first processing process is as follows: for the i-th (i ∈ [1, n]) neighboring image block, when the i-th neighboring image block is an affine coding block, the construction device determines whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit. If the motion information candidate set from the coding unit to which the i-th neighboring image block belongs does not exist in the motion information candidate list, the construction device determines motion information candidates of x control points of the current image block according to the motion information of x control points of the i-th neighboring image block, and stores the motion information candidate set including the motion information candidates of x control points of the current image block in the motion information candidate list, where n is the amount of neighboring image blocks of the current image block, and both x and n are integers equal to or greater than 2.

[0016] The reference frame indexes of all control points in the additional selection target motion information set that satisfies the predetermined condition are the same, or the reference frame indexes of all control points in the additional selection target motion information set that satisfies the predetermined condition are the same, and the additional selection target motion vectors of at least two control points in the additional selection target motion information set that satisfies the predetermined condition are different.

[0017] In both the first processing process and the processing process of additional selection target motion information set in the present application, whether to add a new motion information candidate set to the motion information candidate list is determined based on the result of the determination, and it is not necessary to first calculate a new motion information candidate set and then determine whether the same motion information exists in the motion information candidate list, which substantially reduces the calculation complexity and delay.

[0018] Optionally, in another possible implementation form of the present application, the additional selected motion information set includes at least additional selected motion information of a first control point and additional selected motion information of a second control point, where the first control point and the second control point are neighboring control points of the current image block.

[0019] Optionally, in another possible implementation form of the present application, the method used by the construction device to merge the first motion information candidate list and the second motion information candidate list according to preset rules to obtain a third motion information candidate list is as follows: To generate a target motion information candidate list, the construction device places all motion information candidate sets in the first motion information candidate list before all motion information candidate sets in the second motion information candidate list, and if the amount of motion information candidate sets in the target motion information candidate list is less than a predetermined amount value, fills the target motion information candidate list with the second predetermined motion information candidate set until the amount of motion information candidate sets in the target motion information candidate list is equal to the predetermined amount value, and uses the target motion information candidate list obtained after filling as a third motion information candidate list, and if the amount of motion information candidate sets in the target motion information candidate list is equal to the predetermined amount value, uses the target motion information candidate list as the third motion information candidate list, or if the amount of motion information candidate sets in the target motion information candidate list is greater than the predetermined amount value, deletes at least one motion information candidate from the target motion information candidate list so that the amount of motion information candidate sets in the target motion information candidate list obtained after the deletion is equal to the predetermined amount value, and uses the target motion information candidate list obtained after the deletion as the third motion information candidate list.

[0020] According to a third aspect, a method for constructing a motion information candidate list is provided. The construction device scans all neighboring image blocks of a current image block in a first preset order to generate a motion information candidate list including at least one motion information candidate set, the motion information candidate set including motion information candidates of x control points of the current image block, where x is an integer equal to or greater than 2. After scanning all neighboring image blocks of the current image block, the construction device sequentially scans at least two additional selection-target motion information sets in a second preset order, and stores the additional selection-target motion information sets that satisfy a preset condition in the motion information candidate list until the amount of the motion information candidate sets in the motion information candidate list is equal to a preset amount value or until all the additional selection-target motion information sets have been scanned. If the amount of the motion information candidate sets in the motion information candidate list is less than the preset amount value after all the additional selection-target motion information sets have been scanned, the construction device fills the motion information candidate list with a third preset motion information candidate set until the amount of the motion information candidate sets in the motion information candidate list is equal to a preset amount value. In this specification, the additional selected motion information set includes additional selected motion information of at least two control points of the current image block, and the additional selected motion information of the control points is motion information of an inter-coded image block adjacent to the current image block. The reference frame indexes of all control points in the additional selected motion information set that meets a preset condition are the same, or the reference frame indexes of all control points in the additional selected motion information set that meets a preset condition are the same, and the additional selected motion vectors of at least two control points in the additional selected motion information set that meets the preset condition are different.

[0021] It can be seen that when processing the additional selected motion information set, the construction device only determines the reference frame index and the additional selected motion vector of all control points in the additional selected motion information set, and determines whether to store the additional selected motion information set in the motion information candidate list according to the determination result. The construction device does not need to perform motion vector scaling, and does not need to first calculate a new motion information candidate set, and then determine whether the same motion information exists in the motion information candidate list. This reduces the calculation complexity and delay.

[0022] Optionally, in another possible implementation form of the present application, the additional selected motion information set includes at least additional selected motion information of a first control point and additional selected motion information of a second control point, where the first control point and the second control point are neighboring control points of the current image block.

[0023] According to a fourth aspect, an inter prediction method is provided. An inter prediction device obtains and parses a bitstream to obtain a first index value and a motion information difference, and the first index value in this specification is an index value of a motion information predictor of a control point of a current image block in a motion information candidate list, and the motion information candidate list is generated by the aforementioned construction device by using the construction method described in the first aspect, the second aspect, the third aspect, or any one of these possible implementations. Then, the inter prediction device obtains a motion information predictor of a control point from the motion information candidate list based on the first index value, and determines the motion information of the control point based on the motion information predictor of the control point and the motion information difference. In this manner, the inter prediction device can determine a prediction pixel of the current image block based on the motion information of the control point.

[0024] The inter prediction device performs inter prediction based on the motion information candidate list generated according to the construction method described in the first aspect, the second aspect, the third aspect, or any one of these possible implementations. In the process in which the construction device in the present application generates the motion information candidate list, the calculation complexity and delay are substantially reduced. Therefore, the inter prediction device performs inter prediction relatively quickly.

[0025] According to a fifth aspect, an inter-prediction method is provided. The inter-prediction device obtains and parses a bitstream to obtain a second index value, the second index value being an index value of the motion information of the control point of the current image block in the motion information candidate list, and the motion information candidate list is generated by the above-mentioned construction device by using the construction method described in the first aspect, the second aspect, the third aspect, or any one of these possible implementations.Then, the inter-prediction device obtains the motion information of the control point from the motion information candidate list according to the second index value, and determines the prediction pixel of the current image block according to the motion information of the control point.

[0026] The inter prediction device performs inter prediction based on the motion information candidate list generated according to the construction method described in the first aspect, the second aspect, the third aspect, or any one of these possible implementations. In the process in which the construction device in the present application generates the motion information candidate list, the calculation complexity and delay are substantially reduced. Therefore, the inter prediction device performs inter prediction relatively quickly.

[0027] According to a sixth aspect, an apparatus for constructing a motion information candidate list is provided. The apparatus for constructing a motion information candidate list includes a first processing unit, a second processing unit, a filling unit, and a storage unit.

[0028] Specifically, the first processing unit is configured to sequentially execute the first processing process on all neighboring image blocks of the current image block in a first preset order until the amount of the motion information candidate set in the motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned, and the motion information candidate list is stored in the storage unit. The first processing process is as follows: for the i-th neighboring image block, when the i-th neighboring image block is an affine coding block, determine whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit; if the motion information candidate set from the coding unit to which the i-th neighboring image block belongs does not exist in the motion information candidate list, determine motion information candidates for x control points of the current image block according to the motion information of x control points of the i-th neighboring image block, and store the motion information candidate set including the motion information candidates for x control points of the current image block in the motion information candidate list, where i ∈ [1, n], n is the amount of neighboring image blocks of the current image block, and x and n are both integers equal to or greater than 2. The second processing unit is configured to sequentially scan at least two additional selection-target motion information sets in a second predetermined order after an amount of the motion information candidate sets in the motion information candidate list stored in the storage unit is equal to a first preset value or after all neighboring image blocks have been scanned, and store the additional selection-target motion information sets that satisfy a preset condition in the motion information candidate list until an amount of the motion information candidate sets in the motion information candidate list is equal to a preset amount value or until all additional selection-target motion information sets have been scanned. The preset amount value is greater than the first preset value, and the additional selection-target motion information sets include additional selection-target motion information of at least two control points of the current image block, and the additional selection-target motion information of the control points is motion information of inter-coded image blocks adjacent to the current image block.The filling unit is configured to fill the motion information candidate list with the first preset motion information candidate set until the amount of the motion information candidate set in the motion information candidate list is equal to the preset amount value, if the amount of the motion information candidate set in the motion information candidate list is less than the preset amount value after the second processing unit scans all additional selection target motion information sets.

[0029] Optionally, in one possible implementation form of the present application, the reference frame indexes of all control points in the additional selected motion information set that satisfies the preset condition are the same, or the reference frame indexes of all control points in the additional selected motion information set that satisfies the preset condition are the same and the additional selected motion vectors of at least two control points in the additional selected motion information set that satisfies the preset condition are different.

[0030] Optionally, in another possible implementation form of the present application, the first processing unit is specifically configured to determine whether information of the upper left corner of the coding unit to which the i-th neighboring image block belongs is the same as information of the upper left corner of the target coding unit, the target coding unit being the coding unit to which the neighboring image block belongs that is used to calculate the motion information candidate set stored in the motion information candidate list.

[0031] Optionally, in another possible implementation form of the present application, the additional selected motion information set includes at least additional selected motion information of a first control point and additional selected motion information of a second control point, where the first control point and the second control point are neighboring control points of the current image block.

[0032] For a specific description of the sixth aspect or any one of its possible implementations in this application, please refer to the detailed description of the first aspect or any one of its possible implementations. In addition, for the beneficial effects of the sixth aspect or any one of its possible implementations, please refer to the analysis of the beneficial effects of the first aspect or any one of its possible implementations. Details will not be described again here.

[0033] According to a seventh aspect, an apparatus for constructing a motion information candidate list is provided. The apparatus for constructing a motion information candidate list includes a first processing unit, a second processing unit, a merging unit, and a storage unit.

[0034] Specifically, the first processing unit is configured to sequentially execute the first processing process on all neighboring image blocks of the current image block in a first preset order until the amount of the motion information candidate set in the first motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned, and the first motion information candidate list is stored in the storage unit. The first processing process is as follows: for the i-th neighboring image block, when the i-th neighboring image block is an affine coding block, determine whether the i-th neighboring image block and the motion information candidate set stored in the first motion information candidate list are from the same coding unit; if the motion information candidate set from the coding unit to which the i-th neighboring image block belongs does not exist in the first motion information candidate list, determine the motion information candidates of x control points of the current image block according to the motion information of x control points of the i-th neighboring image block, and store the motion information set including the motion information candidates of x control points of the current image block in the first motion information candidate list, where i ∈ [1, n], n is the amount of neighboring image blocks of the current image block, and x and n are both integers equal to or greater than 2. The second processing unit is configured to sequentially scan at least two additional selected motion information sets in a second predetermined order and store the additional selected motion information sets that satisfy the predetermined condition in the second motion information candidate list until an amount of motion information candidate sets in the second motion information candidate list is equal to a second predetermined value or until all the additional selected motion information sets have been scanned, the additional selected motion information sets include additional selected motion information of at least two control points of the current image block, the additional selected motion information of the control points is motion information of inter-coded image blocks adjacent to the current image block, and the second motion information candidate list is stored in the storage unit.The reference frame indexes of all control points in the additional selected motion information set that meets the preset condition are the same, or the reference frame indexes of all control points in the additional selected motion information set that meets the preset condition are the same, and the additional selected motion vectors of at least two control points in the additional selected motion information set that meets the preset condition are different. The merging unit is configured to merge the first motion information candidate list and the second motion information candidate list in the storage unit according to a preset rule to obtain a third motion information candidate list, where the amount of the motion information candidate sets in the third motion information candidate list is equal to a preset amount value, and the sum of the first preset value and the second preset value is equal to or greater than the preset amount value.

[0035] Optionally, in one possible implementation form of the present application, the additional selected motion information set includes at least additional selected motion information of a first control point and additional selected motion information of a second control point, where the first control point and the second control point are neighboring control points of the current image block.

[0036] Optionally, in another possible implementation form of the present application, the merging unit places all the motion information candidate sets in the first motion information candidate list before all the motion information candidate sets in the second motion information candidate list to generate a target motion information candidate list; if the amount of the motion information candidate sets in the target motion information candidate list is less than a preset amount value, the merging unit fills the target motion information candidate list with the second preset motion information candidate set until the amount of the motion information candidate sets in the target motion information candidate list is equal to the preset amount value; and the target motion information obtained after filling is set as a third motion information candidate list. The method is particularly configured to use a candidate list, and if the amount of motion information candidate sets in the target motion information candidate list is equal to a predetermined amount value, use the target motion information candidate list as a third motion information candidate list, or if the amount of motion information candidate sets in the target motion information candidate list is greater than the predetermined amount value, delete at least one motion information candidate from the target motion information candidate list so that the amount of motion information candidate sets in the target motion information candidate list obtained after the deletion is equal to the predetermined amount value, and use the target motion information candidate list obtained after the deletion as the third motion information candidate list.

[0037] Optionally, in another possible implementation form of the present application, the first processing unit is specifically configured to determine whether information of the upper left corner of the coding unit to which the i-th neighboring image block belongs is the same as information of the upper left corner of the target coding unit, the target coding unit being the coding unit to which the neighboring image block belongs that is used to calculate the motion information candidate set stored in the motion information candidate list.

[0038] For a specific description of the seventh aspect or any one of its possible implementations in this application, please refer to the detailed description of the second aspect or any one of its possible implementations. In addition, for the beneficial effects of the seventh aspect or any one of its possible implementations, please refer to the analysis of the beneficial effects of the second aspect or any one of its possible implementations. Details will not be described again here.

[0039] According to an eighth aspect, an apparatus for constructing a motion information candidate list is provided. The apparatus for constructing a motion information candidate list includes a first processing unit, a second processing unit, a filling unit, and a storage unit.

[0040] Specifically, the first processing unit is configured to scan all neighboring image blocks of the current image block in a first preset order to generate a motion information candidate list including at least one motion information candidate set, the motion information candidate set including motion information candidates of x control points of the current image block, and the motion information candidate list is stored in the storage unit, where x is an integer equal to or greater than 2. The second processing unit is configured to sequentially scan at least two additional selection-target motion information sets in a second preset order after the first processing unit scans all neighboring image blocks of the current image block, and store the additional selection-target motion information sets that satisfy the preset condition in the motion information candidate list until an amount of the motion information candidate sets in the motion information candidate list is equal to a value of a preset amount or until all the additional selection-target motion information sets have been scanned, the additional selection-target motion information sets including additional selection-target motion information of at least two control points of the current image block, and the additional selection-target motion information of the control points is motion information of inter-coded image blocks adjacent to the current image block. The reference frame indexes of all control points in the additional selected motion information set that satisfies the preset condition are the same, or the reference frame indexes of all control points in the additional selected motion information set that satisfies the preset condition are the same and the additional selected motion vectors of at least two control points in the additional selected motion information set that satisfies the preset condition are different. The filling unit is configured to fill the motion information candidate list with a third motion information candidate set until the amount of the motion information candidate set in the motion information candidate list is equal to the preset amount value when the amount of the motion information candidate set in the motion information candidate list is less than the preset amount value after the second processing unit scans all the additional selected motion information sets.

[0041] Optionally, in one possible implementation form of the present application, the additional selected motion information set includes at least additional selected motion information of a first control point and additional selected motion information of a second control point, where the first control point and the second control point are neighboring control points of the current image block.

[0042] For a specific description of the eighth aspect or any one of its possible implementations in this application, please refer to the detailed description of the third aspect or any one of its possible implementations. In addition, for the beneficial effects of the eighth aspect or any one of its possible implementations, please refer to the analysis of the beneficial effects of the third aspect or any one of its possible implementations. Details will not be described again here.

[0043] According to a ninth aspect, there is provided an inter prediction apparatus, the inter prediction apparatus including: an obtaining unit and a determining unit.

[0044] Specifically, the acquisition unit is configured to acquire and parse the bitstream to acquire a first index value and a motion information differential, the first index value being an index value of a motion information predictor of the control point of the current image block in a motion information candidate list, the motion information candidate list being generated by the construction device of the sixth aspect, the seventh aspect, the eighth aspect, or any one of these possible implementations by using the construction method of the first aspect, the second aspect, the third aspect, or any one of these possible implementations. The acquisition unit is further configured to acquire a motion information predictor of the control point from the motion information candidate list based on the first index value. The determination unit is configured to determine motion information of the control point based on the motion information predictor of the control point and the motion information differential acquired by the acquisition unit. The determination unit is further configured to determine a predicted pixel of the current image block based on the motion information of the control point.

[0045] For the specific description of the ninth aspect in this application, please refer to the detailed description of the fourth aspect. In addition, for the beneficial effects of the ninth aspect, please refer to the analysis of the beneficial effects of the fourth aspect. Details will not be described again here.

[0046] According to a tenth aspect, there is provided an inter prediction apparatus. The inter prediction apparatus includes an obtaining unit and a determining unit.

[0047] The acquisition unit is configured to acquire and parse the bitstream to acquire a second index value, the second index value being an index value of the motion information of the control point of the current image block in the motion information candidate list, the motion information candidate list being generated by the construction device of the sixth aspect, the seventh aspect, the eighth aspect, or any one of these possible implementations by using the construction method of the first aspect, the second aspect, the third aspect, or any one of these possible implementations. The acquisition unit is further configured to acquire the motion information of the control point from the motion information candidate list based on the second index value. The determination unit is configured to determine a predicted pixel of the current image block based on the motion information of the control point acquired by the acquisition unit.

[0048] For the specific description of the tenth aspect in this application, please refer to the detailed description of the fifth aspect. In addition, for the beneficial effects of the tenth aspect, please refer to the analysis of the beneficial effects of the fifth aspect. Details will not be described again here.

[0049] According to an eleventh aspect, a terminal is provided. The terminal includes one or more processors, a memory, and a communication interface. The memory and the communication interface are connected to the one or more processors. The memory is configured to store a computer program code. The computer program code includes instructions. When the one or more processors execute the instructions, the terminal performs a method for constructing a motion information candidate list in the first aspect, the second aspect, the third aspect, or any one of these possible implementations, or performs an inter-prediction method in the fourth aspect, the fifth aspect, or any one of these possible implementations.

[0050] According to a twelfth aspect, a video decoder is provided, the video decoder includes a non-volatile storage medium and a central processing unit. The non-volatile storage medium stores an executable program. The central processing unit is connected to the non-volatile storage medium and performs the method for constructing a motion information candidate list in the first aspect, the second aspect, the third aspect, or any one of these possible implementations, or performs the inter-prediction method in the fourth aspect, the fifth aspect, or any one of these possible implementations.

[0051] According to a thirteenth aspect, a decoder is provided. The decoder includes an apparatus for constructing a motion information candidate list in the sixth aspect, the seventh aspect, the eighth aspect, or any one of these possible implementations, an inter prediction device in the ninth aspect, the tenth aspect, or any one of these possible implementations, and a reconstruction module. The reconstruction module is configured to determine a reconstructed pixel value of a current image block based on a prediction pixel obtained by the inter prediction device.

[0052] According to a fourteenth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions. When the instructions are executed in a terminal in the eleventh aspect, the terminal is enabled to execute the method for constructing a motion information candidate list in the first aspect, the second aspect, the third aspect, or any one of these possible implementations, or to execute the inter-prediction method in the fourth aspect, the fifth aspect, or any one of these possible implementations.

[0053] According to a fifteenth aspect, a computer program product is provided, the computer program product includes instructions. When the computer program product is executed on a terminal according to the eleventh aspect, the terminal is enabled to execute the method for constructing a motion information candidate list according to the first aspect, the second aspect, the third aspect, or any one of their possible implementations, or to execute the inter-prediction method according to the fourth aspect, the fifth aspect, or any one of their possible implementations.

[0054] In this application, the name of the device for constructing the motion information candidate list and the inter-prediction device does not impose any constraint on the device or function module. In actual implementation, the device or function module may have other names. As long as the function of the device or function module is similar to that of the device or function module in this application, the device or function module is within the scope of the claims in this application and their equivalent technologies.

[0055] For a specific description of the eleventh to fifteenth aspects and their possible implementations in this application, please refer to the detailed description of the first, second, third, fourth, fifth, or any one of these possible implementations. In addition, for the beneficial effects of the eleventh to fifteenth aspects and their possible implementations, please refer to the analysis of the beneficial effects of the first, second, third, fourth, fifth, or any one of these possible implementations. The details will not be described again here.

[0056] These and other aspects of the present application may be more succinctly and understandably understood in the following description. [Brief description of the drawings]

[0057] [Figure 1] FIG. 2 illustrates the principle of the inherited control point motion vector prediction method according to an embodiment of the present application; [Diagram 2] FIG. 2 illustrates the principle of the construction control point motion vector prediction method according to an embodiment of the present application; [Diagram 3] 1 is a schematic structural diagram of a video coding system according to an embodiment of the present application; [Figure 4] 1 is a schematic structural diagram of a video encoder according to an embodiment of the present application; [Diagram 5] FIG. 2 is a schematic structural diagram of a video decoder according to an embodiment of the present application; [Figure 6] 1 is a first schematic flowchart of a method for constructing a motion information candidate list according to an embodiment of the present application; [Figure 7] 4 is a second schematic flowchart of a method for constructing a motion information candidate list according to an embodiment of the present application; [Figure 8] 11 is a third schematic flowchart of a method for constructing a motion information candidate list according to an embodiment of the present application; [Figure 9] FIG. 1 is a first schematic structural diagram of an apparatus for constructing a motion information candidate list according to an embodiment of the present application; [Figure 10] FIG. 2 is a second schematic structural diagram of an apparatus for constructing a motion information candidate list according to an embodiment of the present application; [Figure 11] FIG. 1 is a first schematic structural diagram of an inter prediction device according to an embodiment of the present application; [Figure 12] FIG. 2 is a second schematic structural diagram of an inter prediction device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0058] In the embodiments of the present application, the words "example" or "for example" are used to denote providing an example, illustration, or explanation. Any embodiment or design manner described in the embodiments of the present application as an "example" or "for example" should not be described as preferred or advantageous over another embodiment or design manner. Indeed, use of words such as "example", "for example", and the like is intended to present related concepts in a particular manner.

[0059] For ease of understanding of the embodiments of the present application, relevant elements in the embodiments of the present application are described herein first.

[0060] image encoding: the process of compressing an image sequence into a bitstream

[0061] Image decoding: The process of recovering a bitstream into a reconstructed image according to specific syntax rules and a specific processing method.

[0062] Currently, the video image encoding process is as follows: the encoder side first partitions an original image frame into multiple non-overlapping parts, and each part can be used as an image block. Then, the encoder side performs operations such as prediction, transform, and quantization on each image block to obtain a bitstream corresponding to the image block. The prediction is performed to obtain a predictive block of the image block, so that only the difference (or called residual or residual block) between the image block and the predictive block of the image block is coded and transmitted, thereby reducing transmission overhead. Finally, the encoder side transmits the bitstream corresponding to the image block to the decoder side.

[0063] Correspondingly, after receiving the bitstream, the decoder side performs a video decoding process. Specifically, the decoder side performs operations such as prediction, inverse quantization, and inverse transform on the received bitstream to obtain a reconstructed image block (or called image block after reconstruction). This process is called an image reconstruction process (or image reconstruction process). Then, the decoder side collects the reconstructed blocks of each image block in the original image to obtain a reconstructed image of the original image, and reproduces the reconstructed image.

[0064] Existing video image coding techniques include intra prediction and inter prediction. Inter prediction is a prediction that is completed by coding / decoding an image block by using the correlation between a current frame and a reference frame of the current frame. The current frame may have one or more reference frames. Specifically, a predicted image block of a current image block is generated based on pixels in the reference frame of the current image block.

[0065] In general, the predicted image block of the current image block may be generated based on only one reference image block, or the predicted image block of the current image block may be generated based on at least two reference image blocks. Generating the predicted image block of the current image block based on one reference image block is called unidirectional prediction, and generating the predicted image block of the current image block based on at least two reference image blocks is called bidirectional inter prediction. The at least two reference image blocks in bidirectional inter prediction may be from the same reference frame or different reference frames. In other words, the "direction" in this application is a general definition. One direction in this application corresponds to one reference image block.

[0066] Motion compensation (MC) is the process of predicting a current image block by using reference image blocks.

[0067] In most coding frameworks, a video sequence includes a series of pictures, and the pictures are divided into at least one slice, and each slice is further divided into image blocks. Video encoding / decoding is performed by image blocks. The encoding / decoding may be performed row by row, from the top-left position of the image, from left to right and from top to bottom. In this specification, an image block may be a block included in a coding unit (CU) in the High Efficiency Video Coding (HEVC) standard. For example, an image block may be a luma block in a CU, or a chroma block in a CU. This is not particularly limited in the embodiments of this application.

[0068] In this application, the image block being coded / decoded is called the current image block, and the image in which the current image block is located is called the current frame.

[0069] In the current frame, neighboring image blocks of the current image block (e.g., to the left, above, or to the right of the current block) may be coded / decoded to obtain a reconstructed image. The neighboring image blocks are called reconstructed image blocks. Information such as the coding mode and reconstructed pixels of the reconstructed image blocks are available.

[0070] A frame that was encoded / decoded before the current frame is encoded / decoded is called a reconstructed frame.

[0071] A motion vector (MV) is an important parameter in the inter prediction process, which represents the spatial displacement of a coded image block relative to a current image block. Usually, a motion vector can be obtained by using a motion estimation (ME) method, such as motion search. In a preliminary inter prediction technique, the encoder side sends the motion vector of the current image block in the bitstream, so that the decoder side reproduces the predicted pixels of the current image block to obtain a reconstructed block. To further improve the coding efficiency, a method is provided for differentially coding the motion vector by using a reference motion vector. Specifically, only the motion vector difference (MVD) is coded.

[0072] In order to enable the decoder side and the encoder side to use the same reference image block, the encoder side needs to transmit the motion information of each image block to the decoder side in the bitstream. If the encoder side directly encodes the motion vector of each image block, a large amount of transmission resources will be consumed. Since the motion vectors of spatially neighboring image blocks are strongly correlated, the motion vector of the current image block can be predicted based on the motion vectors of the neighboring encoded image blocks. The motion vector obtained through prediction is called MVP, and the difference between the motion vector of the current image block and the MVP is called MVD.

[0073] In the video coding standard H.264, multi-reference frame prediction is used in the motion estimation process to improve the accuracy of prediction. Specifically, a buffer is created to store multiple reconstructed frames, and all reconstructed frames in the buffer are searched for the optimal reference image block for motion compensation to make the redundancy in the time domain more eliminated. In the video coding standard H.264, two buffers, namely reference frame list 0 (reference list 0) and reference frame list 1 (reference list 1), are used in inter prediction. The reference frame in which the optimal reference block in each list is located is marked by an index value, namely ref_idx_l0 and ref_idx_11. In each reference frame list, the motion information of the reference image block includes a reference frame index value (ref_idx_l0 or ref_idx_l1), an index value of the MVP in the motion information candidate list, and an MVD. The decoder side may find the correct reference image block in the selected reference frame based on the reference frame index value, the index value of the MVP in the motion information candidate list, and the MVD.

[0074] Currently, the inter prediction modes frequently used in the HEVC standard are an advanced motion vector prediction (AMVP) mode and a merge mode.

[0075] In AMVP mode, the encoder side constructs a motion information candidate list by using the motion vectors of coded image blocks spatially or temporally adjacent to the current image block, and determines the optimal motion vector in the motion information candidate list as the MVP of the current image block based on the rate-distortion cost. In addition, the encoder side performs motion search in the neighborhood centered on the MVP to obtain the motion vector of the current image block. The encoder side sends the index value, reference frame index value, and MVD of the MVP in the motion information candidate list to the decoder side.

[0076] In the merge mode, the encoder side constructs a motion information candidate list by using the motion vectors of the coded image blocks spatially or temporally adjacent to the current image block, and determines an optimal motion vector in the motion information candidate list as the MV of the current image block based on the rate-distortion cost. The encoder side sends an index value of the position of the optimal motion vector in the motion information candidate list to the decoder side.

[0077] In existing video coding standards, translational motion model-based motion compensation techniques are used, and the motion of all samples in an image block is assumed to be consistent. However, in the real world, many objects, such as rotating objects, roller coasters that rotate in different directions, fireworks, and some stunts in movies, do not have translational motion. The translational motion model-based motion compensation techniques in existing coding standards are used for these moving objects, and coding efficiency is greatly affected. Therefore, research on non-translational motion models is being conducted in the industry.

[0078] In the non-translational motion model prediction mode, the encoder / decoder side derives the motion information of all sub-blocks of the current image block by using the same motion model, and performs motion compensation based on the motion information of all sub-blocks to obtain a predicted image block, which improves prediction efficiency. The motion models frequently used by the encoder / decoder side are the four-parameter affine motion model or the six-parameter affine motion model.

[0079] For example, a four-parameter affine motion model may be represented by using two sample motion vectors and two sample coordinates relative to the pixel at the top left corner of the current image block. In this specification, the samples used to represent the motion model parameters are called control points. If the sample at the top left corner (0,0) of the current image block and the sample at the top right corner (W,0) of the current image block are control points, and the motion vectors of the top left corner and top right corner of the current image block are (vx0,vy0) and (vx1,vy1), respectively, the motion information of each sub-block of the current image block is obtained according to the following formula (1). In the following formula (1), (x,y) are the coordinates of the sub-block relative to the pixel at the top left corner of the current image block, (vx,vy) are the motion vectors of the sub-block, and W is the width of the current image block.

[0080]

number

[0081] For example, a six-parameter affine motion model may be represented by using three sample motion vectors and three sample coordinates relative to the pixel at the top left corner of the current image block. If the sample at the top left corner (0,0) of the current image block, the sample at the top right corner (W,0) of the current image block, and the sample at the bottom left corner (0,H) of the current image block are control points, and the motion vectors of the top left corner, top right corner, and bottom left corner of the current image block are (vx0,vy0), (vx1,vy1), and (vx2,vy2), respectively, the motion information of each sub-block of the current image block is obtained according to the following formula (2): In the following formula (2), (x,y) are the coordinates of the sub-block relative to the pixel at the top left corner of the current image block, (vx,vy) are the motion vectors of the sub-block, and W and H are the width and height of the current image block, respectively.

[0082]

number

[0083] For ease of explanation, in the embodiment of the present application, the image block predicted by using the non-translational motion model prediction mode is called an affine coding block. Of course, alternatively, this type of image block may be represented by using another name, which is not particularly limited in the embodiment of the present application.

[0084] It can be seen from the above description of the non-translational motion model prediction mode that the motion information of the control points of the affine coding block needs to be obtained in the prediction process.

[0085] Usually, the encoder / decoder side can obtain the motion information of the control points by using the Affine Advanced Motion Vector Prediction mode (Affine AMVP mode) or the Affine Merge mode.

[0086] The process of the encoder / decoder side obtaining the motion information of the control points of the current image block using the affine AMVP mode is as follows.

[0087] Step 1: The encoder / decoder side constructs a control point motion information candidate list.

[0088] The encoder / decoder side constructs the motion information candidate list by using the inherited control point motion vector prediction method (see below) and / or the constructed control point motion vector prediction method (see below).

[0089] The motion information candidate list includes a plurality of motion information candidate sets, and each of the motion information candidates may include motion vectors of two (four-parameter affine model) or three (six-parameter affine model) control points.

[0090] Optionally, the encoder / decoder side may prune and sort the motion information candidate list according to a pre-set rule, truncate or fill the motion information candidate list, and obtain a motion information candidate set of a certain amount (corresponding to a value of the amount that will be pre-set later).

[0091] Step 2: The encoder / decoder side determines an optimal motion information candidate set, and uses the optimal motion information candidate as the motion information predictor of the control point.

[0092] The encoder side uses each of the motion information candidate sets in the motion information candidate list to obtain the motion vector of each sub-block of the current image block according to the above formula (1) / (2), obtains the pixel value of the position in the reference frame pointed to by the motion vector of each sub-block, and performs affine transformation motion compensation using the pixel value as the pixel predictor of the sub-block. The encoder side calculates the average value of the difference between the original value and the predictor of all samples in the current image block, and selects the motion information candidate set corresponding to the smallest average value as the optimal motion information candidate set. The encoder side sends the index number of the optimal motion information candidate set in the motion information candidate list to the decoder side.

[0093] The decoder side parses the index number and determines the control point motion vectors predictor (CPMVP) in the motion information candidate list based on the index number.

[0094] Step 3: The encoder / decoder side determines the motion vectors of the control points.

[0095] The encoder side performs motion search within a certain range using the motion vectors in the control point motion information predictor (called control point motion vector predictor) as the starting point of the search to obtain control point motion vectors (CPMV), and transmits the differences (control point motion vectors differences, CPMVD) between the control point motion vectors and the control point motion vector predictor to the decoder side.

[0096] The decoder side parses the motion vector differential of the control point, and adds the motion vector differential of the control point and the motion vector predictor of the control point to obtain the motion vector of the control point.

[0097] The process of obtaining the motion vector of the control point of the current image block using the affine merge mode at the encoder / decoder side is as follows.

[0098] Step I: The encoder / decoder side builds a control point motion information candidate list.

[0099] The encoder / decoder side constructs the control point motion information candidate list by using the inherited control point motion vector prediction method and / or the constructed control point motion vector prediction method.

[0100] Optionally, the encoder / decoder side may prune and sort the control point motion information candidate list according to a pre-set rule, truncate or fill the control point motion information candidate list, and obtain a motion information candidate set of a certain amount (corresponding to a value of the amount that will be pre-set later).

[0101] Step II: The encoder / decoder side determines the motion information of the control points.

[0102] The encoder side uses each of the motion information candidate sets in the motion information candidate list to obtain the motion information of each sub-block of the current image block according to the above formula (1) / (2), obtains the pixel value of the position in the reference frame pointed to by the motion information of each sub-block, and performs affine motion compensation using the pixel value as the pixel predictor of the sub-block. The encoder side calculates the average value of the difference between the original value and the predictor of all samples in the current coding block, and selects the motion information candidate set corresponding to the smallest average value as the motion information of the control point of the current image block. The encoder side sends the index value of the motion information of the control point in the motion information candidate list to the decoder side.

[0103] The decoder side parses the bitstream to obtain the index value of the motion information of the control point in the motion information candidate list, and determines the motion information of the control point in the motion information candidate list based on the index value.

[0104] From the above description, it can be seen that the encoder / decoder side can construct a motion information candidate list by using inherited control point motion vectors and / or constructed control point motion vectors. The following describes the inherited control point motion vector prediction method and the constructed control point motion vector prediction method separately.

[0105] In the inherited control point motion vector prediction method, the encoder / decoder side derives the motion vector of the control point of the current image block by using the motion model of the coded affine coding block that is spatially adjacent to the current image block.

[0106] For example, as shown in FIG. 1, if image block B2, image block B1, image block B0, image block A1, and image block B0 are spatially adjacent to the current image block, the encoder / decoder side scans (checks in order) the spatially neighboring image blocks of the current image block in a preset order, for example, in the order of image block A1, image block B1, image block B0, image block A0, and image block B2, determines an affine coding block among the spatially neighboring image blocks, obtains a motion vector of the control point of the affine coding block, and derives a motion vector of the control point of the current image block or a motion vector predictor of the control point of the current image block based on the motion model used for the affine coding block.

[0107] For ease of understanding, with reference to Fig. 1, the inherited control point motion vector prediction method is described by using an example where the image block A1 is an affine coding block. As shown in Fig. 1, if the sample at the upper left corner (x0, y0) of the current image block and the sample at the upper right corner (x1, y1) of the current image block are control points, the motion vectors of the upper left corner and the upper right corner of the current image block are (vx0, vy0) and (vx1, vy1) respectively.

[0108] If the image block A1 is a four-parameter affine coding block, the sample at the upper left corner (x4, y4) of the image block A1 and the sample at the upper right corner (x5, y5) of the image block A1 are control points, and the motion vectors of the upper left corner and the upper right corner of the image block A1 are (vx4, vy4) and (vx5, vy5) respectively. The encoder / decoder side calculates the motion vector of the upper left corner of the current image block according to the following formula (3), and calculates the motion vector of the upper right corner of the current image block according to the following formula (4). In other words, the motion vector of the control point of the current image block is calculated based on a four-parameter affine motion model.

[0109]

number

[0110] If the sample at the top-left corner (x0,y0) of the current image block, the sample at the top-right corner (x1,y1) of the current image block, and the sample at the bottom-left corner (x2,y2) of the current image block are control points, then the motion vectors for the top-left corner, top-right corner, and bottom-left corner of the current image block are (vx0,vy0), (vx1,vy1), and (vx2,vy2), respectively.

[0111] If the image block A1 is a six-parameter affine coding block, the sample at the upper left corner (x4, y4) of the image block A1, the sample at the upper right corner (x5, y5) of the image block A1, and the sample at the lower left corner (x6, y6) of the image block A1 are control points, and the motion vectors of the upper left corner, upper right corner, and lower left corner of the image block A1 are (vx4, vy4), (vx5, vy5), and (vx6, vy6), respectively. The encoder / decoder side calculates the motion vector of the upper left corner of the current image block according to the following formula (5), calculates the motion vector of the upper right corner of the current image block according to the following formula (6), and calculates the motion vector of the lower left corner of the current image block according to the following formula (7). In other words, the motion vector of the control point of the current image block is calculated based on a six-parameter affine motion model.

[0112]

number

[0113] Of course, the encoder side may alternatively calculate the motion vectors of the control points of the current image block by using motion models other than the four-parameter affine motion model and the six-parameter affine motion model.

[0114] In the constructed control point motion vector prediction method, the encoder / decoder side combines the motion vectors of the coded image blocks spatially / temporally adjacent to the control points of the current image block to obtain motion vector candidates for the control points of the current image block, and does not need to consider whether the coded spatially / temporally neighboring image blocks are affine coding blocks.

[0115] The constructed control point motion vector prediction method can be implemented in the following two ways.

[0116] In a first implementation form, the encoder / decoder side uses motion vectors of encoded image blocks spatially / temporally adjacent to the current image block as motion vector candidates of the control points of the current image block, and then selects an optimal motion vector candidate from the motion vector candidates of the control points as the motion vector of the control point.

[0117] For example, as shown in Figure 2, the top-left corner and the top-right corner of the current image block are the control points of the current image block. If the coded image blocks adjacent to the top-left corner of the current image block are image block A2, image block B2, and image block B3, and the coded image blocks adjacent to the top-right corner of the current image block are image block B0 and image block B1, the encoder / decoder side calculates the motion vector v of the image block A2 as the control point of the current image block. A2 , the motion vector v of image block B2 B2 , and the motion vector v of image block B3 B3 is used as the candidate motion vector for the top-left corner of the current image block, and the motion vector v B0 and the motion vector v of image block B1 B1 Then, the encoder / decoder side combines each of the motion vector candidates for the top-left corner of the current image block with each of the motion vector candidates for the top-right corner of the current image block to generate a motion vector candidate 2-tuple queue {(v A2,v B1 ),(v A2 ,v B0 ),(v B2 ,v B1 ),(v B2 ,v B0 ),(v B3 ,v B1 ),(v B3 ,v B0 )}. Then, the encoder / decoder side may select the best motion vector candidate from the motion vector candidate 2-tuple queue as the motion vector of the control point of the current image block.

[0118] In the second implementation, the encoder / decoder side first calculates additional selection-target motion vectors for each control point of the current image block. Then, the encoder / decoder side combines the motion vectors of each control point by using a non-translational motion model to generate a motion information candidate list. Finally, the encoder / decoder side selects an optimal motion information candidate from the motion information candidate list as the motion information of the control point of the current image block.

[0119] For example, as shown in Figure 2, in an embodiment of the present application, CPk (k = 1, 2, 3, 4) is used to represent the kth control point of the current image block, and the coordinates of CP1, CP2, CP3, and CP4 are (0, 0), (W, 0), (0, H), and (W, H), respectively, where W and H are the width and height of the current image block, respectively. Image block A2, image block B2, image block B3, image block B0, and image block B1 are all spatially adjacent to the current image block and can be used to predict CP1, CP2, or CP3. Image block T is temporally adjacent to the current image block and can be used to predict CP4.

[0120] Step A: The encoder / decoder side obtains additional candidate motion vectors for each control point of the current image block in the following order:

[0121] The encoder / decoder side sequentially queries image block B2, image block A2, and image block B3 for CP1. If image block B2 is obtained through inter-coding, the encoder / decoder side uses the motion vector of image block B2 as the additional selection target motion vector vcp1 for CP1. If image block B2 is not obtained through inter-coding, the encoder / decoder side sequentially detects image block A2 and image block B3. If image block B2, image block A2, and image block B3 are not obtained through inter-coding, the encoder / decoder side cannot determine the additional selection target motion vector for CP1.

[0122] For CP2, the encoder / decoder side sequentially queries the image block B0 and the image block B1, and determines an additional selection candidate motion vector vcp2 for CP2 based on the query result.

[0123] For CP3, the encoder / decoder side sequentially queries the image block A0 and the image block A1, and determines an additional selection candidate motion vector vcp3 for CP3 based on the query result.

[0124] For CP4, the encoder / decoder side uses the motion vector of image block T as an additional candidate motion vector vcp4 for CP4.

[0125] Step B: The encoder / decoder side generates a control point motion information candidate list by combining the additional selection candidate motion vectors of the control points described above.

[0126] Specifically, the encoder / decoder side combines the additional selected motion vectors of any two control points to construct a four-parameter affine motion model. The additional selected motion vectors of two control points can be combined as follows: (vcp1,vcp4), (vcp2,vcp3), (vcp1,vcp2), (vcp2,vcp4), (vcp1,vcp3), and (vcp3,vcp4). For example, in the embodiment of the present application, the four-parameter affine motion model constructed by using vcp1 and vcp2 is denoted as Affine(vcp1,vcp2).

[0127] The encoder / decoder side combines the additional selected motion vectors of any three control points to construct a six-parameter affine motion model. The additional selected motion vectors of three control points can be combined as (vcp1,vcp2,vcp4), (vcp1,vcp2,vcp3), (vcp2,vcp3,vcp4), and (vcp1,vcp3,vcp4). For example, in the embodiment of the present application, the six-parameter affine motion model constructed by using vcp1, vcp2, and vcp3 is denoted as Affine(vcp1,vcp2,vcp3).

[0128] The encoder / decoder side combines the additional selected motion vectors of the above four control points to construct an 8-parameter affine motion model. For example, in the embodiment of this application, the 8-parameter bilinear model constructed by using vcp1, vcp2, vcp3, and vcp4 is denoted as Bilinear(vcp1,vcp2,vcp3,vcp4).

[0129] In practical application, the encoder / decoder side performs scanning in the above order. If the additional selectable motion vectors in the set cannot be obtained, the encoder / decoder side cannot determine the set. Otherwise, the encoder / decoder side determines the reference frame indexes of all control points in the set, and scales the additional selectable motion vectors in the model according to the following equation (8). Furthermore, if all the additional selectable motion vectors obtained after scaling are consistent, the set obtained by the encoder / decoder side is invalid. Otherwise, the encoder / decoder side adds the set to the motion information candidate list.

[0130] In other words, the encoder / decoder side first preliminarily calculates the additional candidate motion vector of each control point, and combines the obtained additional candidate motion vectors in the above-mentioned order. If the reference frame indexes of the additional candidate motion vectors in the set are not consistent, the encoder / decoder side scales the additional candidate motion vectors in the set. If the additional candidate motion vectors obtained after scaling are consistent, the encoder / decoder side adds the set to the motion information candidate list.

[0131]

number

[0132] In equation (8), CurPoc represents the sequence number of the current frame, DesPoc represents the sequence number of the reference frame of the current image block, SrcPoc represents the sequence number of the reference frame of the control point, and MV s denotes the MV obtained after scaling.

[0133] It should be noted that the above set of additional selected motion vectors of different control points can be converted between each other.

[0134] For example, the encoder / decoder side may convert (vcp1, vcp2) to (vcp1, vcp2, vcp3) according to the following equation (9).

[0135]

number

[0136] Specifically, vcp1 is (vx0, vy0), specifically, vcp2 is (vx1, vy1), and specifically, vcp3 is (vx2, vy2).

[0137] The encoder / decoder side may convert (vcp1,vcp3) to (vcp1,vcp2,vcp3) or (vcp1,vcp2) according to the following equation (10).

[0138]

number

[0139] The encoder / decoder side may convert (vcp2,vcp3) to (vcp1,vcp2,vcp3) or (vcp1,vcp2) according to the following equation (11).

[0140]

number

[0141] The encoder / decoder side may convert (vcp1,vcp4) to (vcp1,vcp2,vcp3) or (vcp1,vcp2) according to the following equations (12) and (13).

[0142]

number

[0143] Specifically, vcp4 is (vx3,vy3).

[0144] The encoder / decoder side may convert (vcp2,vcp4) to (vcp1,vcp2,vcp3) or (vcp1,vcp2) according to the following equations (14) and (15).

[0145]

number

[0146] The encoder / decoder side may convert (vcp3,vcp4) to (vcp1,vcp2,vcp3) or (vcp1,vcp2) according to the following equations (16) and (17).

[0147]

number

[0148] The encoder / decoder side may convert (vcp1, vcp2, vcp4) to (vcp1, vcp2, vcp3) according to the following equation (18).

[0149]

number

[0150] The encoder / decoder side may convert (vcp2, vcp3, vcp4) to (vcp1, vcp2, vcp3) according to the following equation (19).

[0151]

number

[0152] The encoder / decoder side may convert (vcp1, vcp3, vcp4) to (vcp1, vcp2, vcp3) according to the following equation (20).

[0153]

number

[0154] In the prior art, the method used by the encoder / decoder side to construct a control point motion information candidate list is usually as follows: S1: The encoder / decoder side calculates motion information candidates of multiple control points of the current image block by using an inherited control point motion vector prediction method, and adds the motion information candidates to the motion information candidate list; S2: The encoder / decoder side calculates motion information candidates of multiple control points of the current image block by using a constructed control point motion vector prediction method, and adds the motion information candidates to the motion information candidate list; S3: If the amount of motion information candidate sets in the motion information candidate list is less than a preset amount value, the encoder / decoder side fills the motion information candidate list with the preset motion information candidate sets until the amount of motion information candidate sets in the motion information candidate list reaches a preset amount value.

[0155] In S1 and S2, before adding a new motion information candidate set to the motion information candidate list every time, the encoder / decoder side first checks whether the same motion information set already exists in the motion information candidate list. If the same motion information set exists in the motion information candidate list, the encoder / decoder side does not add the new motion information candidate to the list. This process is called pruning of the motion information candidate list. Pruning of the motion information candidate list can prevent the same motion information from being in the motion information candidate list, and can avoid redundant rate-distortion cost calculation. However, the computational complexity of this process is also relatively high, and the delay is relatively long.

[0156] In addition, it is very complicated for the encoder / decoder side to calculate the motion information candidate of the control point by using the inherited control point motion vector prediction method. When the motion information candidate of the control point is calculated by using the constructed control point motion vector prediction method, if the reference frame index in the additional selection target motion information set is not consistent, the encoder / decoder side needs to further scale the additional selection target motion vector. The calculation complexity of scaling is large, which also increases the delay.

[0157] In order to solve the above-mentioned problem, the present application provides a method for constructing a motion information candidate list and a corresponding construction device. The method or construction device sequentially executes a first processing process on all neighboring image blocks of a current image block in a first preset order until the amount of motion information candidate sets in the motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned. After the amount of motion information candidate sets in the motion information candidate list is equal to the first preset value or after all neighboring image blocks have been scanned, the construction device sequentially scans at least two additional selection target motion information sets in a second preset order, and stores the additional selection target motion information sets that meet a preset condition in the motion information candidate list until the amount of motion information candidate sets in the motion information candidate list is equal to a preset amount value or until all additional selection target motion information sets have been scanned. In this specification, the value of the preset amount is greater than the first preset value, and the additional selection target motion information set includes additional selection target motion information of at least two control points of the current image block, and the additional selection target motion information of the control points is motion information of an inter-coded image block adjacent to the current image block. If the amount of the motion information candidate sets in the motion information candidate list is less than the preset amount value after all the additional selection target motion information sets are scanned, the construction device fills the motion information candidate list with the first preset amount value until the amount of the motion information candidate sets in the motion information candidate list is equal to the preset amount value.

[0158] Alternatively, the construction device may sequentially execute the first processing process on all neighboring image blocks of the current image block in a first preset order until the amount of motion information candidate sets in the motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned. In addition, the construction device further sequentially scans at least two additional selection target motion information sets in a second preset order, and stores the additional selection target motion information sets that satisfy the preset condition in the second motion information candidate list until the amount of motion information candidate sets in the second motion information candidate list is equal to a second preset value or until all additional selection target motion information sets have been scanned. Then, the construction device merges the first motion information candidate list and the second motion information candidate list with a preset rule to obtain a third motion information candidate list. The amount of motion information candidate sets in the third motion information candidate list is equal to a preset amount value. The sum of the first preset value and the second preset value is equal to or greater than the preset amount value.

[0159] Alternatively, the construction device may scan all neighboring image blocks of the current image block in a first preset order to generate a motion information candidate list including at least one motion information candidate set, where the motion information candidate set includes motion information candidates of x control points of the current image block, where x is an integer equal to or greater than 2. After scanning all neighboring image blocks of the current image block, the construction device sequentially scans at least two additional selection-target motion information sets in a second preset order, and stores the additional selection-target motion information sets that meet the preset condition in the motion information candidate list until the amount of the motion information candidate sets in the motion information candidate list is equal to the preset amount value or until all the additional selection-target motion information sets have been scanned. If the amount of the motion information candidate sets in the motion information candidate list is less than the preset amount value after all the additional selection-target motion information sets have been scanned, the construction device fills the motion information candidate list with a third preset motion information candidate set until the amount of the motion information candidate sets in the motion information candidate list is equal to the preset amount value.

[0160] Specifically, the first processing process is as follows: for the i-th (i ∈ [1, n]) neighboring image block, when the i-th neighboring image block is an affine coding block, the construction device determines whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit. If the motion information candidate set from the coding unit to which the i-th neighboring image block belongs does not exist in the motion information candidate list, the construction device determines motion information candidates of x control points of the current image block according to the motion information of x control points of the i-th neighboring image block, and stores the motion information candidate set including the motion information candidates of x control points of the current image block in the motion information candidate list, where n is the amount of neighboring image blocks of the current image block, and both x and n are integers equal to or greater than 2.

[0161] The additional selected motion information set includes additional selected motion information of at least two control points of the current image block, and the additional selected motion information of the control points is motion information of an inter-coded image block adjacent to the current image block. The reference frame indexes of all the control points in the additional selected motion information set that satisfies a preset condition are the same, or the reference frame indexes of all the control points in the additional selected motion information set that satisfies a preset condition are the same, and the additional selected motion vectors of at least two control points in the additional selected motion information set that satisfies the preset condition are different.

[0162] It can be seen that in both the first processing process and the processing process of additional selection target motion information, whether to add new motion information candidate set to motion information candidate list is determined based on the result of the determination, and it is not necessary to first calculate new motion information candidate set, and then determine whether the same motion information exists in the motion information candidate list.This substantially reduces the calculation complexity and delay.

[0163] In addition, the present application further provides an inter-prediction method. The inter-prediction device parses a bitstream to obtain an index value (i.e., a first index value) and a motion information difference, and the index value is an index value of a motion information predictor of a control point of a current image block in a motion information candidate list generated by a construction device by using the above-mentioned construction method. In this manner, the inter-prediction device obtains a motion information predictor of a control point from the motion information candidate list according to the first index value, determines the motion information of the control point according to the motion information predictor of the control point and the motion information difference, and determines a predicted pixel of the current image block according to the motion information of the control point.

[0164] Alternatively, the inter prediction device may parse the bitstream to obtain an index value (i.e., a second index value) of the motion information of the control point of the current image block in the motion information candidate list generated by the construction device by using the above-mentioned construction method. In this manner, the inter prediction device obtains the motion information of the control point from the motion information candidate list based on the second index value, and determines the prediction pixel of the current image block based on the motion information of the control point.

[0165] The construction method provided in the present application substantially reduces computational complexity and delay. Therefore, the inter prediction performed based on the motion information candidate list generated by using the construction method also substantially reduces computational complexity and delay.

[0166] The method for constructing a motion information candidate list provided in the present application may be executed by a construction device, a video coding device, a video codec, or another device having a video coding function. The inter prediction method provided in the present application may be executed by an inter prediction device, a video coding device, a video codec, or another device having a video coding function. The construction device and the inter prediction device may be disposed in the same device, or may be disposed independently. This is not particularly limited in the embodiment of the present application.

[0167] The method for constructing a motion information candidate list and the inter-prediction method provided in this application are applicable to a video coding system. A video encoder 100 and a video decoder 200 in the video coding system are configured to generate a motion information candidate list and predict a current image block according to the example of the method for constructing a motion information candidate list in this application.

[0168] FIG. 3 shows the structure of a video coding system. As shown in FIG. 3, the video coding system includes a source device 10 and a destination device 20. The source device 10 generates encoded video data. The source device 10 may also be called a video encoding device or a video encoding device. The destination device 20 may decode the encoded video data generated by the source device 10. The destination device 20 may also be called a video decoding device or a video decoding device. The source device 10 and / or the destination device 20 may include at least one processor and a memory connected to the at least one processor. The memory may include, but is not limited to, a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, or any other medium that can be configured to store required program code in the form of instructions or data structures and can be accessed by a computer. This is not particularly limited in this application.

[0169] The source device 10 and destination device 20 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handheld telephone sets such as “smart” phones, television sets, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or similar devices.

[0170] The destination device 20 may receive the encoded video data from the source device 10 through the link 30. The link 30 may include one or more media and / or devices capable of transferring the encoded video data from the source device 10 to the destination device 20. In an example, the link 30 may include one or more communication media that enable the source device 10 to transmit the encoded video data directly to the destination device 20 in real time. In this example, the source device 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the destination device 20. The one or more communication media may include wireless and / or wired communication media, e.g., radio frequency (RF) spectrum or one or more physical transmission cables. The one or more communication media may be part of a packet-based network, e.g., a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include a router, a switch, a base station, or another device that implements communication from the source device 10 to the destination device 20.

[0171] In another example, the encoded video data may be output to storage device 40 through output interface 140. Similarly, the encoded video data may be accessed from storage device 40 through input interface 240. Storage device 40 may include multiple types of locally accessible data storage media, such as Blu-ray® discs, high density digital video discs (DVDs), compact disc read-Only memory (CD-ROMs), flash memory, or another suitable digital storage medium configured to store the encoded video data.

[0172] In another example, storage device 40 may correspond to a file server or another intermediate storage device that stores the encoded video data generated by source device 10. In this example, destination device 20 may obtain the stored video data from storage device 40 through streaming transmission or download. The file server may be any type of server capable of storing encoded video data and transmitting the encoded video data to destination device 20. For example, the file server may include a world wide web (Web) server (e.g., used for websites), a file transfer protocol (FTP) server, a network attached storage (NAS) device, and a local disk drive.

[0173] Destination device 20 may access the encoded video data through any standard data connection (e.g., an Internet connection). Exemplary types of data connections include a wireless channel or a wired connection (e.g., a cable modem), or a set of these, that may be used to access encoded video data stored on a file server. The transmission of the encoded video data from the file server may be a streaming transmission, a download transmission, or a set of these.

[0174] The method for constructing a motion information candidate list and the inter-prediction method in the present application are not limited to wireless application scenarios.For example, the method for constructing a motion information candidate list and the inter-prediction method in the present application may be applied to video coding to support multiple multimedia applications, such as the following applications: terrestrial television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., over the Internet), encoding video data stored in a data storage medium, decoding video data stored in a data storage medium, or another application.In some examples, the video coding system may be configured to support unidirectional or bidirectional video transmission to support applications such as streaming video transmission, video playback, video broadcasting, and / or video telephony.

[0175] It should be noted that the video coding system shown in FIG. 3 is merely an example of a video coding system and does not limit the video coding system in this application. The method for constructing a motion information candidate list provided in this application may also be applicable to a scenario where there is no data communication between an encoding device and a decoding device. In other examples, the video data to be encoded or the encoded video data may be retrieved from a local memory, may be transmitted in a streaming manner over a network, etc. The video encoding device may encode the video data to be encoded and store the encoded video data in a memory. The video decoding device may also retrieve the encoded video data from the memory and decode the encoded video data.

[0176] 3, source device 10 includes video source 101, video encoder 102, and output interface 103. In some examples, output interface 103 may include a modulator / demodulator (modem) and / or a transmitter. Video source 101 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video input interface for receiving video data from a video content provider, and / or a computer plafix system for generating video data, or a set of the aforementioned video data sources.

[0177] Video encoder 102 may encode video data from video source 101. In some examples, source device 10 transmits the encoded video data directly to destination device 20 through output interface 103. In other examples, the encoded video data may alternatively be stored on storage device 40, so that destination device 20 later accesses the encoded video data for decoding and / or playback.

[0178] In the example of Figure 3, destination device 20 includes a display device 201, a video decoder 202, and an input interface 203. In some examples, input interface 203 includes a receiver and / or a modem. Input interface 203 may receive encoded video data over link 30 and / or from storage device 40. Display device 201 may be integrated with destination device 20 or disposed outside destination device 20. Typically, display device 201 displays the decoded video data. Display device 201 may include multiple types of display devices, for example, a liquid crystal display, a plasma display, an organic light emitting diode display, or another type of display device.

[0179] Optionally, the video encoder 102 and the video decoder 202 may be integrated with an audio encoder and an audio decoder, respectively, and may include appropriate multiplexer-demultiplexer units or other hardware and software for encoding both audio and video in a combined data stream or separate data streams.

[0180] The video encoder 102 and the video decoder 202 may each include at least one microprocessor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA), discrete logic, hardware, or any set thereof. When the method for constructing a motion information candidate list provided in the present application is implemented by using software, the instructions used for the software may be stored in a suitable non-volatile computer-readable storage medium, and at least one processor may be used to execute the instructions in the hardware to implement the present application. Any one of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered as at least one processor. The video encoder 102 may be included in an encoder, the video decoder 202 may be included in a decoder, and the encoder or decoder may be part of a combined encoder / decoder (codec) in a corresponding device.

[0181] The video encoder 102 and the video decoder 202 in the present application may perform operations according to a video compression standard (e.g., HEVC) or another industry standard, which is not particularly limited in the present application.

[0182] Refer to Fig. 6. Both the video encoder 102 and the video decoder 202 are configured to sequentially execute a first processing process on all neighboring image blocks of a current image block in a first preset order until the amount of the motion information candidate set in the motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned (S601); after S601, sequentially scan at least two additional selection target motion information sets in a second preset order, store the additional selection target motion information sets that meet a preset condition in the motion information candidate list until the amount of the motion information candidate set in the motion information candidate list is equal to the preset amount value or until all additional selection target motion information sets have been scanned (S602); and if the amount of the motion information candidate set in the motion information candidate list is less than the preset amount value after all additional selection target motion information sets have been scanned, fill the motion information candidate list with the first preset motion information candidate set until the amount of the motion information candidate set in the motion information candidate list is equal to the preset amount value (S603).

[0183] Refer to Fig. 7. The video encoder 102 and the video decoder 202 are alternatively configured to: sequentially execute a first processing process on all neighboring image blocks of a current image block in a first preset order until an amount of the motion information candidate set in the motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned (S701); sequentially scan at least two additional selection-target motion information sets in a second preset order to store additional selection-target motion information sets that meet a preset condition in the second motion information candidate list until an amount of the motion information candidate set in the second motion information candidate list is equal to a second preset value or until all additional selection-target motion information sets have been scanned (S702); and merge the first motion information candidate list and the second motion information candidate list according to a preset rule to obtain a third motion information candidate list (S703).

[0184] Refer to Fig. 8. The video encoder 102 and the video decoder 202 are alternatively configured to scan all neighboring image blocks of a current image block in a first preset order to generate a motion information candidate list including at least one motion information candidate set (S800), sequentially scan at least two additional selection-target motion information sets in a second preset order after scanning all neighboring image blocks of the current image block, store the additional selection-target motion information sets that satisfy a preset condition in the motion information candidate list until the amount of the motion information candidate sets in the motion information candidate list is equal to a preset amount value or until all the additional selection-target motion information sets have been scanned (S801), and fill the motion information candidate list with a third preset motion information candidate set until the amount of the motion information candidate sets in the motion information candidate list is equal to a preset amount value after all the additional selection-target motion information sets have been scanned.

[0185] In addition, the video encoder 102 is further configured to select an optimal motion information candidate set from the motion information candidate list as a motion information predictor for the current image block, determine motion information for the current image block, calculate a difference between the motion information predictor and the motion information, and transmit the difference value and an index value of the optimal motion information candidate set in the motion information candidate list to the peer end.

[0186] The video decoder 202 is further configured to parse the bitstream to obtain a first index value and a motion information differential, in other words, to determine an index value and a motion information differential of a motion information predictor of a control point of a current image block in a motion information candidate list, to obtain a motion information predictor of the control point, and to determine motion information of the control point based on the motion information predictor and the motion information differential of the control point. In this manner, the video decoder 202 may determine a predicted pixel of the current image block based on the motion information of the control point.

[0187] Optionally, the video decoder 202 is further configured to parse the bitstream to obtain a second index value, in other words, determine the index value of the motion information of the control point of the current image block in the motion information candidate list, and obtain the motion information of the control point. In this manner, the video decoder 202 may determine a prediction pixel of the current image block based on the motion information of the control point.

[0188] FIG. 4 is a schematic structural diagram of a video encoder 102 according to an embodiment of the present application. As shown in FIG. 4, the video encoder 102 is configured to output video to a post-processing entity 41. The post-processing entity 41 represents an example of a video entity capable of processing encoded video data from the video encoder 102, such as a media-aware network element (MANE) or an integrator / editor. In some cases, the post-processing entity 41 may be an example of a network entity. In some video encoding systems, the post-processing entity 41 and the video encoder 102 may be components of separate devices. In other cases, the functions of the post-processing entity 41 may be performed by the same device that includes the video encoder 100. In one example, the post-processing entity 41 is an example of the storage device 40 of FIG. 3.

[0189] 4, the video encoder 102 includes a transformer 301, a quantizer 302, an entropy encoder 303, a filter 306, a memory 307, a prediction processing unit 308, and an adder 312. The prediction processing unit 308 includes an intra predictor 309 and an inter predictor 310. To reconstruct image blocks, the video encoder 102 further includes an inverse quantizer 304, an inverse transformer 305, and an adder 311. The filter 306 is defined as representing one or more loop filters, e.g., a deblocking filter, an adaptive loop filter, and a sample adaptive offset filter.

[0190] The memory 307 may store video data encoded by components of the video encoder 102. The video data stored in the memory 307 may be obtained from the video source 101. The memory 307 may be a reference image memory that stores reference video data used by the video encoder 102 to encode the video data in an intra-coding mode or an inter-coding mode. The memory 307 may be a dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetic RAM (MRAM), resistive RAM (RRAM), or another type of memory device.

[0191] The video encoder 102 receives video data and stores the video data in a video data memory. The partition unit partitions the video data into several image blocks, which may be further partitioned into smaller blocks, for example, based on a quadtree structure or a binary tree structure. The partitioning may further include partitioning into slices, tiles, or other larger units. The video encoder 102 is typically a component for encoding image blocks in a video slice to be encoded. A slice may be partitioned into multiple image blocks (or into sets of image blocks called tiles).

[0192] An intra predictor 309 in prediction processing unit 308 may perform intra predictive coding on the current image block relative to one or more neighboring image blocks in the same frame or slice as the current image block to eliminate spatial redundancy. An inter predictor 310 in prediction processing unit 308 may perform inter predictive coding on the current image block relative to one or more predictive image blocks in one or more reference images to eliminate temporal redundancy.

[0193] The prediction processing unit 308 may provide the obtained intra-coded and inter-coded image blocks to an adder 310 to generate residual blocks, and provide the residual blocks to an adder 309 to reconstruct the coded blocks used as reference images.

[0194] After the prediction processing unit 308 generates a predictive image block for a current image block through inter-prediction and intra-prediction, the video encoder 102 generates a residual image block by subtracting the predictive image from the current image block to be encoded. The adder 312 represents one or more components that perform this subtraction operation. The residual video data in the residual block may be included in one or more transform units (TUs) and applied to a transformer 301. The transformer 301 converts the residual video data into residual transform coefficients through a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform. The transformer 301 may convert the residual video data from a pixel value domain to a transform domain, e.g., a frequency domain.

[0195] The transformer 301 may send the obtained transform coefficients to the quantizer 302. The quantizer 302 quantizes the transform coefficients to further reduce the bit rate. In some examples, the quantizer 302 may further scan a matrix including the quantized transform coefficients. Alternatively, the entropy encoder 303 may perform the scan.

[0196] After quantization, the entropy encoder 303 performs entropy coding on the quantized transform coefficients. For example, the entropy encoder 303 may perform context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), or another entropy coding method or technique. After the entropy encoder 303 performs the entropy coding, the encoded bitstream may be transmitted to the video decoder 202 or archived for later transmission or later retrieval by the video decoder 202. The entropy encoder 303 may further perform entropy coding on syntax elements of the current image block to be coded.

[0197] The inverse quantizer 304 and the inverse transformer 305 perform inverse quantization and inverse transformation, respectively, to reconstruct a residual block in a pixel domain, for example to be used later as a reference block of a reference image. The adder 311 adds the reconstructed residual block to a predicted image block generated by the inter predictor 310 or the intra predictor 309 to generate a reconstructed image block. The predicted image block of an image block may be obtained by performing processing (such as interpolation) on a reference image block of the image block.

[0198] It should be understood that other structural variations of the video encoder 102 may be used to encode the video stream. For example, for some image blocks or image frames, the video encoder 102 may directly quantize the residual signal, and correspondingly, no processing by the transformer 301 and the inverse transformer 305 is required. Alternatively, for some image blocks or image frames, the video encoder 102 does not generate residual data, and correspondingly, no processing by the transformer 301, the quantizer 302, the inverse quantizer 304, and the inverse transformer 305 is required. Alternatively, the video encoder 102 may directly store the reconstructed image block as a reference block without processing by the filter 306. Alternatively, the quantizer 302 and the inverse quantizer 304 in the video encoder 102 may be combined.

[0199] 5 is a schematic structural diagram of a video decoder 202 according to an embodiment of the present application. As shown in FIG. 5, the video decoder 202 includes an entropy decoder 401, an inverse quantizer 402, an inverse transformer 403, a filter 404, a memory 405, a prediction processing unit 406, and an adder 409. The prediction processing unit 406 includes an intra predictor 407 and an inter predictor 408. In some examples, the video decoder 202 may perform a decoding process that is generally the reverse of the encoding process performed by the video encoder 102 of FIG. 4.

[0200] In the decoding process, the video decoder 202 receives a bitstream from the video encoder 102. The video decoder 202 may receive video data from the network entity 42 and may optionally further store the video data in a video data memory (not shown in the figure). The video data memory may store video data, e.g., an encoded bitstream, to be decoded by the components of the video decoder 202. The video data stored in the video data memory may be obtained from a local video source, e.g., the storage device 40 or a camera, through a wired or wireless network communication of the video data, or by accessing a physical data storage medium. Although the video data memory is not shown in FIG. 5, the video data memory and the memory 405 may be the same memory or may be memories disposed separately. The video data memory and the memory 405 may each be constituted by any one of a number of types of memory devices, e.g., dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or another type of memory device. In various examples, the video data memory may be integrated onto a chip with other components of the video decoder 202, or may be disposed off-chip relative to those components.

[0201] Network entity 42 may be, for example, a server, a MANE, a video editor / splitter, or another device configured to implement one or more of the techniques described above. Network entity 42 may or may not include a video encoder, such as video encoder 102. Before network entity 42 transmits the bitstream to video decoder 202, network entity 42 may perform some of the techniques described in this application. In some video decoding systems, network entity 42 and video decoder 202 may be components of separate devices. In other cases, the functions of network entity 42 may be performed by the same device that includes video decoder 202. In some cases, network entity 42 may be an example of storage device 40 of FIG. 3.

[0202] The entropy decoder 401 of the video decoder 202 performs entropy decoding on the bitstream to generate quantized coefficients and some syntax elements. The entropy decoder 401 forwards the syntax elements to a filter 404. The video decoder 202 may receive the syntax elements / one syntax element at the video slice level and / or the image block level.

[0203] The inverse quantizer 402 performs inverse quantization, or in other words, inverse quantizes, on the quantized transform coefficients provided in the bitstream and decoded by the entropy decoder 401. The inverse quantization process may include determining the degree of quantization to be applied by using a quantization parameter calculated by the video encoder 102 for each image block in the video slice, as well as determining the degree of inverse quantization to be applied. The inverse transformer 403 applies an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to produce pixel domain residual blocks.

[0204] After the prediction processing unit 406 generates a predicted image block for the current image block or a sub-block of the current image block, the video decoder 202 adds the residual block from the inverse transformer 403 and the corresponding predicted image block generated by the prediction processing unit 406 to obtain a reconstructed block, i.e., a decoded image block. The adder 409 (also called a reconstructor 409) represents a component that performs this addition operation. When necessary, a filter (in or after the decoding loop) may be further used to smooth pixels or improve the video quality in another manner. The filter 404 may be one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter.

[0205] It should be understood that other structural variations of the video decoder 202 may be used to decode the bitstream. For example, for some image blocks or frames, the entropy decoder 401 of the video decoder 202 does not obtain quantized coefficients through decoding, and correspondingly, no processing by the inverse quantizer 402 and the inverse transformer 403 is required. For example, the inverse quantizer 402 and the inverse transformer 403 in the video decoder 202 may be combined.

[0206] Based on the video coding system shown in FIG. 3, the video encoder 102 shown in FIG. 4, and the video decoder 202 shown in FIG. 5, the following describes in detail the method for constructing a motion information candidate list and the inter prediction method provided in this application.

[0207] 6 is a schematic flowchart of a method for constructing a motion information candidate list according to an embodiment of the present application. The method shown in FIG. 6 is executed by a constructing device. The constructing device may be the video decoder 202 or the video encoder 102 of FIG. 3.

[0208] As shown in FIG. 6, the method for constructing a motion information candidate list in this embodiment of the present application may include the following steps.

[0209] S601: The construction device sequentially executes a first processing process on all neighboring image blocks of a current image block in a first preset order until the amount of motion information candidate sets in the motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned.

[0210] The first processing process is as follows: for the i-th neighboring image block, when the i-th neighboring image block is an affine coding block, the construction device determines whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit. If the motion information candidate set from the coding unit to which the i-th neighboring image block belongs does not exist in the motion information candidate list, the construction device determines motion information candidates for x control points of the current image block according to the motion information of the x control points of the i-th neighboring image block, and stores the motion information candidate set including the motion information candidates for the x control points of the current image block in the motion information candidate list, where i ∈ [1, n], n is the amount of neighboring image blocks of the current image block, and both x and n are integers equal to or greater than 2.

[0211] Optionally, the method used by the construction device to determine whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit is as follows: the construction device determines whether the information of the upper left corner of the coding unit to which the i-th neighboring image block belongs is the same as the information of the upper left corner of the target coding unit, and the target coding unit is the coding unit to which the neighboring image block used to calculate the motion information candidate set stored in the motion information candidate list belongs. In this specification, the information of the upper left corner can be the position information of the upper left corner.

[0212] For example, if image block A0 and image block B1 are adjacent to the current image block, the coding unit to which image block B1 belongs is CU2, and the coordinates of the upper left corner of CU2 are (x0, y0), and the coding unit to which image block A0 belongs is CU1, and the coordinates of the upper left corner of CU1 are (x1, y1). If the motion information candidate list is empty in this case, the construction device determines a motion information candidate of the control point of the current image block based on the motion information of the control point of image block B1, and stores the motion information candidate in the motion information candidate list. If the motion information candidate list is not empty in this case and includes a motion information candidate determined based on the motion information of the control point of image block A0, the construction device determines whether x1≠x0||y1≠y0 is satisfied. If the construction device determines that x1≠x0||y1≠y0, the construction device determines a motion information candidate of the control point of the current image block based on the motion information of the control point of image block B1, and stores the motion information candidate in the motion information candidate list. Otherwise, the constructor scans the next neighboring image block.

[0213] Optionally, the construction device may generate a candidate list that is used to store the coordinates of the upper left corner of the target coding unit. In this manner, the construction device directly determines whether the information of the upper left corner of the coding unit to which the i-th neighboring image block belongs exists in the candidate list. Alternatively, the construction device may store the coordinates of the upper left corner of the target coding unit in the motion information candidate list. This is not particularly limited in this embodiment of the present application.

[0214] It should be noted that the construction device may alternatively determine whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit by using another method. This is not particularly limited in this embodiment of the present application. For example, the construction device may allocate a count value to each coding unit, or allocate a unique ID to each coding unit. In this manner, the construction device performs the determination by comparing the count value or based on the ID.

[0215] It can be seen that the first processing process is similar to the above-mentioned inherited control point motion vector prediction method. The difference is that in the first processing process, the construction device determines whether the i-th neighboring image block and the motion information candidate set stored in the motion information candidate list are from the same coding unit, and determines whether to determine the motion information candidate of x control points of the current image block according to the motion information of x control points of the i-th neighboring image block, and further stores the motion information candidate set including the motion information candidate of x control points of the current image block in the motion information candidate list. When adding a new motion information candidate to the motion information candidate list every time, the construction device does not need to determine whether the new motion information candidate exists in the motion information candidate list. This substantially reduces the calculation complexity and delay.

[0216] The first preset value in the embodiment of the present application may be a maximum inheritance list length MaxNumInheritMrgCand. The first preset value is an integer smaller than a preset amount value, for example, 1, 2, or 3. The preset amount value is a maximum length value of motion information candidate. The construction device sequentially executes the first processing process on all neighboring image blocks of the current image block in a first preset order. When the amount of motion information candidate sets in the motion information candidate list reaches the first preset value, the construction device ends the generation of the motion information candidate list. Otherwise, the construction device continues to scan the next neighboring image block.

[0217] In this embodiment of the present application, the number of times that the constructing device scans the neighboring image by using the first processing process is limited by a first preset value.In other words, the number of times that the inherited control point motion vector prediction method is executed is reduced.Because the calculation complexity of the inherited control point motion vector prediction method is relatively high, after the number of times that the inherited control point motion vector prediction method is executed is reduced, the calculation complexity is relatively reduced.

[0218] S602: After executing S601, the construction device sequentially scans at least two additional selection target motion information sets in a second preset order, and stores the additional selection target motion information sets that satisfy the preset condition in the motion information candidate list until the amount of motion information candidate sets in the motion information candidate list is equal to a preset amount value or until all additional selection target motion information sets have been scanned.

[0219] The value of the preset amount is greater than the first preset value. The additional selected motion information set includes additional selected motion information of at least two control points of the current image block, and the additional selected motion information of the control points is motion information of inter-coded image blocks adjacent to the current image block.

[0220] Specifically, after the amount of motion information candidate sets in the motion information candidate list is equal to a first preset value, the construction device determines additional selected motion information of each control point of the current image block, and combines the obtained additional selected motion information of the control points to obtain at least two additional selected motion information sets.

[0221] Furthermore, the builder implements one of three optimal implementations:

[0222] In the first optimal implementation form, please refer to the description of steps A and B for the method used by the construction device to generate at least two additional sets of motion information to be selected, sequentially scan the at least two additional sets of motion information to be selected, and store the additional selected motion information sets that meet the predetermined conditions in the motion information candidate list.

[0223] In a second optimal implementation form, after at least two additional selected motion information sets are generated, the method used by the constructing device to sequentially scan the at least two additional selected motion information sets and store the additional selected motion information sets that meet the preset condition in the motion information candidate list may be that for each additional selected motion information set, the constructing device determines whether the reference frame indexes of all control points in the additional selected motion information set are the same. When the reference frame indexes of all control points in the additional selected motion information set are the same, the constructing device stores the additional selected motion information set in the motion information candidate list. The constructing device does not determine the motion vectors of the control points in the set or perform motion vector scaling. This is different from the above step A and step B, and avoids the relatively high calculation complexity caused by motion vector scaling. In other words, in this scenario, the reference frame indexes of all control points in the additional selected motion information set that meets the preset condition are the same.

[0224] In a third optimal implementation form, after at least two additional selected motion information sets are generated, the method used by the construction device to sequentially scan at least two additional selected motion information sets and store the additional selected motion information sets that meet the preset condition in the motion information candidate list is as follows: for each additional selected motion information set, the construction device determines whether the reference frame indexes of all control points in the additional selected motion information set are the same. When the reference frame indexes of all control points in the additional selected motion information set are the same, the construction device determines whether the additional selected motion vectors of all control points in the additional selected motion information set are the same. If the additional selected motion vectors of all control points in the additional selected motion information set are different, the construction device stores the additional selected motion information set in the motion information candidate list. The construction device does not perform motion vector scaling. This is different from the above step A and step B, and avoids the relatively high calculation complexity caused by motion vector scaling. In other words, in this scenario, the reference frame indexes of all control points in the additional selected motion information set that meets the preset condition are the same, and the additional selected motion vectors of at least two control points in the additional selected motion information set that meets the preset condition are different.

[0225] Optionally, the additional selected motion information set generated by the constructing device may further include at least additional selected motion information of a first control point and additional selected motion information of a second control point, where the first control point and the second control point are neighboring control points of the current image block. In other words, the additional selected motion information set in this embodiment of the present application cannot only include additional selected motion information of diagonal control points of the current image block.

[0226] For example, referring to the above step B, the construction device of the present application may combine additional selected motion vectors of any two control points to construct a four-parameter affine motion model. The additional selected motion vectors of two control points may be combined as follows: (vcp1,vcp2), (vcp2,vcp4), (vcp1,vcp3), and (vcp3,vcp4). For example, in this embodiment of the present application, the four-parameter affine motion model constructed by using vcp1 and vcp2 is denoted as Affine(vcp1,vcp2).

[0227] The constructor combines the additional selected motion vectors of any three control points to construct a six-parameter affine motion model. The additional selected motion vectors of three control points can be combined as follows: (vcp1, vcp2, vcp4), (vcp1, vcp2, vcp3), (vcp2, vcp3, vcp4), and (vcp1, vcp3, vcp4). For example, in this embodiment of the present application, the six-parameter affine motion model constructed by using vcp1, vcp2, and vcp3 is denoted as Affine(vcp1, vcp2, vcp3).

[0228] The constructor combines the additional selected motion vectors of the aforementioned four control points to construct an eight-parameter affine motion model. For example, in this embodiment of the present application, the eight-parameter bilinear model constructed by using vcp1, vcp2, vcp3, and vcp4 is denoted as Bilinear(vcp1,vcp2,vcp3,vcp4).

[0229] The construction device sequentially determines the reference frame index and the additional selection target motion vector in each additional selection target motion information set in the order of Affine(vcp1,vcp2,vcp3) → Affine(vcp1,vcp2,vcp4) → Affine(vcp1,vcp3,vcp4) → Affine(vcp2,vcp3,vcp4) → Affine(vcp1,vcp2) → Affine(vcp1,vcp3) → Affine(vcp2,vcp4) → Affine(vcp3,vcp4).

[0230] S603: After all additional selection target motion information sets are scanned, if the amount of motion information candidate sets in the motion information candidate list is less than a predetermined amount value, the construction device fills the motion information candidate list with the first predetermined motion information candidate set until the amount of motion information candidate sets in the motion information candidate list is equal to the predetermined amount value.

[0231] In this embodiment of the present application, the first preset value is set so that the number of times of performing the first processing process is substantially limited. This reduces the calculation complexity and delay. In addition, the additional selection target motion information set may at least include the motion information of two neighboring control points, and the construction device no longer performs motion vector scaling. This further reduces the calculation complexity and delay.

[0232] 7 is another schematic flowchart of a method for constructing a motion information candidate list according to an embodiment of the present application. The method shown in FIG. 7 is executed by a constructing device. The constructing device may be the video decoder 202 or the video encoder 102 of FIG. 3.

[0233] As shown in FIG. 7, the method for constructing a motion information candidate list in this embodiment of the present application may include the following steps:

[0234] S701: The construction device sequentially executes a first processing process on all neighboring image blocks of a current image block in a first preset order until the amount of motion information candidate sets in a first motion information candidate list is equal to a first preset value or until all neighboring image blocks have been scanned.

[0235] For S701, please refer to the above description of S601, and the details will not be described again here.

[0236] S702: The construction device sequentially scans at least two additional selected motion information sets in a second preset order, and stores the additional selected motion information sets that satisfy the preset condition in the second motion information candidate list until the amount of motion information candidate sets in the second motion information candidate list is equal to a second preset value or until all the additional selected motion information sets have been scanned.

[0237] The additional selected motion information set includes additional selected motion information of at least two control points of the current image block, and the additional selected motion information of the control points is motion information of inter-coded image blocks adjacent to the current image block.

[0238] The reference frame indexes of all the control points in the additional selected motion information set that meets the preset condition are the same. Alternatively, the reference frame indexes of all the control points in the additional selected motion information set that meets the preset condition are the same, and the additional selected motion vectors of at least two control points in the additional selected motion information set that meets the preset condition are different.

[0239] For S702, please refer to the description of the second or third optimal implementation of S602 above. Details will not be described again here.

[0240] S703: The construction device merges the first motion information candidate list and the second motion information candidate list according to a preset rule to obtain a third motion information candidate list.

[0241] Optionally, the constructor may place the (partial) motion information candidate in the first motion information candidate list before or after the (partial) motion information candidate in the second motion information candidate list, which is not particularly limited in this embodiment of the present application.

[0242] Optionally, the construction device may directly merge the first motion information candidate list and the second motion information candidate list. Alternatively, the construction device may determine whether the same motion information candidate set exists in the first motion information candidate list and the second motion information candidate list. If the same motion information candidate set exists in the first motion information candidate list and the second motion information candidate list, the construction device deletes the repeated motion information candidate set.

[0243] In this specification, an explanation is given by using an example in which the construction device places all motion information candidates in a first motion information candidate list before all motion information candidates in a second motion information candidate list.

[0244] Specifically, the construction device places all the motion information candidates in the first motion information candidate list before all the motion information candidates in the second motion information candidate list to generate a target motion information candidate list. Then, the construction device determines whether the amount of the motion information candidate set in the target motion information candidate list is less than a preset amount value. If the amount of the motion information candidate set in the target motion information candidate list is less than a preset amount value, the construction device fills the target motion information candidate list with the second preset motion information candidate set until the amount of the motion information candidate set in the target motion information candidate list is equal to the preset amount value. In this manner, the construction device uses the target motion information candidate list obtained after filling as the third motion information candidate list. If the amount of the motion information candidate set in the target motion information candidate list is equal to the preset amount value, the construction device uses the target motion information candidate list as the third motion information candidate list. If the amount of motion information candidate sets in the target motion information candidate list is greater than a predetermined amount value, the construction device deletes at least one motion information candidate in the target motion information candidate list so that the amount of motion information candidate sets in the target motion information candidate list obtained after the deletion is equal to the predetermined amount value, and uses the target motion information candidate obtained after the deletion as a third motion information candidate list.

[0245] Of course, in addition to the above example, based on the value of the preset amount, the construction device may also select A motion information candidate sets from the first motion information candidate list, and select B motion information candidate sets from the second motion information candidate list to generate a third motion information candidate set including A motion information candidate sets and B motion information candidate sets. In this specification, the sum of A and B is equal to the value of the preset amount. In this embodiment of the present application, the method for generating the third motion information candidate list is not particularly limited.

[0246] It should be noted that the builder may execute S701 before S702, or may execute S702 before S701, or may execute S701 and S702 simultaneously, which is not particularly limited in this embodiment of the present application.

[0247] The construction device constructs the first motion information candidate list and the second motion information candidate list separately, and then integrates the first motion information candidate list and the second motion information candidate list. In the process of constructing the first motion information candidate list and the second motion information candidate list, the construction device limits the length of the first motion information candidate list and the second motion information candidate list. This substantially reduces the complexity and delay of calculation.

[0248] 8 is another schematic flowchart of a method for constructing a motion information candidate list according to an embodiment of the present application. The method shown in FIG. 8 is executed by a constructing device. The constructing device may be the video decoder 202 or the video encoder 102 of FIG. 3.

[0249] As shown in FIG. 8, the method for building a motion information candidate list in this embodiment of the present application may include the following steps.

[0250] S800: A construction device scans all neighboring image blocks of a current image block in a first preset order, and generates a motion information candidate list including at least one motion information candidate set.

[0251] It should be noted that in the process of scanning the neighboring positions of the current image block, it is further necessary to determine whether the image block where the scanned neighboring positions are located is an affine coding block. If the image block is an affine coding block, an affine coding block can be obtained. In addition, the control point motion information candidate of the current image block is derived based on the motion information of the control point of the affine coding block. If the image block where the neighboring positions are located is a translation block, the control point motion information candidate of the current image block cannot be derived based on the translation block. Therefore, in step S800, if the image block where the neighboring positions are located is a translation block, another neighboring position of the current image block needs to be continuously scanned until the image block where the scanned neighboring positions are located is an affine coding block.

[0252] The motion information candidate set includes motion information candidates for x control points of the current image block, where x is an integer equal to or greater than 2.

[0253] For the process of S800, please refer to the description of the above inherited control point motion vector prediction method or the description of Figure 1. The details will not be described again here.

[0254] S801: After scanning all neighboring image blocks of the current image block, the construction device sequentially scans at least two additional selection target motion information sets in a second preset order, and stores the additional selection target motion information sets that meet the preset condition in the motion information candidate list until the amount of motion information candidate sets in the motion information candidate list is equal to a preset amount value or until all the additional selection target motion information sets have been scanned.

[0255] For S801, please refer to the description of the second or third optimal implementation of S602 above. Details will not be described again here.

[0256] S802: After all additional selection target motion information sets are scanned, if the amount of motion information candidate sets in the motion information candidate list is less than a predetermined amount value, the construction device fills the motion information candidate list with a third predetermined motion information candidate set until the amount of motion information candidate sets in the motion information candidate list is equal to the predetermined amount value.

[0257] For S802, please refer to the above description of S603, and the details will not be described again here.

[0258] When processing the additional selected motion information set, the construction device only determines the reference frame index and the additional selected motion vector of all control points in the additional selected motion information set, and determines whether the additional selected motion information set should be stored in the motion information candidate list according to the determination result.The construction device does not need to perform motion vector scaling, and does not need to first calculate a new motion information candidate set and then determine whether the same motion information exists in the motion information candidate list.This reduces the calculation complexity and delay.

[0259] In addition, the embodiment of the present application further provides an inter prediction method. Specifically, the inter prediction device obtains and parses a bitstream to obtain a first index value and a motion information difference, the first index value being an index value of a motion information predictor of a control point of a current image block in a motion information candidate list, and the motion information candidate list is generated by the aforementioned construction device by using the construction method described in any one of Figures 6 to 8. The inter prediction device obtains a motion information predictor of a control point from the motion information candidate list based on the first index value. The inter prediction device determines motion information of a control point based on the motion information predictor of the control point and the motion information difference. The inter prediction device determines a prediction pixel of a current image block based on the motion information of the control point.

[0260] It is easy to understand that the inter prediction method is basically to generate a motion information candidate list according to the construction method provided in the embodiment of the present application, and to perform inter prediction based on the motion information candidate list by using a pre-set affine AMVP mode by the decoder side.

[0261] The embodiment of the present application further provides an inter prediction method. Specifically, the inter prediction device obtains and parses a bitstream to obtain a second index value, the second index value being an index value of the motion information of the control point of the current image block in the motion information candidate list, and the motion information candidate list is generated by the aforementioned construction device by using the construction method described in any one of Figures 6 to 8. The inter prediction device obtains the motion information of the control point from the motion information candidate list according to the second index value. The inter prediction device determines a prediction pixel of the current image block according to the motion information of the control point.

[0262] It is easy to understand that the inter prediction method is basically to generate a motion information candidate list according to the construction method provided in the embodiment of the present application, and to perform inter prediction based on the motion information candidate list by using the affine merge mode at the decoder side.

[0263] An embodiment of the present application provides an apparatus for constructing a motion information candidate list. The constructing apparatus may be a video decoder, a video encoder, or a decoder. Specifically, the apparatus for constructing a motion information candidate list is configured to perform the steps performed by the constructing apparatus in the above-mentioned method for constructing a motion information candidate list. The constructing apparatus provided in this embodiment of the present application may include modules corresponding to the corresponding steps.

[0264] In the embodiment of the present application, the device for constructing a motion information candidate list can be divided into functional modules based on the above-mentioned method example. For example, each functional module can be obtained through division based on corresponding function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware, or in the form of software functional module. In the embodiment of the present application, the division into modules is an example, and is merely a logical division of functions, and may be other divisions in actual implementation.

[0265] When each functional module is obtained through division based on the corresponding function, Fig. 9 is a possible schematic structural diagram of an apparatus for constructing a motion information candidate list in the above embodiment. As shown in Fig. 9, the apparatus 9 for constructing a motion information candidate list includes a first processing unit 90, a second processing unit 91, a filling unit 92, a merging unit 93 and a storage unit 94.

[0266] The first processing unit 90 is configured to build a motion information candidate list to support an apparatus for performing S601, S701, S801, etc. in the above-mentioned embodiments and / or for use in another process of the techniques described in this specification.

[0267] The second processing unit 91 is configured to build a motion information candidate list to support an apparatus for performing S602, S702, S802, etc. in the above-mentioned embodiments, and / or is used in another process of the techniques described in this specification.

[0268] The filling unit 92 is configured to build a motion information candidate list to support an apparatus for performing S603, S802, etc. in the above-mentioned embodiments and / or for use in another process of the techniques described herein.

[0269] The merging unit 93 is configured to build a motion information candidate list to support an apparatus for performing, for example, S703 in the above embodiment, and / or for use in another process of the techniques described herein.

[0270] The storage unit 94 may be configured to store program codes and data of the apparatus for building the motion information candidate list, and may be further configured to store the motion information candidate list.

[0271] All the relevant contents of the steps in the above method embodiments may be cited in the functional descriptions of the corresponding functional modules, and the details will not be described again here.

[0272] When an integrated unit is used, FIG. 10 is a schematic structural diagram of an apparatus for constructing a motion information candidate list provided in an embodiment of the present application. In FIG. 10, the apparatus 10 for constructing a motion information candidate list includes a processing module 100 and a communication module 101. The processing module 100 is configured to control and manage the behavior of the apparatus for constructing a motion information candidate list, for example, to execute the steps executed by the first processing unit 90, the second processing unit 91, the filling unit 92, and the merging unit 93, and / or to execute another process of the technique described herein. The communication module 101 is configured to support the interaction between the apparatus for constructing a motion information candidate list and another device. As shown in FIG. 10, the apparatus for constructing a motion information candidate list may further include a storage module 102. The storage module 102 is configured to store the program code and data of the apparatus for constructing a motion information candidate list, for example, to store the contents stored by the storage unit 94.

[0273] The processing module 100 may be a processor or controller, for example, a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an ASIC, an FPGA or another programmable logic device, a transistor logic device, a hardware component, or any set thereof. The processing module 100 may implement or execute various exemplary logic blocks, modules, and circuits described with reference to the contents disclosed in this application. The processor may alternatively be a set for implementing a computation function, for example, a set including one or more microprocessors, or a combination of a DSP and a microprocessor. The communication module 101 may be a transceiver, an RF circuit, a communication interface, etc. The storage module 102 may be a memory.

[0274] All relevant contents of the scenarios in the above method embodiments can be cited in the functional description of the corresponding functional modules, and the details will not be described again here.

[0275] Both the apparatus 9 for constructing a motion information candidate list and the apparatus 10 for constructing a motion information candidate list may execute the method for constructing a motion information candidate list shown in Figures 6 to 8. The apparatus 9 for constructing a motion information candidate list and the apparatus 10 for constructing a motion information candidate list may be specifically a video decoding apparatus or another device having a video coding function, respectively. The apparatus 9 for constructing a motion information candidate list and the apparatus 10 for constructing a motion information candidate list may be configured to perform image prediction in a decoding process.

[0276] An embodiment of the present application provides an inter prediction device. The inter prediction device may be a video decoder, a video encoder, or a decoder. Specifically, the inter prediction device is configured to perform steps performed by the inter prediction device in the aforementioned inter prediction method. The inter prediction device provided in the embodiment of the present application may include modules corresponding to the corresponding steps.

[0277] In the embodiment of the present application, the inter-prediction device may be divided into functional modules according to the above-mentioned method examples. For example, each functional module may be obtained through division based on corresponding functions, or two or more functions may be integrated into one processing module. The integrated module may be implemented in the form of hardware, or may be implemented in the form of software functional modules. In the embodiment of the present application, the division into modules is an example, and is merely a logical division of functions, and may be other divisions in actual implementation.

[0278] When each functional module is obtained through division based on the corresponding function, FIG. 11 is a possible schematic structural diagram of the inter-prediction device in the above embodiment. As shown in FIG. 11, the inter-prediction device 11 includes an acquisition unit 110, a decision unit 111, and a storage unit 112.

[0279] The acquisition unit 110 is configured to support an inter-prediction device to perform the aforementioned steps such as "obtaining and parsing a bitstream to obtain a first index value and a motion information differential," "obtaining a motion information predictor for a control point from a motion information candidate list based on the first index value," "obtaining and parsing a bitstream to obtain a second index value," and "obtaining motion information for a control point from a motion information candidate list based on the second index value," and / or is used in another process of the techniques described in this specification.

[0280] The determination unit 111 is configured to support an inter-prediction device and / or is used in another process of the techniques described in this specification to perform the aforementioned steps such as "determine motion information of a control point based on an obtained motion information predictor of the control point and an obtained motion information differential", "determine a predicted pixel of a current image block based on the motion information of the control point", and "determine a predicted pixel of a current image block based on the motion information of the control point".

[0281] Storage unit 112 may be configured to store program codes and data for the inter prediction device.

[0282] All the relevant contents of the steps in the above method embodiments can be cited in the functional description of the corresponding functional modules, and the details will not be described again here.

[0283] When an integrated unit is used, FIG. 12 is a schematic structural diagram of an inter prediction device provided in an embodiment of the present application. In FIG. 12, the inter prediction device 12 includes a processing module 120 and a communication module 121. The processing module 120 is configured to control and manage the behavior of the inter prediction device, for example, to perform the steps performed by the obtaining unit 110 and the determining unit 111, and / or to perform another process of the technology described herein. The communication module 121 is configured to support the interaction between the inter prediction device and another device. As shown in FIG. 12, the inter prediction device may further include a storage module 122. The storage module 122 is configured to store the program code and data of the inter prediction device, for example, to store the content stored by the storage unit 112.

[0284] The processing module 120 may be a processor or controller, such as a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA or another programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The processing module 120 may implement or execute various exemplary logic blocks, modules, and circuits described with reference to the disclosed subject matter in this application. The processor may alternatively be a combination for implementing a computational function, such as a combination including one or more microprocessors, or a combination of a DSP and a microprocessor. The communication module 121 may be a transceiver, an RF circuit, a communication interface, etc. The storage module 122 may be a memory.

[0285] All relevant contents of the scenarios in the above method embodiments can be cited in the functional descriptions of the corresponding functional modules, and the details will not be described again here.

[0286] Both the inter prediction device 11 and the inter prediction device 12 may perform the above-mentioned inter prediction method. The inter prediction device 11 and the inter prediction device 12 may be specifically a video decoding device or another device having a video coding function. The inter prediction device 11 and the inter prediction device 12 may be configured to perform image prediction in the decoding process.

[0287] The present application further provides a terminal. The terminal includes one or more processors, a memory, and a communication interface. The memory and the communication interface are connected to the one or more processors. The memory is configured to store a computer program code. The computer program code includes instructions. When the one or more processors execute the instructions, the terminal executes the method for constructing a motion information candidate list or the inter-prediction method in the embodiment of the present application.

[0288] A terminal herein may be a video display device, a smart phone, a portable computer, or another device capable of processing or playing video.

[0289] The present application further provides a video decoder, including a non-volatile storage medium and a central processing unit. The non-volatile storage medium stores an executable program. The central processing unit is connected to the non-volatile storage medium and executes the executable program to perform the method for constructing a motion information candidate list and the inter-prediction method in the embodiment of the present application.

[0290] The present application further provides a decoder. The decoder includes an apparatus for constructing a motion information candidate list in an embodiment of the present application (apparatus 9 for constructing a motion information candidate list or apparatus 10 for constructing a motion information candidate list), an inter-prediction apparatus in an embodiment of the present application (inter-prediction apparatus 11 or inter-prediction apparatus 12), and a reconstruction module. The reconstruction module is configured to determine a reconstructed pixel value of a current image block according to a prediction pixel obtained by the inter-prediction apparatus.

[0291] Another embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium includes one or more program codes. The one or more programs include instructions. When a processor of a terminal executes the program codes, the terminal executes the method for constructing a motion information candidate list in any one of Figures 6 to 8, or executes an inter-prediction method.

[0292] In another embodiment of the present application, a computer program product is further provided. The computer program product includes computer-executable instructions, and the computer-executable instructions are stored in a computer-readable storage medium. At least one processor of a terminal can read the computer-executable instructions from the computer-readable storage medium. The at least one processor executes the computer-executable instructions to enable the terminal to execute the method for constructing a motion information candidate list in any one of Figures 6 to 8 or to execute an inter-prediction method.

[0293] All or part of the above-mentioned embodiments may be implemented by using software, hardware, firmware, or any combination thereof. When a software program is used to implement the embodiments, the embodiments may be fully or partially implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded into a computer and run on the computer, the procedures or functions according to the embodiments of the present application are generated in whole or in part.

[0294] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or another programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired (e.g., coaxial cable, optical fiber, or digital subscriber line (DSL)) or wireless (e.g., infrared, radio, or microwave) manner. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device, such as a server or data center, that incorporates one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a DVD), a semiconductor medium (e.g., a solid-state disk, SSD), etc.

[0295] The above description of the implementation form allows those skilled in the art to clearly understand that for convenience and concise description, the above division into functional modules is regarded as an example for explanation. In actual application, the above functions may be allocated and implemented into different functional modules based on requirements, that is, the internal structure of the device is divided into different functional modules to implement all or part of the above-described functions.

[0296] In some embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the described apparatus embodiments are merely examples. For example, the division into modules or units is merely a division of logical functions, and in actual implementation, other divisions may be possible. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not implemented. In addition, the shown or discussed mutual couplings or direct couplings or communication connections may be implemented by using some interface. Indirect couplings or communication connections between apparatuses or units may be implemented in electronic, mechanical, or other forms.

[0297] The units described as separate parts may or may not be physically separated, and the parts shown as units may be one or more physical units, located in one place or distributed in different places. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0298] In addition, the functional units in the embodiments of the present application may be integrated into one processing unit, or each of the units may exist physically alone, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0299] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be implemented in the form of a software product basically, or the part contributing to the prior art or all or part of the technical solutions may be implemented in the form of a software product. The software product is stored in a storage medium and includes some instructions for instructing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method in the embodiments of the present application. The aforementioned storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0300] The above description is merely a specific implementation form of the present application, and is not intended to limit the protection scope of the present application. Any variation or replacement within the technical scope disclosed in this application shall be within the protection scope of the present application. Therefore, the protection scope of the present application shall depend on the protection scope of the claims. [Explanation of symbols]

[0301] 9. Apparatus for constructing a candidate list of motion information 10 Source Device 11 Inter-prediction device 12 Inter-prediction device 20 Destination Device 30 Links 40 Storage device 90 First Processing Unit 91 Second Processing Unit 92 Filling Unit 93 Merged Units 94 Storage Unit 101 Video Sources 102 Video Encoder 103 Output Interface 110 Acquired Units 111 Decision Unit 112 Storage Unit 120 Processing Module 121 Communication Module 122 Memory Module 201 Display device 202 Video Decoder 203 Input Interface 301 Converter 302 Quantizer 303 Entropy Encoder 304 Inverse quantizer 305 Inverse Converter 306 Filters 307 Memory 308 Prediction Processing Unit 309 Intra Predictor 310 Inter Predictor 311 Adder 312 Adder 401 Entropy Encoder 402 Inverse quantizer 403 Inverse converter 404 Filter 405 Memory 406 Prediction Processing Unit 407 Intra Predictor 408 Inter Predictor

Claims

1. 1. A method for constructing a motion vector candidate list, comprising: generating a motion information candidate list including at least one motion information candidate set, the at least one motion information candidate set being obtained from neighboring affine image blocks of the current image block by using an inherited control point motion vector prediction method, the motion information candidate set including motion information candidates of x control points of the current image block, where x is an integer equal to or greater than 2; storing additional selection target motion information sets that satisfy a preset condition in the motion information candidate list until an amount of motion information candidate sets in the motion information candidate list is equal to a preset amount value, the additional selection target motion information sets including additional selection target motion information of at least two control points of the current image block, the additional selection target motion information of the control points being motion information of inter-coded image blocks adjacent to the current image block; The reference frame indexes of all control points in the additional selection target motion information set that satisfies the preset condition are the same; when the amount of the motion information candidate set in the motion information candidate list is less than the predetermined amount value, filling the motion information candidate list with a third predetermined motion information candidate set until the amount of the motion information candidate set in the motion information candidate list is equal to the predetermined amount value; A construction method comprising:

2. The construction method of claim 1 , wherein the additional selected motion information set includes at least additional selected motion information of a first control point and additional selected motion information of a second control point, the first control point and the second control point being neighboring control points of the current image block.

3. The construction method according to claim 1 or 2, wherein the neighbouring affine image blocks of the current image block are located at a number of predefined positions or at one predefined position to the left and / or above the current image block.

4. The construction method of claim 1 or 2, wherein the additional set of selectable motion information includes the additional selectable motion information of the at least two control points of the current image block, the additional selectable motion information of each control point among the at least two control points originates from an image block at a specific position adjacent to the control point, and an inter prediction mode is used for the image block at the specific adjacent position.

5. obtaining and parsing a bitstream to obtain a first index value and a motion information difference, the first index value being an index value of a motion information predictor of a control point of a current image block in a motion information candidate list, the motion information candidate list being generated by using the construction method according to any one of claims 1 to 4; obtaining the motion information predictor of the control point from the motion information candidate list based on the first index value; determining motion information of the control point based on the motion information predictor and the motion information differential of the control point; determining a prediction pixel of the current image block based on the motion information of the control points; The inter prediction method has the following features:

6. obtaining and parsing a bitstream to obtain a second index value, the second index value being an index value of motion information of a control point of a current image block in the motion information candidate list, the motion information candidate list being generated by using the construction method according to any one of claims 1 to 4; obtaining the motion information of the control point from the motion information candidate list based on the second index value; determining a prediction pixel of the current image block based on the motion information of the control points; The inter prediction method has the following features:

7. 1. An apparatus for constructing a motion vector candidate list, comprising: A first processing unit configured to generate a motion information candidate list including at least one motion information candidate set, the at least one motion information candidate set being obtained from neighboring affine image blocks of a current image block by using an inherited control point motion vector prediction method, the motion information candidate set including motion information candidates of x control points of the current image block, the motion information candidate list being stored in a storage unit, x being an integer equal to or greater than 2; a second processing unit configured to store an additional selection target motion information set that satisfies a preset condition in the motion information candidate list until an amount of the motion information candidate set in the motion information candidate list is equal to a preset amount value, the additional selection target motion information set including additional selection target motion information of at least two control points of the current image block, the additional selection target motion information of the control points being motion information of an inter-coded image block adjacent to the current image block; A second processing unit, in which the reference frame indexes of all control points in the additional selection target motion information set that meets the preset condition are the same; a filling unit configured to fill the motion information candidate list with a third preset motion information candidate set when an amount of the motion information candidate set in the motion information candidate list is less than the preset amount value, until the amount of the motion information candidate set in the motion information candidate list is equal to the preset amount value; An apparatus comprising:

8. The construction device of claim 7, wherein the additional selected motion information set includes at least additional selected motion information of a first control point and additional selected motion information of a second control point, the first control point and the second control point being neighboring control points of the current image block.

9. 9. The construction apparatus according to claim 7 or 8, wherein the neighbouring affine image blocks of the current image block are located at a plurality of predefined positions or at one predefined position to the left and / or above the current image block.

10. The construction device of claim 7 or 8, wherein the additional set of selectable motion information includes the additional selectable motion information of the at least two control points of the current image block, the additional selectable motion information of each control point among the at least two control points originating from an image block at a specific position adjacent to the control point, and an inter prediction mode is used for the image block at the specific adjacent position.

11. An acquisition unit configured to acquire and parse a bitstream to acquire a first index value and a motion information differential, the first index value being an index value of a motion information predictor of a control point of a current image block in a motion information candidate list, the motion information candidate list being generated by a construction device according to any one of claims 7 to 10; an acquisition unit, the acquisition unit being further configured to acquire the motion information predictor of the control point from the motion information candidate list based on the first index value; a determining unit configured to determine motion information of the control points based on the motion information predictor and the motion information differential of the control points obtained by the obtaining unit, a determining unit, the determining unit being further configured to determine a predicted pixel of the current image block based on the motion information of the control points; An inter prediction device comprising:

12. an obtaining unit configured to obtain and parse a bitstream to obtain a second index value, the second index value being an index value of motion information of a control point of a current image block in a motion information candidate list, the motion information candidate list being generated by a construction device according to any one of claims 7 to 10; an acquisition unit, the acquisition unit being further configured to acquire the motion information of the control point from the motion information candidate list based on the second index value; a determining unit configured to determine a predicted pixel of the current image block based on the motion information of the control points obtained by the obtaining unit; An inter prediction device comprising:

Images