Method and device for candidate list pruning

Abstract

To provide a coding and decoding method which improves the performance of a coding tool.SOLUTION: Video signal coding and decoding functions generate a list of potential candidates, e.g., predictors which are used for coding and decoding. Video signal coding component candidates are operated prior to potential registration in the candidate list. The candidates are changed by the operation and then checked in order to confirm the other equal candidate has already existed in the candidate list. In a case where no equal candidate exists in the list, the changed candidate is added to the candidate list. In a case where an equal candidate has already existed in the list, the changed candidate is not added to the list. The operation to be performed includes rounding and clipping.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] FIELD OF THE INVENTION The present principles relate to the field of video compression. [Background technology]

[0002] Background of the Invention Video coding organizations are striving to develop more efficient coding standards and tools. Some of these standards are High Efficiency Video Coding (ISO / IEC ITU-T) H.265 (HEVC), MPEG-4 Part 10, Advanced Video Coding (MPEG-4 AVC), and Joint Exploration Test Model (JEM) or H.266. The tool builds a list of possible candidates for things like motion vector prediction and intra prediction, with the goal of finding the best candidate in terms of RDO (Rate Distortion Optimization).

[0003] By ensuring uniqueness of candidates within these lists, the performance of tools with lists of limited size can be improved by adding diversity within the lists, and / or the complexity of tools with unlimited lists can be reduced by, for example, removing identical candidates within these lists.

[0004] Under the aspects described herein, several embodiments are described relating to candidate list pruning. Summary of the Invention

[0005] Summary of the Invention These and other drawbacks and disadvantages of the prior art are addressed by aspects of the present invention, which are directed to methods and apparatus for candidate list pruning to improve coding efficiency and / or reduce the complexity of a video codec by ensuring uniqueness of the different candidates used.

[0006] Most of these video codec tools build a list of candidates before selecting the best one. To improve their performance and / or reduce their complexity, the candidates in these lists should be unique to ensure full diversity with a minimum number of candidates.

[0007] Candidate pruning can be generic so that it is applied systematically to all tools, or it can be specific to each toolset.

[0008] According to another aspect of the described embodiments, there is provided a method for encoding or decoding a video signal using candidate list pruning, the method including generating at least one modified candidate by performing at least one operation on candidate coding components of the video signal, the method further including determining whether the at least one modified candidate is already present in the candidate list, and adding the at least one modified candidate if the at least one modified candidate is not previously present in the candidate list.

[0009] According to another aspect of the described embodiments, there is provided an apparatus for encoding or decoding a video signal using candidate list pruning. The apparatus includes a memory and a processor. and a processor configured to encode or decode the video signal using the candidate list by performing at least one operation on candidate coding components of the video signal to generate at least one modified candidate, determining whether the at least one modified candidate is already present in the candidate list, and adding the at least one modified candidate if the at least one modified candidate was not previously present in the candidate list.

[0010] According to another aspect of the described embodiments, there is provided a non-transitory computer-readable medium containing data content generated according to any of the encoding method embodiments or by any of the encoder apparatus embodiments, for playback using a processor.

[0011] According to another aspect of the described embodiments, there is provided a signal including video data generated according to any of the encoding method embodiments or by any of the encoder apparatus embodiments, for playback using a processor.

[0012] According to another aspect of the described embodiments, there is provided a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method described in any one of the decoding method embodiments or the decoding apparatus embodiments.

[0013] These and other aspects, features and advantages of the present principles will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings. [Brief explanation of the drawings]

[0014] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1] The AMVP derivation process of two motion vector predictors is shown. [Figure 2]1 shows the merge derivation process of seven predictors, their associated reference frames in the reference frame list, and IC flags. [Figure 3] 1 illustrates the affine AMVP derivation process of two affine pairs of motion vector predictors for a particular reference frame in a particular reference frame list. [Figure 4] 1 illustrates the AMVP derivation process of two unique motion vector predictors for a particular reference frame in a particular reference frame list. [Figure 5] 1 illustrates the first two steps of the AMVP derivation process for two unique motion vector predictors for a particular reference frame in a particular reference frame list. [Figure 6] 1 illustrates the first two steps of the AMVP derivation process for two unique motion vector predictors for a particular reference frame in a particular reference frame list. [Figure 7] 1 shows the merge derivation process of seven unique predictors, their associated reference frames in the reference frame list, and IC flags. [Figure 8] 1 illustrates the affine AMVP derivation process of two unique affine pairs of motion vector predictors for a particular reference frame in a particular reference frame list. [Figure 9] 1 illustrates a general encoder to which the described embodiments may be applied. [Figure 10] 1 illustrates a general decoder to which the described embodiments can be applied. [Figure 11] 1 illustrates one embodiment of a method used to encode or decode a video signal using candidate list pruning. [Figure 12] 1 illustrates one embodiment of an apparatus that may be used to encode or decode a video signal using candidate list pruning. DETAILED DESCRIPTION OF THE INVENTION

[0015] Detailed Description SUMMARY A candidate list pruning technique for video encoding and decoding is described herein.

[0016] Most video codec tools build a list of candidates before selecting the best one to use in the coding or decoding process. These candidates are potential video coding components used in coding or decoding a video signal. Video components may be, for example, motion vectors, predictions, quantization parameters, and other elements. To improve their performance and / or reduce their complexity, the candidates in these lists should be unique to ensure full diversity with a minimum number of candidates. Candidate pruning can be generic in that it can be systematically applied to all tools, or it can be specific to each toolset. Some such tools using lists are presented below, although the general aspects described herein are not limited to these tools.

[0017] 1. AMVP Prediction AMVP (Advanced Motion Vector Prediction) prediction is performed for each reference frame list (L The goal is to build a list of two predictors for each reference frame (L0 and L1). These two AMVP predictors consist of motion vectors.

[0018] For a particular reference frame of a particular reference frame list, predictors are derived from spatial and / or temporal neighboring blocks (coding units / prediction units). Further operations such as Frame Rate Up Conversion (FRUC) derivation (i.e., motion vector refinement), clipping, and / or rounding operations may then be performed on these predictors. Such a predictor derivation process is as shown in Figure 1.

[0019] 2. Merge Prediction Merge prediction consists in building a set of seven motion vector predictors with associated reference frames and reference frame lists, and corresponding IC flags (ie local illumination compensation flags).

[0020] These seven merge predictors consist of: -Motion Vector - Reference Frame Index - Reference Frame List -IC flag

[0021] Predictors are derived from spatial and / or temporal neighborhoods from which motion vectors and associated reference frames and IC flags are chosen. Further operations, such as clipping and / or rounding operations, can then be performed on these predictors.

[0022] Such a predictor derivation process is as shown in FIG. 2, and the process stops as soon as seven candidates are obtained.

[0023] 3.FRUC Prediction The Frame Rate Up Conversion (FRUC) process uses merge predictors as described in the previous section, but does not use the IC flag information of these predictors. The FRUC merge predictor derived from the merge candidates consists of: -Motion Vector - Reference Frame Index - Reference Frame List

[0024] The predictor is a spatial and / or spatial vector from which the motion vector and associated reference frame are selected. are derived from temporal neighbors. Further operations, such as clipping and / or rounding operations, can then be performed on these predictors. The process of deriving such a predictor is as shown in FIG.

[0025] 4. Affine AMVP Prediction Affine AMVP prediction is quite similar to the one in classical AMVP, but affine motion estimation and compensation is performed using two motion vectors, later called affine motion vector pairs, which consist in finding two affine motion vector predictor pairs for each reference frame in each reference frame list (L0 and L1). The affine AMVP predictor consists of: -Motion Vector Pair

[0026] For a particular reference frame of a particular reference frame list, predictors are derived from spatial and / or temporal neighboring blocks, and further operations such as clipping and / or rounding operations may then be performed on these predictors. The process of deriving such a predictor is shown in FIG.

[0027] 5. Intra prediction Intra prediction consists in testing several directional predictions (9 in AVC, up to 35 in HEVC, and up to 67 in JEM) to find the best one. In more recent codecs, several new tools have been added to intra prediction, such as MPM (Most Probable Mode), RSAF (Reference Sample Adaptive Filter), and PDPC (Position Dependent intra Prediction Combination).

[0028] In either case, some or all predictions are tested, and thus this reduction in the number of predictions may reduce complexity.

[0029] The main aspects of the described embodiments are as follows: During the construction of any candidate list, each time a new candidate is added to the list, -Perform all operations that may change this candidate, -Compare this new candidate with all already selected candidates in the list, -If it is not already there, add it to the list.

[0030] This is done to diversify the list by ensuring the uniqueness of each candidate, or to reduce the list size and the resulting complexity of dealing with it.

[0031] The following embodiment describes how to apply this to each coding mode (1: AMVP, 2: Merge, 3: FRUC, 4: Affine, 5: Intra).

[0032] In the first embodiment, the diversity of the AMVP-generated list of predictors can be increased in several ways.

[0033] In a first sub-embodiment of the first embodiment (AMVP scheme), operations that may modify candidates should be applied just before adding them to the list of predictors.

[0034] In the scheme of Figure 1, these operations represent rounding operations (quarter-pixel and integer motion vectors), but can be extended to future operations (clipping, other rounding, etc.) These operations should be performed on candidates together after they have been collected and before they are added to a list if not already present, as shown in Figure 4.

[0035] The "compare / add" function consists in comparing the retrieved candidate with the already selected candidate. Since the AMVP predictor consists only of motion vectors, this function compares the motion vectors.

[0036] In a second sub-embodiment of the first embodiment (the AMVP scheme of FIGS. 1 and 4), the second spatial candidate does not use information about the first selected spatial candidate (if any).

[0037] In that case, it is possible to inform the second spatial motion vector search of the first selected motion vector predictor. In this case, the first step of this scheme would be as shown in Figure 5.

[0038] Since the operation to obtain a spatial candidate can find it in several ways and stops as soon as it is found, the "different" function represents a comparison of the potential candidate against the already selected candidate to ensure different predictor selection.

[0039] When the first two sub-embodiments are combined, it is also possible to perform an operation to change the motion vector predictor (rounding) as soon as a candidate is obtained in order to maximize the number of different candidates, as shown in Figures 6 and 4.

[0040] In a second embodiment, the diversity of the merged generated list of predictors can be improved by comparing each new candidate with previously selected candidates before adding it to the list only if it is not already present.

[0041] Figure 7 illustrates this embodiment in comparison to Figure 2. In this figure, the "compare / add" function consists in (i) comparing the just-fetched predictor with the already selected predictors, and (ii) adding it if it is not already present.

[0042] Since the merge predictor consists of a motion vector, a reference frame index in a reference frame list, and an IC flag, the comparison involves testing these four components.

[0043] In a third embodiment, the diversity of FRUC candidates derived from merging candidates can be improved by (i) using the same improvements as for merging (i.e., as shown in Figure 7), and (ii) removing the IC flag from the merging process of Figure 7.

[0044] FRUC should use the process of the second embodiment shown in FIG. 7 to inherit candidate diversity from merging enhancement.

[0045] To remove the IC flag, the comparisons made in the "compare / add" function and the "differ" function of spatial candidates should only look at three characteristics of the FRUC predictor (motion vector, reference frame index in the reference frame list) instead of the four characteristics of the merge predictor. To do this, these functions need to be informed of the mode used (merge or FRUC).

[0046] In a fourth embodiment, the diversity of the affine AMVP predictors can also be improved by ensuring a list of unique predictors.

[0047] To do this, we first select the predictors before choosing the affine pairs to ensure full diversity. All possible modifications are made. Figure 8 illustrates this process, in which all triplets of the motion vector predictor are rounded and clipped before selecting the two best triplets. The two best triplets may be, for example, the first two different triplets with lower costs (although other criteria may be used to select which triplets should be added as candidates).

[0048] If not enough affine predictors are found, the added AMVP candidates should also be processed using a "compare / add" function, which takes care of comparing the candidate affine pair with a previously selected one before adding it, if it is not already present. The affine AMVP list may not be complete, i.e., filled with two unique affine pairs of motion vectors, since some affine pair candidates may have been removed, for example, because they are the same as those already selected. It may be useful to fill this list with pairs of zero motion vectors.

[0049] In a fifth embodiment, the number of intra predictors can be reduced by removing duplicates.

[0050] Some reference samples used for directional prediction may be equal because they are the same in the predicted frame or because they are not available and are filled in with the first available neighbors.

[0051] In this case, several directions may yield the same prediction that can be easily inferred, in which case it is possible to keep only one of these directions in the candidate test set.

[0052] For example, on the left edge of the frame, all left reference samples are filled with the first top sample. The horizontal prediction gives the same prediction as the full left bottom prediction, which can be removed from the tested candidate set. In one example, this is mode 8 for AVC, modes 2-9 for HEVC, and modes 2-18 for JEM (Joint Exploration Test Model).

[0053] In the sixth embodiment, some of the previous embodiments may be combined together. In the case of AVC and HEVC, only the first two sub-embodiments of the first embodiment, the second embodiment, and the fifth embodiment may be applied. And in the case of JEM, all embodiments may be used. In both cases, any combination of the available embodiments may be observed.

[0054] In another embodiment, several embodiments are combined in JEM as follows:--For AMVP: the first two sub-embodiments of the first embodiment, as illustrated by Figure 4 and Figure 5 respectively. -Merge case: second embodiment illustrated by FIG. For FRUC merge candidates: A third embodiment as shown in FIG. 7 associated with a notified "compare / add" function that removes the IC flag test. For affine AMVP: fourth embodiment as shown in FIG.

[0055] The described embodiment can be applied in both the encoder and the decoder. During encoding, all candidates are collected and compared in several RDO loops to isolate the best one. During decoding, candidates are collected in the same way until all possibilities are exhausted or the best encoded one is found. For example: In AMVP, candidates are collected for each reference frame at encoding time, and at decoding time, the reference frame is known (transmitted) and candidates are collected for this particular reference frame in exactly the same way as at encoding time. -Merge: During encoding, seven candidates are collected and the index of the best candidate is sent, and during decoding, candidates are collected in the same way until the index of the searched candidate is obtained. Therefore, these processes of gathering such candidates are the same on both the encoding and decoding sides. To speed up the decoder, the decoder may stop these processes as soon as a desired candidate is obtained.

[0056] 11 shows one embodiment of a method 1100 for encoding or decoding blocks of video data using candidate list pruning. The method begins at start block 1101 and proceeds to block 1110, which performs at least one operation on coding component candidates to generate one or more modified coding component candidates. Control then passes from block 1110 to block 1120, which determines whether one or more modified coding component candidates are already present in the candidate list. Control then passes from block 1120 to block 1130, which adds at least one modified candidate to the candidate list if it is not already present.

[0057] 12 shows an embodiment of an apparatus 1200 for coding or decoding blocks of video data. The apparatus includes a processor 1210 having input and output ports and is in signal communication with a memory 1220 also having input and output ports. The apparatus can perform any of the method embodiments described above or variations for encoding or decoding video signal data using candidate list pruning.

[0058] The functions of the various elements shown in the figures may be provided through the use of dedicated hardware and hardware capable of executing software in association with appropriate software. If provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by multiple individual processors, some of which may be shared. Additionally, explicit use of the terms "processor" or "controller" should not be construed to refer solely to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor ("DSP") hardware, read-only memory ("ROM") for storing software, random access memory ("RAM"), and non-volatile storage.

[0059] Other hardware, conventional and / or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their functionality may be performed by the operation of program logic, by dedicated logic, by the interaction of program control and dedicated logic, or manually, with the particular technique being selectable by the implementer as more particularly understood from the context.

[0060] This specification illustrates the principles of the present invention. It will thus be recognized that those skilled in the art can devise various arrangements which, although not explicitly described or shown herein, embody the principles of the present invention and are included within its spirit and scope.

[0061] All examples and conditional language set forth in this specification are for educational purposes to aid the reader in understanding the principles of the present invention and concepts contributed by the inventors to advance the art, and are not to be construed as being limited to such specifically set forth examples and conditions.

[0062] Also, principles, aspects, and embodiments of the present invention, as well as specific examples thereof, are described. All statements herein are intended to encompass both structural and functional equivalents thereof. Additionally, such equivalents are intended to include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure).

[0063] Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the principles of the invention. Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudocode, etc., are substantially represented on a computer-readable medium and, as such, represent various processes that may be executed by a computer or processor (whether or not such computer or processor is explicitly shown).

[0064] In the claims herein, any element expressed as a means for performing a certain function is intended to encompass any way of performing that function, including, for example, a) a combination of circuit elements that perform that function, or b) software in any form (thus including firmware, microcode, etc.) in combination with appropriate circuitry that executes the software to perform that function. The principles of the invention as defined by such claims reside in the fact that the functionality provided by the various recited means are combined and brought together in the manner the claims call for. Thus, any means that can provide those functionality are regarded as equivalent as shown herein.

[0065] References herein to "one embodiment" or "an embodiment" of the present principles, as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with that embodiment is included in at least one embodiment of the present principles. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment," as well as other variations thereof, appearing in various places throughout this specification are not necessarily all referring to the same embodiment.

[0066] In conclusion, candidate list pruning for video coding and decoding is provided by at least the described embodiments, the purpose of which is to find the best candidate in terms of RDO among various candidates during video encoding and decoding.

[0067] By ensuring the uniqueness of the candidates in these lists, the performance of tools with lists of limited size can be improved by adding diversity within these lists, and / or the complexity of tools with unlimited lists can be reduced by, for example, removing identical candidates within these lists.

Claims

1. generating at least one modified affine prediction candidate by performing at least one of a plurality of rounding operations on the affine prediction candidate of the video signal; determining whether the at least one modified affine prediction candidate is currently not present in a candidate list; adding the at least one modified affine prediction candidate based on a determination that the at least one modified affine prediction candidate is not currently present in the candidate list, wherein if the candidate list is not complete, an AMVP candidate to be added is compared to affine candidates and added to the candidate list if not already present, and if the candidate list is not full, filling the unfilled portion of the candidate list with zero motion vector pairs; encoding a video signal using the candidate list by

2. A memory; 1. A processor, comprising: generating at least one modified affine prediction candidate by performing at least one of a plurality of rounding operations on the affine prediction candidate of the video signal; determining whether the at least one modified affine prediction candidate is currently not present in a candidate list; adding the at least one modified affine prediction candidate based on a determination that the at least one modified affine prediction candidate is not currently present in the candidate list, wherein if the candidate list is not complete, an AMVP candidate to be added is compared to affine candidates and added to the candidate list if not already present, and if the candidate list is not full, filling the unfilled portion of the candidate list with zero motion vector pairs; a processor configured to encode a video signal using the candidate list by An apparatus comprising:

3. performing at least one of a plurality of rounding operations on affine prediction candidates of a video signal to generate at least one modified affine prediction candidate; determining whether the at least one modified affine prediction candidate is currently not present in a candidate list; adding the at least one modified affine prediction candidate based on a determination that the at least one modified affine prediction candidate is not currently present in the candidate list, wherein if the candidate list is not complete, an AMVP candidate to be added is compared to affine candidates and added to the candidate list if not already present, and if the candidate list is not full, filling the unfilled portion of the candidate list with zero motion vector pairs; decoding a video signal using the candidate list,

4. A memory; 1. A processor, comprising: generating at least one modified affine prediction candidate by performing at least one of a plurality of rounding operations on the affine prediction candidate of the video signal; determining whether the at least one modified affine prediction candidate is currently not present in a candidate list; adding the at least one modified affine prediction candidate based on a determination that the at least one modified affine prediction candidate is not currently present in the candidate list, wherein if the candidate list is not complete, an AMVP candidate to be added is compared to affine candidates and added to the candidate list if not already present, and if the candidate list is not full, filling the unfilled portion of the candidate list with zero motion vector pairs; a processor configured to decode a video signal using the candidate list by An apparatus comprising:

5. The method described in claim 1, wherein the intra prediction candidates include at least one of a motion vector or a reference sample.

6. The method described in claim 1, wherein the at least one rounding operation includes rounding of motion vectors to 1 / 4 pixel resolution and integer motion vector rounding of both predictors in an Advanced Motion Vector Prediction process.

7. The method described in claim 6, wherein the second motion vector is obtained in an area not considered a candidate for the first motion vector.

8. The method described in claim 1, wherein an intra prediction candidate including a motion vector, a reference frame index, a reference frame list, and an IC flag is added to the candidate list if it was not previously present in the candidate list.

9. The method described in claim 1, wherein candidates in a frame rate up conversion process are found, including motion vectors, reference frame indices, and reference frame lists.

10. A non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform the method of claim 3 for playback.

11. A computer program product comprising instructions, which when executed by a computer, cause the computer to perform the method of claim 1.

12. The apparatus of claim 2, wherein the at least one rounding operation includes clipping, rounding, and selecting a different candidate.

13. The apparatus of claim 4, wherein the at least one rounding operation includes clipping, rounding, and selecting a different candidate.

14. The method of claim 1, wherein the first operation is rounding and the second operation is clipping.

15. The method of claim 3, wherein the first operation is rounding and the second operation is clipping.

16. The method described in claim 3, wherein a merge predictor including a motion vector, a reference frame index, a reference frame list, and an IC flag is the candidate added to the candidate list if it was not previously present in the candidate list.

17. The method described in claim 3, wherein candidates in a frame rate up conversion process are found, including motion vectors, reference frame indices, and reference frame lists.

18. The device of claim 4, wherein a merge predictor including a motion vector, a reference frame index, a reference frame list, and an IC flag is the candidate added to the candidate list if it was not previously present in the candidate list.

19. The device described in claim 4, wherein candidates in a frame rate up conversion process are found, including motion vectors, reference frame indices, and reference frame lists.

Images