Searching device and method for template matching prediction technology of atlas image

Abstract

An image encoding / decoding method, apparatus, and recording medium of the present disclosure may comprise the steps of: configuring a merge candidate list; reordering and merging the candidate list; generating a template matching candidate list based on the merge candidate list; and performing a template matching search based on the initial point of the template matching candidate list.

Description

TECHNICAL FIELD

[0001] The disclosure can be used in the field of a method, an apparatus, and a recording medium for encoding / decoding a graph image. BACKGROUND

[0002] MPEG Immersive Video (MIV) coding standard is a compression standard for encoding / decoding and rendering for reproducing images displayed when moving in 3DoF (Degree of Freedom), 3DoF+, and 6DoF.

[0003] 3DoF+ can support motion parallax for very limited movements such as slight head movements in a sitting environment by including a plurality of images of an omnidirectional scene and a plurality of virtual viewpoint images synthesized based on the plurality of images.

[0004] 6DoF can support freedom for movements in six directions (left-right movement, up-down movement, forward-backward movement, pitch, roll, and yaw).

[0005] The goal of MIV coding standard technology is to provide an interface for viewing immersive video and reproduce high-quality viewport images while having compression efficiency of conventional video coding. SUMMARY

[0006] TECHNICAL PROBLEM Since a template matching (TM) technology must perform a sub-pixel search process on all four motion vector prediction (MVP) candidates in a template matching candidate list (TMCL), the TM technology can require high complexity when performing encoding / decoding on a graph image.

[0007] The disclosure aims to reduce the number of applications by applying a search pattern according to a certain condition, thereby reducing the complexity of a video compression codec.

[0008] TECHNICAL SOLUTION An image encoding / decoding method, apparatus, and recording medium of the disclosure can include configuring a merge candidate list, reordering the merge candidate list, generating a template matching candidate list based on the merge candidate list, and performing a template matching search based on an initial point of the template matching candidate list.

[0009] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, configuring a merge candidate list can be performed by using at most 10 motion vector predictions (MVPs).

[0010] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, configuring a merge candidate list can be performed in the order of a spatial MVP, a temporal MVP, a non-adjacent spatial MVP, a history-based MVP, a paired MVP, and a zero MVP.

[0011] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, a template matching candidate list can be generated by using candidates having index values of 0th to 3rd of a merge candidate list.

[0012] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, the template matching candidate list can have at most 4 candidates.

[0013] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, the initial point can be a first candidate of the template matching candidate list.

[0014] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, the template matching search can be performed based on any one of a plurality of search patterns, and the plurality of search patterns can be determined by a search shape and a search range.

[0015] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, the search shape can include a cross search and a diamond search.

[0016] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, the search range can include 4 pixels, an integer pixel, a half pixel, a quarter pixel, and 1 / 8 pixel.

[0017] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, the search pattern can be determined based on a comparison of SAD values for the same search range according to the search shape.

[0018] In the image encoding / decoding method, apparatus, and recording medium of the disclosure, the search pattern can be determined based on a comparison of SAD values for two search ranges of the same search shape.

[0019] Technical effects The disclosure can reduce the number of applications by applying a search pattern according to a specific condition, thereby reducing the complexity of a video compression codec and improving the efficiency of the video compression codec. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The overall structure of a test model (TMIV) for MIV is shown.

[0021] Figure 2 The difference between a natural image and a map image generated from TMIV is shown.

[0022] Figure 3 An example is shown when using at most 10 candidates in a merge candidate list (MCL).

[0023] Figure 4An example of MCL changed after applying ARM C-TM is shown.

[0024] Figure 5 An example of TMCL using the generated MCL candidate configuration is shown.

[0025] Figure 6 An example of a template showing a reference block for a first MVP candidate in a TMCL candidate is shown.

[0026] Figure 7 An example of a search pattern is shown.

[0027] Figure 8 An example of a cross search and a diamond search in a search shape of a search pattern is shown.

[0028] Figure 9 An example of a search pattern performing only a diamond search shape is shown.

[0029] Figure 10 An example of a range for a sub-pixel in a search range is shown.

[0030] Figure 11 A flowchart of a method for performing the TM prediction technique of the present disclosure is shown.

[0031] Figure 12 An apparatus for performing the TM prediction technique of the present disclosure is shown. DETAILED DESCRIPTION

[0032] Since the present disclosure can take various changes and have several embodiments, specific embodiments will be shown in the drawings and described in detail. However, it is not intended to limit the present disclosure to the specific embodiments, and it should be understood that it includes all changes, equivalents or alternatives included in the scope of the idea and technology of the present disclosure. In describing each drawing, like reference numerals are used for like components.

[0033] Terms such as "first", "second", etc. can be used to describe various components, but the components should not be limited by the terms. The terms are used only to distinguish one component from other components. For example, without exceeding the scope of the present disclosure, a first component can be referred to as a second component, and similarly, a second component can also be referred to as a first component. The term "and / or" includes a combination of items of multiple related inputs or any item of multiple related inputs.

[0034] When a component is referred to as "linked" or "connected" to another component, it should be understood that it can be directly linked or connected to that other component, but another component may exist in between. On the other hand, when a component is referred to as "directly linked" or "directly connected" to another component, it should be understood that another component does not exist in between.

[0035] Since the terminology used in this application is only for describing specific embodiments, it is not intended to limit this disclosure. Singular expressions include plural expressions unless they clearly have different meanings in the context. It should be understood in this application that terms such as “comprising” or “having” are used to specify the presence of features, numbers, steps, actions, components, parts, or combinations thereof entered in the specification, but do not preclude the possibility of adding or having one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0036] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the hereinafter, the same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

[0037] Figure 1 The overall structure of the test model (TMIV) for MIV is shown.

[0038] Currently, TMIV can perform preprocessing on multi-view video to generate a small number of spectral images and perform compression using traditional video codecs such as High Efficiency Video Coding (HEVC), Universal Video Coding (VVC), etc.

[0039] Figure 2 The differences between the natural image and the atlas image generated from TMIV are shown.

[0040] In the standardization process of traditional video codecs, the test sequences used as Common Test Conditions (CTC) are mostly images obtained from nature, and there is a problem that the characteristics of the spectral images generated from TMIV are not taken into account.

[0041] Figure 3 An example is shown when using up to 10 candidates in the merged candidate list (MCL).

[0042] Template matching (TM) techniques derive motion vectors for inter-frame prediction by using the encoder / decoder templates surrounding the currently encoded block. In this case, the template can have an arbitrary size of N×M, and N and M can not only be defined as positive integers, but can also have the same value. For example, N×M can have a size of 3×3.

[0043] A motion vector with a small SAD cost between the coding / decoding template and the template of the reference block can be found and used as the motion vector of the current block.

[0044] Furthermore, the technique can be applied to the advanced motion vector prediction (AMVP) mode and the merge mode. However, to help describe the technique, the technique is described based on the TM technique for the merge mode.

[0045] Hereinafter, the process of the disclosure for performing the TM technique can be configured with several steps.

[0046] The first step can include configuring a merge candidate list (MCL).

[0047] For example, the configuration of the MCL can be performed by filling the MCL with up to 10 MVPs in the order of the spatial MVP, the temporal MVP, the non-adjacent spatial motion vector predictor (NA-SMVP), the history-based MVP, the paired MVP, and the zero MVP. Here, the maximum number of MVP candidates in the MCL can be signaled from a bitstream or can be implicitly determined based on the characteristics (size, shape, partitioning method, etc.) of the current block.

[0048] Furthermore, the method for filling the MCL by using each MVP can be the same as the method for filling a general MCL.

[0049] The second step can include performing template matching-based adaptive merge candidate reordering (ARMC-TM) on the generated MCL.

[0050] In this step, the candidates of the generated MCL can be reordered.

[0051] Figure 4 An example of the MCL changed after the application of the ARMC-TM is shown.

[0052] The ARMC-TM can be a technique of performing reordering based on template matching by utilizing the motion information of the generated MCL.

[0053] The third step can include generating a TM candidate list (TMCL).

[0054] Specifically, this step can be performed by generating the TMCL using the generated MCL candidates.

[0055] As an example, the TMCL can be generated by using the 0th index to the 3rd index among the generated MCL candidates.

[0056] As an example, the TMCL can be generated by using the MCL candidate indexes signaled from a bitstream.

[0057] The generated TMCL can use up to 4 MVP candidates. Here, the maximum number of MVP candidates in the TMCL can be signaled from the bitstream or can be implicitly determined based on the characteristics (size, shape, partitioning method, maximum number of candidates of MCL, etc.) of the current block.

[0058] Figure 5 An example of TMCL configured by using the generated MCL candidates is shown.

[0059] The fourth step can include performing a TM search process based on the initial point.

[0060] Specifically, this step can perform a TM search process using the MVP corresponding to the index of the TMCL as a starting point.

[0061] Figure 6 An example showing a template of a reference block for the first MVP candidate among the TMCL candidates is shown. Specifically, Figure 6 An example of performing a TM search process using the first MVP candidate as an initial point is shown.

[0062] Figure 7 An example of a search pattern is shown.

[0063] The search pattern can be determined by a search shape (cross search, diamond search, etc.) and a search range (4 pixels, integer pixel, half pixel, quarter pixel, 1 / 8 pixel, etc.).

[0064] Referring to Figure 7 which shows that the search shape includes only cross search and diamond search, and each search range includes 4 pixels, integer pixel, half pixel, quarter pixel, and 1 / 8 pixel.

[0065] As an example, the search range for performing template matching based on the initial point can be any one of 4 pixels, integer pixel, half pixel, quarter pixel, and 1 / 8 pixel, as shown in Figure 7 .

[0066] Figure 8 An example of cross search and diamond search in the search shape of the search pattern is shown.

[0067] The cross search can have a shape that searches in the up, down, left, and right directions based on the initial point. The diamond search can have a shape that searches in a diamond shape based on the initial point.

[0068] In this step, the final prediction factor can be generated after performing processing for all combinations of the four MVP candidates with respect to the search pattern.

[0069] The TM technique can be a method of estimating a motion vector by comparing SAD costs between reference templates based on a coding / decoding template around a current coding block in order to implicitly derive a motion vector for the current coding block in an encoder / decoder.

[0070] In this case, a search pattern for calculating a SAD cost can be determined by a shape of a search pattern and a search range. The shape of the search pattern can include a cross search, a diamond search, a TZ search, a full search, etc. The shape of the search pattern can be predefined in the apparatus, or can be implicitly determined based on characteristics of the current block, or can be determined by search shape information included in search pattern information signaled from a bitstream.

[0071] The search range can include 4-pixel, 1-pixel, 1 / 2-pixel, 1 / 4-pixel, 1 / 8-pixel, and 1 / 16-pixel. The search range can be predefined in the apparatus, or can be implicitly determined based on characteristics of the current block, or can be determined by search range information included in search pattern information signaled from a bitstream.

[0072] As an embodiment, when calculating a SAD cost for a current coding block, a cross search and a diamond search can be used as a search shape, 1-pixel, 1 / 2-pixel, 1 / 4-pixel, and 1 / 8-pixel can be used as a search range, and a search (or SAD cost calculation) can be performed for all possible combinations of the search shape and the search range.

[0073] On the contrary, when calculating a SAD cost, a search (or SAD cost calculation) can be performed for only some search patterns according to a comparison of SAD values for the same search range for each search shape.

[0074] As an example, when a SAD value calculated by a diamond search pattern of a search range of whole pixels is smaller than a SAD value of a cross search pattern of a search range of whole pixels, a search (or SAD cost calculation) for sub-pixels (1 / 2-pixel, 1 / 4-pixel, 1 / 8-pixel, etc.) can be performed for only a search pattern of a diamond search shape.

[0075] Further, a search (or SAD cost calculation) can be performed for only some search patterns according to a comparison of SAD values for different search ranges in the same search shape.

[0076] As an example, for a certain search shape, if a SAD value for a search range of whole pixels is smaller when compared to a SAD value for a search range of quarter pixels, a search (or calculation) for half pixels can be performed in a search pattern of the corresponding search shape only.

[0077] As an example, for a particular search shape, if the integer-pel SAD value is larger when compared to the SAD value of the search range quarter-pel when the search range is quarter-pel, then only the 1 / 8-pel search (or computation) can be performed in the search mode of the corresponding search shape.

[0078] As an example, for a particular search shape, if the integer-pel SAD value is smaller by at least N times when compared to the SAD value of the search range half-pel, then only the four-pel search (or computation) can be performed in the search mode of the corresponding search shape. Here, N can have a value greater than or equal to 0. The value of N can be predefined in the device, or can be determined implicitly based on the characteristics of the current block, or can be determined through the search range information included in the search mode information signaled from the bitstream.

[0079] Figure 9 An example of a search mode that performs only the diamond search shape is shown.

[0080] Figure 10 An example of a range of sub-pixels in the search range is shown.

[0081] Figure 11 A flowchart of a method for performing the TM prediction technique of the present disclosure is shown.

[0082] The TM prediction technique of the present disclosure can include configuring a merge candidate list (S1101), reordering the merge candidate list (S1102), generating a template matching candidate list (TMCL) based on the merge candidate list (S1103), and performing a template matching search based on an initial point of the template matching candidate list (S1104). Since the specific details of each step are described above, they are omitted.

[0083] Figure 12 An apparatus for performing the TM prediction technique of the present disclosure is shown.

[0084] The apparatus can include a merge candidate list configuration unit (1201) that performs the step of configuring a merge candidate list, a merge candidate list reordering unit (1202) that performs the step of reordering the merge candidate list, a template matching candidate list generation unit (1203) that performs the step of generating a template matching candidate list (TMCL) based on the merge candidate list, and a template matching search unit (1204) that performs the step of performing a template matching search based on an initial point of the template matching candidate list.

[0085] For clarity of description, the exemplary methods of the disclosure are represented as a series of operations, but are not intended to limit the order of execution of the steps, and each step can be executed simultaneously or in a different order if necessary. To implement the methods according to the disclosure, other steps can be additionally included in the exemplary steps, or some steps can be excluded and the remaining steps can be included, or some steps can be excluded and other steps can be additionally included.

[0086] Various embodiments of the disclosure do not enumerate all possible combinations, but are intended to describe representative aspects of the disclosure, and matters described in various embodiments can be independently applied or applied in combination of two or more.

[0087] In addition, various embodiments of the disclosure can be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, implementation can be performed by one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0088] The scope of the disclosure includes software or machine-readable instructions (e.g., operating systems, applications, firmware, programs, etc.) that perform operations of the methods according to various embodiments on a device or computer, and a non-transitory computer-readable medium that stores such software or instructions, etc., and is executable on a device or computer.

[0089] Industrial applicability The image encoding / decoding method, apparatus, and recording medium of the disclosure can be industrially applied to the field of encoding / decoding atlas images.

Claims

1. An image encoding method, the method comprising: Configure the candidate list for merging; The candidate list to be merged is reordered; Generate a template matching candidate list based on the merged candidate list; as well as Perform a template matching search based on the initial point of the template matching candidate list.

2. The method of claim 1, wherein: The configuration of the merged candidate list is performed by predicting the MVP using up to 10 motion vectors.

3. The method of claim 1, wherein: The merge candidate list is configured to be executed in the following order: spatial MVP, temporal MVP, non-adjacent spatial MVP, history-based MVP, paired MVP, and zero MVP.

4. The method of claim 1, wherein: The template matching candidate list is generated by using candidates with index values ​​0 to 3 of the merged candidate list.

5. The method of claim 1, wherein: The template matching candidate list has a maximum of 4 candidates.

6. The method of claim 1, wherein: The initial point is the first candidate in the template matching candidate list.

7. The method of claim 1, wherein: The template matching search is performed based on any one of multiple search patterns. The multiple search patterns are determined by the search shape and the search range.

8. The method of claim 7, wherein: The search shapes include cross-shaped search and diamond-shaped search.

9. The method of claim 8, wherein: The search range includes 4 pixels, whole pixels, half pixels, quarter pixels, and 1 / 8 pixels.

10. The method of claim 9, wherein: The search pattern is determined based on a comparison of SAD values ​​for the same search range according to the search shape.

11. The method of claim 9, wherein: The search pattern is determined by comparing the SAD values ​​of two search ranges with the same search shape.

12. An image decoding method, the method comprising: Configure the candidate list for merging; The candidate list to be merged is reordered; Generate a template matching candidate list based on the merged candidate list; as well as Perform a template matching search based on the initial point of the template matching candidate list.

13. The method of claim 12, wherein: The template matching search is performed based on any one of multiple search patterns. The multiple search patterns are determined by the search shape and the search range.

14. The method of claim 13, wherein: The search pattern is determined based on a comparison of SAD values ​​for the same search range according to the search shape.

15. A computer-readable recording medium for storing a bitstream generated by an image encoding method, wherein, The image encoding method includes: Configure the candidate list for merging; The candidate list to be merged is reordered; Generate a template matching candidate list based on the merged candidate list; and Perform a template matching search based on the initial point of the template matching candidate list.

Images