Coding and decoding of patch data units for point cloud coding.

Abstract

To provide a method and apparatus of point-cloud coding executed by a decoder, and a decoding device and a coding device.SOLUTION: The method includes the steps of: receiving coded patch information data, by a receiver of the decoder; obtaining, by the processor of the decoder, a patch corresponding to the coded patch information data, the patch having a patch type (patch_mode); determining, by the processor of the decoder, whether the patch type for the patch is the last patch type; and finishing, by the processor of the decoder, a restoration process corresponding to the coded patch information data when the patch type is the last patch type.SELECTED DRAWING: Figure 18

Description

Technical Field

[0001] Cross - Reference to Related Applications This patent application claims the benefit of U.S. Provisional Patent Application No. 62 / 817,391, filed on March 12, 2019, by Vladyslav Zakharchenko et al., entitled "Patch Data Unit Coding and Decoding for Point - Cloud Coding", which is incorporated herein by reference.

[0002] This disclosure generally relates to point - cloud coding and, more specifically, to high - level syntax for point - cloud coding.

Background Art

[0003] Point clouds are employed in a variety of applications, including the entertainment industry, intelligent vehicle navigation, geospatial surveys, three - dimensional (3D) modeling and visualization of real - world objects, etc. Considering the non - uniform sampling geometry of point clouds, a compact representation for the storage and transmission of such data is useful. Compared to other 3D representations, irregular point clouds are more general and applicable to a wider range of sensors and data acquisition strategies. For example, when performing 3D rendering in a virtual reality world or remote rendering in a telepresence environment, the rendering of virtual graphics and real - time commands is processed as a high - density point - cloud dataset.

Summary of the Invention

Means for Solving the Problems

[0004] The first aspect relates to a method of point cloud coding (PCC) performed by a decoder. The method includes the steps of: receiving encoded patch information data by a receiver of the decoder; obtaining a patch corresponding to the encoded patch information data by a processor of the decoder, wherein the patch has a patch type (patch_mode); determining by the processor whether the patch type for the patch is the last patch type; and terminating a recovery process corresponding to the encoded patch information data by the processor if the patch type is the last patch type.

[0005] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0006] A second aspect relates to a method of point cloud coding (PCC) performed by a decoder. The method includes the steps of: receiving encoded patch information data by the decoder's receiver; obtaining a patch corresponding to the encoded patch information data by the decoder's processor, wherein the patch has a patch type (patch_mode); determining by the processor whether the patch type for the patch is a skip patch type; decoding a reference patch index (spdu_patch_index) corresponding to the patch if the patch type is a skip patch type; determining a reference index (refIdx) for the patch based on a reference frame index ([refFrmIdx]) corresponding to the patch and the reference patch index decoded when the patch type is a skip patch type; and restoring a three-dimensional representation of the patch based on the reference index determined when the patch type is a skip patch type.

[0007] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0008] In the first implementation of the method according to the second embodiment itself, the method further comprises the steps of: determining that the patch type for a patch is an intra-patch type; determining a reference index (refIdx) for a patch based on the frame index ([frmIdx]) corresponding to the patch and a decremented patch index; decoding the patch using the two-dimensional (2D) component and the three-dimensional (3D) component corresponding to the patch; and restoring a three-dimensional representation based on the decoded patch.

[0009] In the second embodiment of the method, or in any of the aforementioned implementations of the second embodiment, the method further comprises the steps of: determining that the patch type for a patch is an interpatch type; decoding a second reference patch index (dpdu_patch_index) corresponding to the patch; determining a reference index (refIdx) for the patch based on a reference frame index ([refFrmIdx]) corresponding to the patch and the decoded second reference patch index; decoding the patch using a two-dimensional (2D) component and a three-dimensional (3D) component corresponding to the patch; and restoring a three-dimensional representation based on the decoded patch.

[0010] In a third implementation of the method according to the second embodiment itself or any of the aforementioned implementations of the second embodiment, the method further comprises the steps of: determining that the patch type for a patch is a pulse code modulation (PCM) patch type; determining a reference index (refIdx) for a patch based on a frame index ([frmIdx]) corresponding to the patch and a decremented patch index; decoding the patch using a two-dimensional (2D) component for independent points corresponding to the patch and a three-dimensional (3D) component for independent points corresponding to the patch; and restoring a stereoscopic representation based on the decoded patch.

[0011] In a fourth implementation of the method according to the second embodiment itself or any of the aforementioned implementations of the second embodiment, the method further comprises the step of receiving an input corresponding to encoded patch information data, wherein the input comprises one or more of patch_mode, patch index, reference index, frame index, and reference frame index.

[0012] In a fourth implementation of the method according to the second embodiment itself or any of the aforementioned implementations of the second embodiment, the method further comprises the step of displaying an image generated based on a restored stereoscopic representation on a display of an electronic device.

[0013] A third aspect relates to a method of point cloud coding (PCC) performed by a decoder. The method includes the steps of: receiving encoded patch information data with the decoder's receiver; obtaining a patch corresponding to the encoded patch information data with the decoder's processor, wherein the patch has a patch type (patch_mode); determining with the processor whether the patch type for the patch is a skip patch type; decoding a reference patch index (spdu_patch_index) corresponding to the patch if the patch type is a skip patch type; and determining with the processor whether the patch type is a skip patch type. The process includes the steps of: determining a reference index (refIdx) for a patch based on the reference patch index decoded when the patch type is a skip patch type; the processor restoring a three-dimensional representation of the patch based on the reference index determined when the patch type is a skip patch type; the processor determining whether more patch availability flags have a first value or a second value; storing the restored three-dimensional representation in the decoder's memory when more patch availability flags have a first value; and terminating the process of restoring the encoded patch information when more patch availability flags have a second value.

[0014] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0015] In the first implementation of the method according to the third aspect itself, the method further comprises the steps of: determining that the patch type for a patch is an intra-patch type; determining a reference index (refIdx) for a patch based on the frame index ([frmIdx]) corresponding to the patch and a decremented patch index; decoding the patch using the two-dimensional (2D) component and the three-dimensional (3D) component corresponding to the patch; and restoring a three-dimensional representation based on the decoded patch.

[0016] In a second implementation of the method according to the third embodiment itself, the method further comprises the steps of: determining that the patch type for a patch is an interpatch type; decoding a second reference patch index (dpdu_patch_index) corresponding to the patch; determining a reference index (refIdx) for the patch based on a reference frame index ([refFrmIdx]) corresponding to the patch and the decoded second reference patch index; decoding the patch using the two-dimensional (2D) component and the three-dimensional (3D) component corresponding to the patch; and restoring a three-dimensional representation based on the decoded patch.

[0017] In a third implementation of the method according to the third embodiment itself, the method further comprises the steps of: determining that the patch type for a patch is a pulse code modulation (PCM) patch type; determining a reference index (refIdx) for a patch based on the frame index ([frmIdx]) corresponding to the patch and a decremented patch index; decoding the patch using a two-dimensional (2D) component for an independent point corresponding to the patch and a three-dimensional (3D) component for an independent point corresponding to the patch; and restoring a three-dimensional representation based on the decoded patch.

[0018] In a fourth implementation of the method according to the third embodiment itself, the method further comprises the step of receiving an input corresponding to encoded patch information data, wherein the input comprises one or more of patch_mode, patch index, reference index, frame index, and reference frame index.

[0019] In a fifth implementation of the method according to the third embodiment itself, the method further comprises the step of displaying an image generated based on the restored stereoscopic representation on the display of an electronic device.

[0020] A fourth aspect relates to a method of point cloud coding (PCC) performed by an encoder. The method includes the steps of: obtaining a patch frame data unit (pfdu) that identifies a patch type (pdfu_patch_mode) for each of a plurality of patches using the encoder's receiver; determining, using the encoder's processor, whether the patch type for a patch from the plurality of patches is the last patch type; encoding patch information data for the patch if the patch type is not the last patch type, using the encoder's processor, wherein the patch information data includes the patch type for the patch; and encoding patch information data for the patch if the patch type is set to the last patch type, using the encoder's processor, wherein the patch information data includes the last patch type for the patch.

[0021] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0022] In a first implementation form of the method according to the fourth aspect itself, the patch type is one of a skip patch type, an inter-patch type, an intra-patch type, and a pulse code modulation (PCM) patch type.

[0023] In a second implementation form of the method according to the fourth aspect itself, the patch frame data unit includes a frame index (frmIdx), a two-dimensional (2D) component corresponding to the patch, and a three-dimensional (3D) component corresponding to the patch.

[0024] In a third implementation form of the method according to the fourth aspect itself, the patch information data includes a frame index, a two-dimensional (2D) component corresponding to the patch, and a three-dimensional (3D) component corresponding to the patch.

[0025] In a fourth implementation form of the method according to the fourth aspect itself, the method further includes a step of determining whether the patch type is the last patch type, and a step of repeating a step of encoding patch information data for a subsequent patch from a plurality of patches until one of the patches from the plurality of patches is determined to have the last patch type.

[0026] In a fifth implementation form of the method according to the fourth aspect itself, the method further includes a step of performing byte alignment after all of the patch information data has been encoded.

[0027] In a sixth implementation form of the method according to the fourth aspect itself, the method further includes a step of generating a compressed patch frame data unit after all of the patch information data has been encoded.

[0028] In a seventh implementation form of the method according to the fourth aspect itself, the method further includes a step of storing the compressed patch frame data unit in the memory of the encoder for transmission to a decoder.

[0029] The fifth aspect relates to a method of point cloud coding (PCC) implemented by an encoder. The method includes steps of: obtaining, by a receiver of the encoder, a patch frame data unit (pfdu) for each of a plurality of patches; adding, by a processor of the encoder, a last patch flag to each of the plurality of patches; determining, by the processor of the encoder, whether a patch type for the patch from the plurality of patches is a last patch type based on a value of the last patch flag; and encoding, by the processor of the encoder, patch information data for the patch when the patch type is not the last patch type, wherein the patch information data includes the patch type for the patch and the last patch flag.

[0030] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0031] In the first implementation of the method according to the fifth aspect itself, the patch type is one of the following: skip patch type, interpatch type, intrapatch type, and pulse code modulation (PCM) patch type.

[0032] In the second implementation of the method according to the fifth aspect itself, the patch frame data unit includes a frame index (frmIdx), a two-dimensional (2D) component corresponding to the patch, and a three-dimensional (3D) component corresponding to the patch.

[0033] In the third implementation of the method according to the fifth aspect itself, the patch information data includes a frame index, a two-dimensional (2D) component corresponding to the patch, and a three-dimensional (3D) component corresponding to the patch.

[0034] In a fourth implementation of the method according to the fifth aspect itself, the method further comprises the steps of: determining whether a patch type is the last patch type based on the value of the last patch flag; and repeatedly encoding patch information data for subsequent patches from a plurality of patches until it is determined that one of the patches from the plurality of patches has the last patch type based on the value of the last patch flag.

[0035] In a fifth implementation of the method according to the fifth aspect itself, the method further comprises the step of performing byte alignment after it has been determined, based on the value of the last patch flag, that one of the patches from a plurality of patches has the last patch type.

[0036] In a sixth implementation of the method according to the fifth aspect itself, the method further comprises the step of generating a compressed patch frame data unit after all of the patch information data has been encoded.

[0037] In a seventh implementation of the method according to the fifth aspect itself, the method further comprises the step of storing a compressed patch frame data unit in the encoder's memory for transmission to the decoder.

[0038] A sixth aspect relates to a decoding device, which includes a receiver configured to receive encoded patch information, a memory coupled to the receiver, the memory storing instructions, and a processor coupled to the memory, the processor configured to execute an instruction causing the decoding device to retrieve a patch corresponding to encoded patch information data, determine whether the patch has a patch type (patch_mode) and whether the patch type for the patch is the last patch type, and if the patch type is the last patch type, to terminate the decryption process corresponding to the encoded patch information data.

[0039] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0040] A seventh aspect relates to a decoding device, which includes a receiver configured to receive encoded patch information, a memory coupled to the receiver, the memory storing instructions, and a processor coupled to the memory, the processor configured to execute instructions causing the decoding device to acquire a patch corresponding to encoded patch information data, determine whether the patch has a patch type (patch_mode) and whether the patch type for the patch is a skip patch type, decode a reference patch index (spdu_patch_index) corresponding to the patch if the patch type is a skip patch type, determine a reference index (refIdx) for the patch based on a reference frame index ([refFrmIdx]) corresponding to the patch and the reference patch index decoded when the patch type is a skip patch type, and restore a three-dimensional representation of the patch based on the reference index determined when the patch type is a skip patch type.

[0041] The use of this decoding device allows for flexible ordering of patches within the patch buffer. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch buffer in any order. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the reference patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the reference patch data unit are identical or exactly the same. The coding technique also allows for the removal of syntactic elements indicating a list of matched patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches in the patch buffer. In addition, the coding technique signals the total number of patches in the patch buffer (e.g., the size of the patch buffer), eliminating the need to replace that process with a special termination patch type. Furthermore, the coding technique allows for the replacement of pulse-coded modulation (PCM) patches, which are typically located at the end of the patch buffer, with a PCM patch type indication. Therefore, the coder / decoder (also known as "codec") in video coding will be improved over current codecs (for example, the process of encoding and decoding patches in V-PCC). In practical terms, an improved video coding process can increase coding efficiency, thereby providing users with a better user experience when video is transmitted, received, and / or viewed.

[0042] The eighth aspect is a receiver configured to receive encoded patch information, a memory coupled to the receiver, the memory storing instructions, and a processor coupled to the memory, the processor acquiring a patch corresponding to encoded patch information data, determining whether the patch has a patch type (patch_mode) and whether the patch type for the patch is a skip patch type, and if the patch type is a skip patch type, decoding the reference patch index (spdu_patch_index) corresponding to the patch, the reference frame index ([refFrmIdx]) corresponding to the patch, and the patch type is a skip patch type The present invention relates to a decoding device including a processor configured to determine a reference index (refIdx) for a patch based on a reference patch index decoded when the patch type is a skip patch type, to restore a three-dimensional representation of the patch based on the reference index determined when the patch type is a skip patch type, to determine whether more patch availability flags have a first value or a second value, to store the restored three-dimensional representation in memory when more patch availability flags have a first value, and to execute an instruction to cause the decoding device to terminate the process of restoring the encoded patch information when more patch availability flags have a second value.

[0043] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0044] In the first implementations of the decoding device according to the sixth, seventh, and eighth aspects themselves, the decoding device further comprises a display configured to display an image generated based on the reconstructed stereoscopic representation.

[0045] The ninth aspect relates to an encoding device comprising: a receiver configured to receive three-dimensional (3D) images; a memory coupled to the receiver, the memory storing instructions; and a processor coupled to the memory, the processor configured to execute instructions causing the encoding device to acquire a patch frame data unit (pfdu) that identifies a patch type (pdfu_patch_mode) for each of a plurality of patches, determine whether the patch type for the patch from the plurality of patches is the last patch type, encode patch information data for the patch if the patch type is not the last patch type, encode patch information data for the patch if the patch information data includes the patch type for the patch, and encode patch information data for the patch if the patch type is set to the last patch type, and the patch information data includes the last patch type for the patch.

[0046] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0047] A tenth aspect relates to an encoding device comprising: a receiver configured to receive three-dimensional (3D) images; a memory coupled to the receiver, the memory storing instructions; and a processor coupled to the memory, the processor configured to execute an instruction causing an encoding device to acquire a patch frame data unit (pfdu) for each of a plurality of patches, add a last patch flag to each of the plurality of patches, determine based on the value of the last patch flag whether the patch type for the patch from the plurality of patches is the last patch type, and if the patch type is not the last patch type, encode patch information data for the patch, so that the patch information data includes the patch type for the patch and the last patch flag.

[0048] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0049] In the first implementations of the encoding device according to the ninth and tenth embodiments themselves, the encoding device further comprises a transmitter coupled to a processor, the transmitter being configured to transmit a bitstream having encoded patch information data to a decoder.

[0050] An eleventh aspect relates to a coding apparatus comprising: a receiver configured to receive a stereoscopic picture for encoding or a bitstream for decoding; a transmitter coupled to the receiver, the transmitter configured to transmit a bitstream to a decoder or a decoded stereoscopic image to a restoration device configured to restore a decoded stereoscopic picture; a memory coupled to at least one of the receiver or transmitter, the memory configured to store instructions; and a processor coupled to the memory, the processor configured to execute instructions stored in the memory and perform any of the methods disclosed herein.

[0051] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0052] In the first implementation of the coding device according to the fifth aspect itself, the coding device further comprises a display configured to display an image based on the restored patch.

[0053] A twelfth aspect relates to a system comprising an encoder and a decoder that communicates with the encoder, wherein the encoder or decoder includes an encoding device, decoding device, or coding apparatus as described herein.

[0054] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0055] A thirteenth aspect relates to a coding means comprising: a receiving means configured to receive a stereoscopic picture for encoding or a bitstream for decoding, restoring, and projecting; a transmitting means coupled to the receiving means, configured to transmit a bitstream to a decoder or a decoded image to a display means; a storage means coupled to at least one of the receiving means or the transmitting means, configured to store instructions; and a processing means coupled to the storage means, configured to execute instructions stored in the storage means to perform any of the methods disclosed herein.

[0056] This coding method allows for flexible ordering of patches within a patch frame data unit. That is, patches with different patch modes (e.g., inter, intra, PCM, etc.) may be included in the patch frame data unit in any order. In addition, further patches may be added to the patch frame data unit. These added patches may be placed at the end of the patch frame data unit or at any random patch index position. The coding technique also allows for the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the base patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the base patch data unit are identical or within an acceptable tolerance. The coding technique also allows for the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. Furthermore, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indicator. Multiple PCM patches are also permitted. Thus, the coder / decoder (aka "codec") in point cloud coding is improved over current codecs (e.g., the process of encoding and decoding patches in V-PCC). In practical terms, the improved point cloud coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0057] For clarity, any one of the embodiments described above may be combined with any one or more of the other embodiments described above to create a novel embodiment within the scope of this disclosure.

[0058] The other features described above will be more clearly understood from the following embodiments for carrying out the invention, in conjunction with the attached drawings and claims.

[0059] For a more complete understanding of this disclosure, the following brief description is made in conjunction with the accompanying drawings and modes for carrying out the invention, and similar reference numbers represent similar parts. [Brief explanation of the drawing]

[0060] [Figure 1] This block diagram shows an exemplary coding system that can utilize context modeling techniques. [Figure 2] A block diagram illustrating an exemplary encoder that can implement context modeling techniques. [Figure 3] An exemplary decoder block that can implement context modeling techniques. [Figure 4] This diagram shows a sequence of point cloud frames, each containing a three-dimensional (3D) point cloud. [Figure 5] This is one of the 3D point clouds in Figure 5 that has been projected onto a bounding box to generate a two-dimensional (2D) projection. [Figure 6A] Figure 5 shows the occupancy map corresponding to the 2D projection from the bounding box. [Figure 6B] This is a diagram of the geometric map corresponding to the 2D projection from the bounding box in Figure 5. [Figure 6C] Figure 5 shows the attribute map corresponding to the 2D projection from the bounding box. [Figure 7] This is a diagram of a video-based point cloud coding (V-PCC) bitstream. [Figure 8] This is a diagram illustrating the coded decomposition of a point cloud sequence for multiple frames. [Figure 9] This is a diagram of a single patch data unit extracted from one of the point cloud frames in the point cloud sequence shown in Figure 8. [Figure 10A] Figure 9 shows the occupancy map corresponding to a single patch data unit. [Figure 10B] Figure 9 shows the geometric map corresponding to a single patch data unit. [Figure 10C] Figure 9 shows the attribute map corresponding to a single patch data unit. [Figure 11] This is a diagram of the 2D components for a patch data unit within a 2D bounding box. [Figure 12A] Figure 11 is a schematic diagram of the 3D components for the patch data unit. [Figure 12B] Figure 11 is a dataset diagram of the 3D components for the patch data unit. [Figure 13] This is a diagram illustrating one embodiment of the patch frame data unit coding process. [Figure 14] This is a diagram illustrating one embodiment of a patch frame data unit coding process that uses the last patch flag. [Figure 15] This is a diagram illustrating one embodiment of a patch information data frame with a description of the patch unit type. [Figure 16] This is a diagram illustrating one embodiment of the patch frame data unit decoding process. [Figure 17] This is a diagram illustrating one embodiment of the patch frame data unit decoding process using the last patch flag. [Figure 18] This is a diagram illustrating one embodiment of a point cloud coding (PCC) method performed by a decoder. [Figure 19] This is a diagram illustrating one embodiment of the PCC method implemented by a decoder. [Figure 20] This is a diagram illustrating one embodiment of the PCC method implemented by a decoder. [Figure 21] This is a diagram illustrating one embodiment of a PCC method performed by an encoder. [Figure 22] This is a diagram illustrating one embodiment of a PCC method performed by an encoder. [Figure 23] This is a schematic diagram of a coding device. [Figure 24] This is a schematic diagram of one embodiment of a means for coding. [Modes for carrying out the invention]

[0061] While exemplary implementations of one or more embodiments are provided below, it should be understood from the outset that the systems and / or methods of the Disclosure may be carried out using any number of techniques, whether currently known or existing. The Disclosure shall not be limited in any respect to the exemplary implementations, drawings, and techniques exemplified below, including the exemplary designs and implementations illustrated and described herein, and may be modified in the entirety of the appended claims and their equivalents.

[0062] Throughout the specification and claims, the following terms have the meanings expressly relating to this specification unless the context clearly indicates otherwise: The reference patch index (e.g., spdu_patch_index) indicates the difference between the index p of the current patch data unit [in the current patch frame data unit] and the index of the reference patch data unit [in the reference patch frame data unit]. The reference index (e.g., refIdx) indicates the actual index of the reference patch data unit in the previously decoded patch frame data unit [in the decoded patch frame data unit buffer]. The reference frame index (e.g., refFrmIdx) indicates the index of the previously decoded patch frame data unit [in the decoded patch frame data unit buffer].

[0063] Video coding standards include Advanced Video Coding (AVC), also known as ITU-T H.261, ITU-T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, ITU-T H.264 or ISO / IEC MPEG-4 Part 10, and High Efficiency Video Coding (HEVC), also known as ITU-T H.265 or MPEG-H Part 2. AVC includes extensions such as Scalable Video Coding (SVC), Multiview Video Coding (MVC), Multiview Video Coding Plus Depth (MVC+D), and 3D AVC (3D-AVC). HEVC includes extensions such as Scalable HEVC (SHVC), Multiview HEVC (MV-HEVC), and 3D HEVC (3D-HEVC).

[0064] A point cloud is a collection of data points in 3D space. Each data point contains parameters that determine its location (e.g., X, Y, Z), color (e.g., R, G, B or Y, U, V), and possibly other properties such as transparency, reflectivity, and acquisition time. Typically, each point in the cloud has the same number of attributes associated with it. Point clouds may be used in a variety of applications, including real-time 3D immersive telepresence, content virtual reality (VR) viewing with bidirectional parallax, 3D free-viewpoint sports playback broadcasting, geographic information systems, cultural heritage, autonomous navigation based on large-scale 3D dynamic maps, and automotive applications.

[0065] The ISO / IEC Video Expert Group (MPEG) began developing a new codec standard for point cloud coding for lossless and lossy compressed point cloud data with substantial coding efficiency and robustness to network environments in 2016. This codec standard will enable point clouds to be manipulated as computer data, stored on various storage media, transmitted and received over existing and future networks, and distributed on existing and future broadcast channels.

[0066] Recently, point cloud coding (PCC) work has been categorized into three categories: PCC Category 1, PCC Category 2, and PCC Category 3. Two separate working drafts have been developed, one for PCC Category 2 (PCC Cat2) and the other for PCC Categories 1 and 3 (PCC Cat13). MPEG output document N17534 contains the latest working draft (WD) for PCC Cat2, and MPEG output document N17533 contains the latest WD for PCC Cat13.

[0067] The main idea behind the design of the PCC Cat2 codec in PCC Cat2 WD is to leverage existing video codecs to compress the geometric and texture information of dynamic point clouds by compressing point cloud data as a set of different video sequences. Specifically, two video sequences are generated and compressed using the video codec, one representing the geometric information of the point cloud data and the other representing the texture information. Additional metadata for interpreting the two video sequences, namely occupancy maps and auxiliary patch information, are also generated and compressed separately.

[0068] Unfortunately, the existing PCC design has flaws. For example, the data units related to a single time instance, i.e., a single access unit (AU), are not decoded in a consecutive order. In PCC Cat2 WD, the data units for textures, geometry, auxiliary information, and occupancy maps per AU are interleaved at the frame group level; that is, the geometry data for all frames within a group is together. The same is often true for texture data, etc. In PCC Cat13 WD, the geometry data units and general attributes per AU are interleaved at the level of the entire PCC bitstream (for example, the same as PCC Cat2 WD when there is only one group of frames that has the same length as the entire PCC bitstream). Interleaving data units belonging to a single AU inherently causes a huge end-to-end delay that is at least equal to the length of the group of frames in the presentation duration within the application system.

[0069] Another drawback concerns the bitstream format. The bitstream format allows for the emulation of start code patterns such as 0x0003, and therefore does not work for transmission over MPEG-2 transport streams (TS) where start code emulation prevention is required. For PCC Cat2, when either HEVC or AVC is used for coding the geometric and texture components, currently only group_of_frames_geometry_video_payload() and group_of_frames_texture_video_payload() have start code emulation prevention in place. For PCC Cat13, start code emulation prevention is not in place anywhere in the bitstream.

[0070] In PCC Cat2 WD, some of the codec information for the geometric bitstream and texture bitstream (e.g., which codec, profile, level, etc.) is deeply embedded in multiple instances of the structures group_of_frames_geometry_video_payload() and group_of_frames_texture_video_payload(). Furthermore, some information such as profiles and levels indicating the ability to decode auxiliary information and occupied map components, as well as point cloud reconstruction, is missing.

[0071] A high-level syntactic design is provided that solves one or more of the aforementioned problems associated with point cloud coding. As described in more detail below, this disclosure uses a type indicator in the data unit header (also known as the PCC Network Access Layer (NAL) header) to specify the type of content in the payload of a PCC NAL unit. In addition, this disclosure uses a group of frame header NAL units to carry a group of frame header parameters. A group of frame header NAL units may also be used to signal the profiling and level of each geometric bitstream or texture bitstream.

[0072] Figure 1 is a block diagram illustrating an exemplary coding system 10 that can utilize the PCC video coding technique. As shown in Figure 1, the coding system 10 includes a source device 12 that provides coded video data to be later decoded by a destination device 14. More specifically, the source device 12 can provide video data to the destination device 14 via a computer-readable medium 16. The source device 12 and destination device 14 may comprise any of a wide range of devices, including desktop computers, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, so-called "smart" pads, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, and the like. In some cases, the source device 12 and destination device 14 may be wireless communication compatible.

[0073] The destination device 14 can receive the encoded video data to be decoded via the computer-readable medium 16. The computer-readable medium 16 may comprise any type of medium or device capable of transferring the encoded video data from the source device 12 to the destination device 14. In one example, the computer-readable medium 16 may comprise a communication medium that enables the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from the source device 12 to the destination device 14.

[0074] In some examples, encoded data may be output to a storage device via the output interface 24. Similarly, encoded data may be accessed from the storage device via the input interface. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray® disc, digital video disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further examples, the storage device may correspond to a file server or another intermediate storage device capable of storing encoded video generated by the source device 12. The destination device 14 can access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing encoded video data and sending that encoded video data to the destination device 14. Exemplary file servers include a web server (for example, a website), a File Transfer Protocol (FTP) server, a network-attached storage (NAS) device, or a local disk drive. The destination device 14 can access the encoded video data via any standard data connection, including an internet connection. This may include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., digital subscriber lines (DSL), cable modems, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. Transmission of encoded video data from the storage device may be streaming transmission, download transmission, or a combination thereof.

[0075] The techniques of this disclosure are not necessarily limited to wireless applications or configurations. The techniques may be applied to video coding that supports any of a variety of multimedia applications, such as wireless television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, the coding system 10 may be configured to support unidirectional or bidirectional video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video phone calls.

[0076] In the example shown in Figure 1, the source device 12 includes a video source 18 configured to provide a stereoscopic image, a projection device 20, a video encoder 22, and an output interface 24. The destination device 14 includes an input interface 26, a video decoder 28, a reconstruction device 30, and a display device 32. According to this disclosure, the video encoder 22 of the source device 12 and / or the video decoder 28 of the destination device 14 may be configured to apply techniques for video coding. In other examples, the source and destination devices may include other components or configurations. For example, the source device 12 may receive video data from an external video source, such as an external camera. Similarly, the destination device 14 may interface with an external display device rather than including an integrated display device.

[0077] The coding system 10 illustrated in Figure 1 is merely an example. The technique for video coding may be performed by any digital video coding and / or decoding device. While the technique of this disclosure is generally performed by a coding device, the technique may also be performed by a video encoder / decoder, commonly referred to as a “codec”. Furthermore, the technique of this disclosure may also be performed by a video preprocessor. The encoder and / or decoder may be a graphics processing unit (GPU) or a similar device.

[0078] Source device 12 and destination device 14 are merely examples of coding devices such that source device 12 generates coded video data for transmission to destination device 14. In some examples, source device 12 and destination device 14 can operate substantially symmetrically such that each of source device 12 and destination device 14 has video coding and decoding components. Thus, coding system 10 can support unidirectional or bidirectional video transmission between video devices 12, 14 for, for example, video streaming, video playback, video broadcasting, or video phone calls.

[0079] The video source 18 of source device 12 may include a video capture device such as a video camera, a video archive containing previously captured video, and / or a video feed interface for receiving stereoscopic images or video from a video content provider. As a further alternative, the video source 18 can generate stereoscopic images and computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video.

[0080] In some cases, when the video source 18 is a video camera, the source device 12 and the destination device 14 can form a so-called camera phone or video phone. However, as stated above, the techniques described herein may generally be applicable to video coding and may be applicable to wireless and / or wired applications.

[0081] The projection device 20 is configured to project a three-dimensional image onto a plane (for example, a bounding box), as will be described in more detail below. In other words, the projection device 20 is configured to convert a three-dimensional (3D) image into one or more two-dimensional (2D) images.

[0082] In any case, the stereoscopic image, captured video, pre-captured video, or computer-generated video may be encoded by the encoder 22. The encoded video information may then be output onto the computer-readable medium 16 via the output interface 24.

[0083] The computer-readable medium 16 may include temporary media such as wireless broadcasting or wired network transmission, or storage media (i.e., non-temporary storage media) such as hard disks, flash drives, compact discs, digital video discs, Blu-ray® discs, or other computer-readable media. In some examples, a network server (not shown) may receive encoded video data from a source device 12, for example, via network transmission, and provide the encoded video data to a destination device 14. Similarly, a computing device in a media production facility, such as a disc stamping facility, may receive encoded video data from a source device 12 and produce a disc containing the encoded video data. Thus, the computer-readable medium 16 may be understood to include one or more computer-readable media in various forms in various examples.

[0084] The input interface 26 of the destination device 14 receives information from the computer-readable medium 16. The information on the computer-readable medium 16 may include syntactic information defined by the encoder 22, which is also used by the decoder 28, and includes syntactic elements that describe the characteristics and / or processing of blocks and other coded units, such as picture groups (GOPs).

[0085] The restoration device 30 is configured to convert one or more planar images back into three-dimensional images, as will be described in more detail below. That is, the restoration device 30 is configured to convert one or more 2D images back into three-dimensional images.

[0086] The display device 32 displays stereoscopic images or decoded video data to the user and may include any of the various display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or another type of display device.

[0087] The encoder 22 and decoder 28 can operate in accordance with video coding standards, such as the High Efficiency Video Coding (HEVC) standard currently under development, and can comply with the HEVC Test Model (HM). Alternatively, the encoder 22 and decoder 28 can operate in accordance with other proprietary or industry standards, such as the International Telecommunication Union Standardization Sector (ITU-T) H.264 standard, which is also called Video Expert Group (MPEG)-4, Part 10, Advanced Video Coding (AVC), H.265 / HEVC, or extensions of such standards. However, the techniques of this disclosure are not limited to any particular coding standard. Other examples of video coding standards include MPEG-2 and ITU-T H.263. Although not shown in Figure 1, in some embodiments, the encoder 22 and decoder 28 may be integrated with an audio encoder and audio decoder, respectively, and may include a suitable multiplexer-demultiplexer (MUX-DEMUX) unit, or other hardware and software, to handle the coding of both audio and video on a common data stream or separate data streams. Where applicable, the MUX-DEMUX unit may comply with the ITU H.223 Multiplexer Protocol or other protocols such as the User Datagram Protocol (UDP).

[0088] The encoder 22 and decoder 28 may each be implemented as one or more suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technique is partially implemented in software, the device can store instructions for the software in a suitable non-temporary computer-readable medium and execute the instructions in hardware using one or more processors to perform the technique of this disclosure. Each of the encoder 22 and decoder 28 may be included in one or more encoders or decoders, any of which may be integrated as part of a composite encoder / decoder (codec) in each device. A device including the encoder 22 and / or decoder 28 may include an integrated circuit, a microprocessor, and / or a wireless communication device such as a mobile phone.

[0089] Figure 2 is a block diagram showing an example of an encoder 22 capable of performing video coding techniques. The encoder 22 can perform intracoding and intercoding of video blocks within a video slice. Intracoding relies on spatial prediction to reduce or remove spatial redundancy in video within a given video frame or video picture. Intercoding relies on temporal prediction to reduce or remove temporal redundancy in video within adjacent frames or pictures in a video sequence. Intra-mode (I-mode) can refer to any of several spatial-based coding modes. Inter-modes, such as unidirectional (also known as single-prediction) prediction (P-mode) or bidirectional prediction (also known as bidirectional prediction) (B-mode), can refer to any of several temporal-based coding modes.

[0090] As shown in Figure 2, the encoder 22 receives the current video block in the video frame to be encoded. In the example in Figure 2, the encoder 22 includes a mode selection unit 40, a reference frame memory 64, an adder 50, a transformation unit 52, a quantization unit 54, and an entropy coding unit 56. Subsequently, the mode selection unit 40 includes a motion compensation unit 44, a motion estimation unit 42, an intra-prediction (also known as intra-predict) unit 46, and a splitting unit 48. For video block reconstruction, the encoder 22 also includes an inverse quantization unit 58, an inverse transformation unit 60, and an adder 62. A deblocking filter (not shown in Figure 2) may also be included to filter block boundaries and remove blocking artifacts from the reconstructed video. If necessary, the deblocking filter typically filters the output of the adder 62. In addition to the deblocking filter, further filters (in-loop or after-loop) may be used. Such filters are not shown for brevity, but the output of adder 50 can be filtered (as an in-loop filter) if necessary.

[0091] During the encoding process, encoder 22 receives video frames or video slices to be encoded. The frames or slices may be divided into multiple video blocks. Motion estimation unit 42 and motion compensation unit 44 perform interpredictive coding of the received video blocks for one or more blocks within one or more reference frames to provide temporal predictions. Alternatively, intrapredictive unit 46 can provide spatial predictions by performing intrapredictive coding of the received video blocks for one or more adjacent blocks within the same frame or slice as the blocks to be encoded. Encoder 22 can perform multiple coding passes to select, for example, an appropriate coding mode for each block of video data.

[0092] Furthermore, the splitting unit 48 can divide blocks of video data into subblocks based on an evaluation of a previous splitting scheme in a previous coding pass. For example, the splitting unit 48 can first split a frame or slice into large coding units (LCUs) and then split each LCU into subcoding units (subCUs) based on rate distortion analysis (e.g., rate distortion optimization). The mode selection unit 40 can further generate a quadtree data structure indicating that the LCUs have been split into subCUs. The leaf nodes CU of the quadtree may include one or more prediction units (PUs) and one or more transformation units (TUs).

[0093] This disclosure uses the term “block” to refer to any CU, PU, ​​or TU in the context of HEVC, or similar data structures in the context of other standards (e.g., macroblocks and their subblocks in H.264 / AVC). A CU includes a coding node, a PU, and a TU associated with the coding node. The size of a CU corresponds to the size of the coding node and its shape is square. The size of a CU may range from 8x8 pixels to the size of a tree block of 64x64 pixels or more. Each CU may contain one or more PUs and one or more TUs. Syntactic data associated with a CU may, for example, describe the division of the CU into one or more PUs. The division mode may differ depending on whether the CU is encoded in skip mode or direct mode, intra-predictive mode, or inter-predictive (also known as inter-predict) mode. A PU may be divided into a non-square shape. Syntactic data associated with a CU may also describe, for example, the division of the CU into one or more TUs according to a quadtree. A TU may be square or non-square (e.g., rectangular) in shape.

[0094] The mode selection unit 40 selects one of the intra or intercoding modes, for example, based on the error result, and provides the resulting intra-coded or inter-coded block to the adder 50 to generate residual block data, and provides it to the adder 62 to reconstruct the coded block for use as a reference frame. The mode selection unit 40 also provides syntactic information such as motion vectors, intra-mode indicators, division information, and other such syntactic information to the entropy coding unit 56.

[0095] The motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are shown separately for conceptual purposes. The motion estimation performed by the motion estimation unit 42 is the process of generating motion vectors that estimate the motion of a video block. The motion vectors may, for example, show the displacement of the PU of a video block in the current video frame or current video picture relative to a predicted block in a reference frame (or other coding unit) relative to the current block coded in the current frame (or other coding unit). The predicted block is a block that is deemed to be an exact match to the coded block in terms of pixel difference, which may be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some examples, the encoder 22 may calculate values ​​for fractional pixel positions of the reference picture stored in the reference frame memory 64. For example, the encoder 22 may interpolate values ​​for 1 / 4 pixel position, 1 / 8 pixel position, or other fractional pixel positions of the reference picture. Therefore, the motion estimation unit 42 can perform motion search for full pixel positions and fractional pixel positions and output motion vectors for fractional pixel predictions.

[0096] The motion estimation unit 42 calculates a motion vector for the PU of a video block in an intercoded slice by comparing the PU's position with the predicted block's position in a reference picture. The reference picture may be selected from a first reference picture list (list 0) or a second reference picture list (list 1), each of which identifies one or more reference pictures stored in the reference frame memory 64. The motion estimation unit 42 transmits the calculated motion vector to the entropy coding unit 56 and the motion compensation unit 44.

[0097] The motion compensation performed by the motion compensation unit 44 may include the extraction or generation of a predicted block based on the motion vector determined by the motion estimation unit 42. Again, in some examples, the motion estimation unit 42 and the motion compensation unit 44 may be functionally integrated. Upon receiving the motion vector for the PU of the current video block, the motion compensation unit 44 can identify the location of the predicted block pointed to by the motion vector in one of the reference picture lists. The adder 50 forms the residual video block by subtracting the pixel values ​​of the predicted block from the coded pixel values ​​of the current video block, as described later, to form a pixel difference value. Generally, the motion estimation unit 42 performs motion estimation for the luminance component, and the motion compensation unit 44 uses the motion vector calculated based on the luminance component for both the chrominance and luminance components. The mode selection unit 40 also generates syntactic elements associated with the video block and video slice for use by the decoder 28 when decoding the video block of the video slice.

[0098] The intra-prediction unit 46 can intra-predict the current block instead of inter-predicting performed by the motion estimation unit 42 and the motion compensation unit 44, as described above. More specifically, the intra-prediction unit 46 can determine an intra-prediction mode and use it to encode the current block. In some examples, the intra-prediction unit 46 can encode the current block using various intra-prediction modes, for example, between separate encoding passes, and the intra-prediction unit 46 (or, in some examples, the mode selection unit 40) can select and use an appropriate intra-prediction mode from the tested modes.

[0099] For example, the intra-prediction unit 46 can use rate distortion analysis to calculate rate distortion values ​​for various tested intra-prediction modes and select the intra-prediction mode with the best rate distortion characteristics among the tested modes. Rate distortion analysis generally identifies the amount of distortion (or error) between a coded block and the original uncoded block that coded and generated the coded block, as well as the bit rate (i.e., number of bits) used to generate the coded block. The intra-prediction unit 46 can calculate a ratio from the distortion and rate for various coded blocks to determine which intra-prediction mode represents the best rate distortion value for a given block.

[0100] In addition, the intra-prediction unit 46 may be configured to encode depth blocks of the depth map using a depth modeling mode (DMM). The mode selection unit 40 can, for example, use rate-distortion optimization (RDO) to determine whether an available DMM produces better coding results than the intra-prediction mode and other DMM modes. Data for texture images corresponding to the depth map may be stored in the reference frame memory 64. The motion estimation unit 42 and the motion compensation unit 44 may also be configured to interpret depth blocks of the depth map.

[0101] After selecting an intra-prediction mode for a block (for example, one of the conventional intra-prediction modes or DMM modes), the intra-prediction unit 46 may provide the entropy coding unit 56 with information indicating the selected intra-prediction mode for the block. The entropy coding unit 56 may encode the information indicating the selected intra-prediction mode. The encoder 22 may include transmitted bitstream configuration data, which may include multiple intra-prediction mode index tables and multiple modified intra-prediction mode index tables (also called codeword mapping tables), definitions for encoding contexts for various blocks, and instructions for the most likely intra-prediction mode, intra-prediction mode index table, and modified intra-prediction mode index table to use for each of the contexts.

[0102] The encoder 22 forms a residual video block by subtracting predicted data from the mode selection unit 40 from the original video block that has been coded. The adder 50 represents one or more components that perform this subtraction operation.

[0103] The transformation processing unit 52 applies a transformation, such as a discrete cosine transform (DCT) or a conceptually similar transformation, to the residual block to generate a video block containing residual transformation coefficient values. The transformation processing unit 52 can perform other transformations conceptually similar to the DCT. Wavelet transforms, integer transforms, subband transforms, or other types of transformations may also be used.

[0104] The conversion processing unit 52 applies the conversion to the residual block to generate a block of residual conversion coefficients. The conversion can convert residual information from the pixel value domain to a conversion domain such as the frequency domain. The conversion processing unit 52 can then transmit the resulting conversion coefficients to the quantization unit 54. The quantization unit 54 quantizes the conversion coefficients to further reduce the bit rate. The quantization process can reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can then perform a scan of the matrix containing the quantized conversion coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0105] Following quantization, the entropy coding unit 56 entropy codes the quantization transformation coefficients. For example, the entropy coding unit 56 can perform context-adaptive variable-length coding (CAVLC) or context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), stochastic interval partitioning entropy (PIPE) coding, or another entropy coding technique. In the case of context-based entropy coding, the context may be based on adjacent blocks. Following entropy coding by the entropy coding unit 56, the encoded bitstream may be sent to another device (e.g., decoder 28) or archived for later transmission or retrieval.

[0106] The inverse quantization unit 58 and the inverse transform unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual blocks in the pixel region for later use, for example, as a reference block. The motion compensation unit 44 can calculate a reference block by adding the residual blocks to a predicted block of one of the frames in the reference frame memory 64. The motion compensation unit 44 can also apply one or more interpolation filters to the reconstructed residual blocks to calculate fractional pixel values ​​for use in motion estimation. The adder 62 adds the reconstructed residual blocks to the motion-compensated predicted blocks generated by the motion compensation unit 44 to produce a reconstructed video block for storage in the reference frame memory 64. The reconstructed video block may be used as a reference block by the motion estimation unit 42 and the motion compensation unit 44 to intercode blocks in subsequent video frames.

[0107] Figure 3 is a block diagram showing an example of a decoder 28 capable of performing video coding techniques. In the example in Figure 3, the decoder 28 includes an entropy decoding unit 70, a motion compensation unit 72, an intra-prediction unit 74, an inverse quantization unit 76, an inverse transform unit 78, a reference frame memory 82, and an adder 80. In some examples, the decoder 28 can perform a decoding path that is the overall inverse of the coding path described with respect to the encoder 22 (Figure 2). The motion compensation unit 72 can generate prediction data based on motion vectors received from the entropy decoding unit 70, and the intra-prediction unit 74 can generate prediction data based on an intra-prediction mode indicator received from the entropy decoding unit 70.

[0108] During the decoding process, the decoder 28 receives from the encoder 22 the encoded video bitstream representing the video blocks of the encoded video slice, and associated syntactic elements. The entropy decoding unit 70 of the decoder 28 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-predictive mode indicators, and other syntactic elements. The entropy decoding unit 70 transfers the motion vectors and other syntactic elements to the motion compensation unit 72. The decoder 28 can receive syntactic elements at the video slice level and / or video block level.

[0109] When a video slice is coded as an intra-coded (I) slice, the intra-prediction unit 74 can generate prediction data for the video block of the current video slice based on the intra-prediction mode and data signaled from blocks decoded before the current frame or picture. When a video frame is coded as an intercoded (e.g., B, P, or GPB) slice, the motion compensation unit 72 generates prediction blocks for the video block of the current video slice based on motion vectors and other syntactic elements received from the entropy decoding unit 70. The prediction blocks may be generated from one of the reference pictures in one of the reference picture lists. The decoder 28 can construct the reference frame lists, List 0 and List 1, using default construction techniques based on the reference pictures stored in the reference frame memory 82.

[0110] The motion compensation unit 72 determines prediction information about the video block of the current video slice by parsing motion vectors and other syntactic elements, and uses the prediction information to generate prediction blocks about the current video block being decoded. For example, the motion compensation unit 72 uses some of the received syntactic elements to identify the prediction mode used to encode the video block of the video slice (e.g., intra-prediction or inter-prediction), the inter-prediction slice type (e.g., B-slice, P-slice, or GPB-slice), construction information about one or more of the reference picture lists for the slice, motion vectors for each intercoded video block of the slice, the inter-prediction state for each intercoded video block of the slice, and other information for decoding the video block in the current video slice.

[0111] The motion compensation unit 72 can also perform interpolation based on an interpolation filter. The motion compensation unit 72 can calculate interpolated values ​​for fractional pixels of a reference block using the interpolation filter used by the encoder 22 during the encoding of the video block. In this case, the motion compensation unit 72 can determine the interpolation filter used by the encoder 22 from the received syntactic elements and use the interpolation filter to generate a predicted block.

[0112] The data for the texture image corresponding to the depth map may be stored in the reference frame memory 82. The motion compensation unit 72 may also be configured to interpret depth blocks of the depth map.

[0113] Traditional coding techniques require patches to be ordered according to their patch mode within a patch frame data unit (also known as a patch buffer). That is, the representation of patch data unit types within an auxiliary information patch data unit in video-based point cloud coding (V-PCC) requires all patch-related information to be arranged in a specific order within the patch frame data unit. First, matched patch data units are added to the patch data unit list, followed by mismatched patch data units. Matched patch data units may be coded using differential coding (interpredictive - dpdu), while mismatched patch data units are coded using absolute values ​​(intrapredictive). That is, all intercoded patches are first enumerated within the patch frame data unit, followed by all intracoded patches, and so on. Therefore, adding a new intercoded patch or a new intracoded patch to the patch frame data unit requires the entire patch frame data unit to be reconstructed using the required order, resulting in inefficient coding.

[0114] This specification discloses a coding technique that enables flexible ordering of patches within a patch frame data unit. That is, patches having different patch modes (e.g., inter, intra, PCM, etc.) may be included in a patch frame data unit in any order. The coding technique also allows the use of skip patch data unit types (spdu). A skip patch data unit indicates that all parameters for the current patch may be inherited from the reference patch when the patch data unit elements for the current patch data unit and the patch data unit elements for the reference patch data unit are identical or exactly the same. The coding technique also allows the removal of syntactic elements indicating the number of matching patches from the signaling, since the signaling of the patch types described herein is sufficient to represent the entire list of patches within the patch frame data unit. In addition, the coding technique eliminates the need to signal the total number of patches in the patch frame data unit (e.g., the size of the patch frame data unit), replacing that process with a special termination patch type. Furthermore, the coding technique allows pulse-coded modulation (PCM) patches, which are typically located at the end of a patch frame data unit, to be replaced with a PCM patch type indication. Therefore, the coder / decoder (also known as "codec") in video coding will be improved over current codecs (for example, the process of encoding and decoding patches in V-PCC). In practical terms, an improved video coding process can increase coding efficiency, thereby providing users with a better user experience when point clouds are transmitted, received, and / or viewed.

[0115] Figure 4 shows a sequence 400 of point cloud frames 402, 404, and 406 (e.g., frame PCC_0, frame PCC_1, frame PCC_4), each containing point cloud 408. Point cloud 408 is a three-dimensional representation of space on a normal 3D grid; that is, point cloud 408 is three-dimensional (3D). As shown in Figure 4, point cloud 408 contains point cloud content 410 within 3D space 412. Point cloud content 410 is represented by a set of points (e.g., voxels) within 3D space 412. A voxel is a volume element that represents some numerical value, such as the color of a point in 3D space, and is used for visualization and analysis of 3D data. Thus, a voxel can be thought of as the 3D equivalent of a pixel in a 2D image.

[0116] Each voxel in the point cloud 408 in Figure 4 has coordinates (e.g., xyz coordinates) and one or more attributes (e.g., red / green / blue (RGB) color components, reflectance, etc.). Although the point cloud content 410 in Figure 4 depicts a person, the point cloud content 410 could be any other 3D object or 3D image in actual use.

[0117] Figure 5 shows the point cloud 400 from Figure 4 projected onto the bounding box 500. As shown in Figure 5, the bounding box 500 contains a patch 502 projected onto its two-dimensional (2D) surface or plane 504. Thus, the patch 502 is a 2D representation of a portion of the 3D image. The patch 502 collectively corresponds to the point cloud content 410 in Figure 4. The data representation in video-based point cloud coding (V-PCC) is sometimes called point cloud compression and relies on this 3D-to-2D conversion.

[0118] The data representation in V-PCC is described as a set of planar 2D images (e.g., patch 502), using, for example, the occupancy map 610 in Figure 6A, the geometric map 612 in Figure 6B, and the attribute map 614 in Figure 6C.

[0119] Figure 6A is a diagram of the occupancy map 610 corresponding to the 2D projection (e.g., patch 502) from the bounding box 500 in Figure 5. The occupancy map 610 is coded in binary format. For example, 0 represents that a portion of the bounding box 600 is not occupied by one of the patches 602. Those portions of the bounding box 600 represented by 0 are not involved in the reconstruction of the 3D representation (e.g., point cloud content 410). In contrast, 1 represents that a portion of the bounding box 600 is occupied by one of the patches 602. Those portions of the bounding box 600 represented by 1 are involved in the reconstruction of the 3D representation (e.g., point cloud content 410).

[0120] Figure 6B is a diagram of the geometric map 612 corresponding to the 2D projection (e.g., patch 502) from the bounding box 500 in Figure 5. The geometric map 612 provides or depicts the contour or topography of each of the patches 602. That is, the geometric map 612 shows the distance of each point in patch 602 from the flat surface (e.g., plane 504) of the bounding box 600.

[0121] Figure 6C is a diagram of the attribute map 614 corresponding to a 2D projection (e.g., patch 502) from the bounding box 500 in Figure 5. The attribute map 614 provides or depicts the attributes of each point in patch 602 within the bounding box 600. The attributes in the attribute map 614 may be, for example, the color components of the points. The color components may be based on the RGB color model, the YUV color model, or another known color model.

[0122] Figure 7 shows a diagram of the V-PCC bitstream 700. The V-PCC bitstream 700 may be used to carry the encoded information necessary to reconstruct a point cloud (for example, point cloud 408 in Figure 4). As shown, the V-PCC bitstream 700 includes a plurality of V-PCC units 702. Each V-PCC unit 702 includes a V-PCC unit header 704 and a V-PCC unit payload 706. The V-PCC unit header 704 describes the V-PCC unit payload 706 carried by the V-PCC unit 702.

[0123] The V-PCC unit payload 706 includes a sequence parameter set 708, patch sequence data 710, occupied video data 712, geometric video data 714, and attribute video data 716 (e.g., texture video data). The sequence parameter set 708 includes parameters and information corresponding to a sequence of point clouds (e.g., a sequence 400 of point cloud frames 402, 404, and 406, each containing point cloud 408 in Figure 4). The occupied video data 712, geometric video data 714, and attribute video data 716 carry occupied data, geometric data, and attribute or texture data corresponding to the point cloud or 3D image.

[0124] The patch sequence data 710 includes parameters and information corresponding to the sequence of patches (for example, patch 502 in Figure 5). The patch sequence data 710 includes a sequence parameter set 720, a frame parameter set 722, a geometric parameter set 724, an attribute parameter set 726, a geometric patch parameter set 728, an attribute patch parameter set 730, and a patch frame 732.

[0125] The sequence parameter set 720 contains parameters and information corresponding to a sequence of 2D patches (e.g., patch 502 in Figure 5) that persists for the entire duration of the point cloud sequence or until a new sequence parameter set is signaled in the bitstream. The frame parameter set 722 contains information corresponding to a sequence of 2D patches that persists for the duration of a single frame of the point cloud sequence. The geometric parameter set 724 contains information corresponding to the process of reconstructing the geometric properties of the patch sequence data. The attribute parameter set 726 contains information corresponding to the process of reconstructing the attribute properties of the patch sequence data. The geometric patch parameter set 728 contains information corresponding to the decoding process of the geometric properties of the patch sequence data. The attribute 730 contains information corresponding to the decoding process of the attribute properties of the patch sequence data.

[0126] A patch frame data unit 732 (called a patch frame) contains a set of patch data units 734. A patch data unit 734 contains patch data (e.g., patch data F0, ..., Fk) for a patch (e.g., patch 502) in a batch. Each of the patch data units 734 has a patch type, which is referred to herein as a patch mode. As will be described in more detail below, a patch mode (e.g., patch_mode) indicates how a particular patch should be handled during the coding process.

[0127] Figure 8 is a diagram of the coded decomposition of a point cloud sequence 800 for multiple frames 802. Frames 802 in Figure 8 are similar to point cloud frames 402, 404, and 406 in Figure 4. Thus, each of the frames 802 contains point cloud content 810 for a 3D point cloud. The point cloud content 810 in each frame 802 in Figure 8 corresponds to the occupancy map 812, geometric map 814, and attribute map 816 in the 2D projection used to reconstruct the point cloud content 810. As illustrated, the 2D projection may contain two or more attribute maps 816.

[0128] As illustrated, the point cloud sequence 800 also includes an auxiliary information patch frame data unit 850 (AUX_INF). In one embodiment, the auxiliary information patch frame data unit 850 includes both 2D and 3D bounding box information (for example, the patch 3D bounding box 1204 and the patch 2D projected bounding box 1208 in Figure 12). The set of auxiliary information patch frame data units 850 corresponds to the patch sequence data 710 in Figure 7. Each auxiliary information patch frame data unit 850 includes a sequence parameter set 720, a frame parameter set 722, a geometric parameter set 724, an attribute parameter set 726, a geometric patch parameter set 728, an attribute patch parameter set 730, and a patch frame 732. In one embodiment, the patch frame 732 is included in the auxiliary information patch frame data unit 850, and the remaining components in the patch sequence data 710 are optional. In one embodiment, the set of occupancy maps 812 corresponds to occupancy video data 712, the set of geometric maps 814 corresponds to geometric video data 714, and the set of attribute maps 816 corresponds to attribute video data 716. In particular, no occupancy maps are present in this image. Each auxiliary information patch data unit 850 corresponds to a point cloud content 810 in one of the frames 802. The patch data unit (e.g., 850) contains information used or required to reconstruct the point cloud content 810 in the 3D point cloud based on the occupancy maps 812, geometric maps 814, and attribute maps 816 in the 2D projection.

[0129] Figure 9 shows a single patch data unit 900 extracted from one of the point cloud frames 802 within the point cloud sequence 800 in Figure 8. The single patch data unit 900 is similar to the patch data unit 734 or auxiliary information patch frame data unit 850 within patch frame 732 in Figure 7. The single patch data unit 900 is depicted relative to the surrounding 3D space 902.

[0130] Figures 10A to 10C show the occupancy map 1000, geometric map 1002, and attribute map 1004 corresponding to the single patch data unit 900 in Figure 9. The occupancy map 1000, geometric map 1002, and attribute map 1004 are similar to the occupancy map 610, geometric map 612, and attribute map 614 in Figures 6A to 6C. However, the occupancy map 1000, geometric map 1002, and attribute map 1004 in Figures 10A to 10C represent only the single patch data unit 900, and not the collection of patches 502 from the entire bounding box 500. Therefore, the occupancy map 1000, geometric map 1002, and attribute map 1004 in Figures 10A to 10C contain only the single patch 1006 corresponding to the single patch data unit 900 in Figure 9.

[0131] Figure 11 shows the 2D components for a patch data unit 1152 within a 2D bounding box 1154. The patch data unit 1152 in Figure 11 is similar to the single patch data unit 900 in Figure 9. As shown in Figure 11, the 2D components represent the patch coordinates (e.g., [u0,v0]) and dimensions (e.g., Size_u0, Size_v0) for the patch data unit 1152. These 2D components are contained within the patch data unit (e.g., patch data unit 734 in Figure 7 or patch data unit 850 in Figure 8).

[0132] Figures 12A and 12B are schematic diagrams 1200 and 1252 of the 3D components for the patch data unit 1152 of Figure 11. As illustrated, schematic diagram 1200 includes the point cloud 3D bounding box 1202, the patch 3D bounding box 1204, the projection plane 1206, and the patch 2D projection bounding box 1208. Various coordinates (e.g., [0,0,0]) and dimensions (e.g., x, y, z, u, v, d1, u1, v1) and vectors (b,t,bt) are used to describe the process of projecting the contents of the point cloud 3D bounding box 1202 onto the patch 2D projection bounding box 1208. Dataset Figure 1252 illustrates how a single patch data unit 1254 (similar to a single patch data unit 900) can be projected onto the projection plane 1206 based on coordinates (e.g., [0,0,0], [u1,v1,d1]) and vectors (e.g., n,bt). These 3D components are also included in the patch data units (e.g., patch data unit 734 in Figure 7 or patch data unit 850 in Figure 8).

[0133] The patch data unit (for example, patch data unit 734 in Figure 7 or patch data unit 850 in Figure 8) contains the information necessary for the 2D to 3D restoration process, as follows (see Figure 11): 2D related components U0 (Patch2dShiftU) V0(Patch2dShiftV) Size_u0(Patch2dSizeU) Size_v0(Patch2dSizeV) And the following (see Figures 12A-12B): 3D related components: u1 (Patch3dShiftTangentAxis) - Distance from the 3D patch bounding box to the point cloud bounding box along the tangent axis (t). v1 (Patch3dShiftBiTangentAxis) - Distance from the 3D patch bounding box to the point cloud bounding box along the deferred line axis (bt). d1 (Patch3dShiftNormalAxis) - Distance from the 3D patch bounding box to the point cloud bounding box along the normal axis (n) n (PatchNormalAxis) - A (normal) axis perpendicular to the patch projection plane (any side of the bounding box can be the projection plane). t (Patch3dShiftTangentAxis) - An axis tangent to the patch surface and perpendicular to the normal axis. bt (Patch3dShiftBiTangentAxis) - Axis perpendicular to the normal and tangent axes. It is represented as follows.

[0134] As described above, conventional coding techniques require that patches (patch data units 734 in Figure 7) be ordered according to their patch mode within the patch frame data unit. That is, the representation of patch data unit types within the patch frame data unit in V-PCC requires that all patch-related information be arranged in a specific order within the patch frame data unit. First, matched patch data units are added to the patch data frame, followed by mismatched patch data units. Matched patch data units may be coded using differential coding (interpredictive - dpdu), and mismatched patch data units may be coded using absolute values ​​(intrapredictive). That is, all intercoded patches are first enumerated within the patch frame data unit, followed by all intracoded patches, and so on. Therefore, in order to add a new intercoded patch or a new intracoded patch to the patch frame data unit, the entire patch frame data unit must be reconstructed using the required order, resulting in inefficient coding. The method detailed below overcomes these coding inefficiencies.

[0135] Figure 13 shows one embodiment of the patch frame data unit coding process 1300. The patch frame data unit coding process 1300 may be used to code (also known as compress) multiple patch data units, such as the patch data unit 734 in Figure 7. The patch frame data unit coding process 1300 may be performed by an encoder. Here, the patch frame data unit corresponding to the 2D bounding box 1154 is a set of patch data units corresponding to the patch data unit 1152. In one embodiment, the input to the coding process 1300 is a set of patch data units corresponding to patch frame data units.

[0136] In block 1302, a patch frame data unit (pfdu) is acquired. The patch frame data unit may be received by the encoder's receiver. The patch frame data unit consists of a set of patch data units. For each of the multiple patch data units (e.g., patch 502), the patch data unit identifies or includes a patch type (pdfu_patch_mode). In one embodiment, the patch type is skip, intra, inter, PCM, or one of the last. The skip patch type indicates that the current patch is identical to a reference patch. Therefore, parameters from the reference patch are used when coding the current patch. The intra patch type indicates that the current patch is coded using intra prediction (also known as intra coding). The inter patch type indicates that the current patch is coded using inter prediction (also known as intercoding). The PCM patch type indicates that the current patch represents a scattered point independent of the 3D point cloud. The last patch type indicates that the current patch is the last patch to be coded, and therefore the compression process of the patch frame data unit should be finished or terminated.

[0137] In block 1304, the patch data unit index (p) is decremented. By decrementing the patch data unit index, it can start from a negative value (for example, -1). Thus, when the patch data unit index is incremented in block 1306, the patch data unit index is effectively initialized to start from zero. In one embodiment, process 1300 may also initially set the patch data unit index to 0 and then increment the patch data unit index after a loop or iteration portion of the process has been executed (as described below).

[0138] In block 1308, it is determined whether the patch data unit type for the current patch data unit from multiple patch data units within the patch frame data unit is the last patch type. If the current patch data unit is not the last patch, process 1300 proceeds to block 1310. In block 1310, patch information data is encoded for the current patch. In one embodiment, the patch information data includes a frame index (frmIdx), a patch index (p), a patch type (pdfu_patch_mode), and corresponding 2D and 3D information. After encoding in block 1310, process 1300 loops back to block 1306, where the patch data unit index is incremented. By incrementing the patch index, a new current patch data unit is considered. These steps are repeated until it is determined in block 1308 that the current patch data unit is the last patch.

[0139] If the current patch data unit is determined to be the last patch data unit, process 1300 proceeds to block 1312. In block 1312, patch information data is encoded for the current patch data unit. In one embodiment, it should be noted that the patch information data includes a frame index (frmIdx), a patch index (p), and a patch type (pdfu_patch_mode), and the last patch data unit does not have 2D or 3D information associated with the patch data unit. Unlike block 1310, the patch type in block 1312 is encoded as the last patch type.

[0140] In block 1314, a byte alignment process is performed. In one embodiment, byte alignment organizes or arranges the bytes of data to a desired orientation or size. Then, in block 1316, a compressed patch frame data unit is realized. The compressed patch frame data unit may be incorporated into a bitstream for transmission to a decoder.

[0141] Figure 14 shows one embodiment of the patch frame data unit coding process 1400 that uses the last patch flag. The patch frame data unit coding process 1400 may be used to code (also known as compress) multiple patch data units, such as the patch data unit 734 in Figure 7. The patch frame data unit coding process 1400 may be performed by an encoder.

[0142] In block 1402, a patch frame data unit (pfdu) is acquired. The patch frame data unit may be received by the encoder's receiver. The patch frame data unit contains multiple patch data units (for example, patch 502).

[0143] In block 1404, the patch index (p) is decremented. By decrementing the patch index, the patch index can start from a negative value (for example, -1). In block 1406, the last patch flag is added to each of the multiple patches. In one embodiment, the last patch flag is specified as pfdu_is_last_patch_flag. The last patch flag may be set or assigned a value depending on the patch type of each patch. For example, the patch flag may indicate that the patch type for the patch is skip, intra, inter, PCM, or last.

[0144] In block 1408, the patch index is incremented to effectively initialize the patch index to start from zero. In block 1410, it is determined whether the patch type for the current patch data unit from multiple patch data units corresponding to the patch frame data unit is the last patch type, based on the value of the last patch flag. If the current patch data unit is not the last patch, process 1400 proceeds to block 1412. In block 1412, patch information data is encoded for the current patch data unit. In one embodiment, the patch information data includes the frame index (frmIdx), the patch index (p), and the patch type (pdfu_patch_mode), as well as the corresponding 2D and 3D information. In block 1412 of process 1400, the patch type may be skip, inter, intra, or PCM, but not the last. In block 1414, the last patch flag is encoded.

[0145] After encoding in block 1414, process 1400 loops back to block 1408, where the patch index is incremented. Incrementing the patch index ensures that the new current patch is considered. These steps are repeated until, based on the value of the last patch flag, the current patch is determined to be the last patch in block 1410.

[0146] If the current patch is determined to be the last patch based on the last patch flag, process 1400 proceeds to block 1416. In block 1416, a byte alignment process is performed. Then, in block 1418, the compressed patch frame data unit is realized. The compressed patch frame data unit may be incorporated into a bitstream for transmission to the decoder.

[0147] Figure 15 shows one embodiment of a patch information data frame 1500 with a description of the patch unit type. The patch information data frame (patch frame data unit) 1500 is similar to the patch frame (patch frame data unit) 732 in Figure 7. As illustrated, the patch information data frame 1500 includes a reference patch frame data unit 1502 (indicated by refFrmIdx), a current patch frame data unit (frmIdx) 1504, and a patch information unit 1506. The reference patch frame data unit 1502 includes, for example, a reference frame used in relation to interpretation or skip prediction. The current patch frame data unit 1504 includes multiple patch data units that are being encoded or decoded. The patch information data unit 1506 includes a patch data unit for a patch (e.g., patch 502) having one of the available patch types. In one embodiment, the patch data unit may be an intra-patch type (pdu) 1508, a PCM patch type (ppdu) 1510, an inter-patch type (dpdu) 1512, and a skip patch type (spdu) 1514. Any number of patches can exist in the patch information unit 1506, and the patches can be in any order.

[0148] Figure 16 shows one embodiment of the patch frame data unit decoding process 1600. The patch frame data unit decoding process 1600 may be used to decode (also known as decompress) a stereoscopic image encoded by an encoder. The patch frame data unit decoding process 1600 may be performed by an encoder (for example, an entropy decoding unit 70).

[0149] In block 1602, encoded patch information data is received and read. The encoded patch information data may be received by the decoder's receiver as part of a bitstream. The encoded patch information data contains the data and information necessary to decode the stereoscopic image encoded by the encoder. For example, the encoded patch information data is compressed and contains multiple patch data units (e.g., patch 502 or patch data unit 850) necessary to reconstruct a stereoscopic image (e.g., a 3D point cloud) from the patch. As part of block 1602, when process 1600 is started, a patch (e.g., an initial patch data unit or the current patch data unit) may be retrieved from multiple patches.

[0150] In block 1604, an input corresponding to encoded patch information data is received. In one embodiment, the input corresponds to one or more compressed patches. In one embodiment, the input comprises one or more of the following: patch_mode, patch index, reference index, frame index, and reference frame index, corresponding to the patch.

[0151] In block 1606, it is determined whether the patch type for the patch is a skip patch type. If the patch type for the patch is a skip patch type, process 1600 proceeds to block 1608. In block 1608, the reference patch index (spdu_patch_index) corresponding to the patch is decoded. That is, the value of the reference patch index is decoded. In one embodiment, spdu_patch_index is the index of the patch in the reference frame, indicated by refFrmIdx.

[0152] In block 1610, the reference index for a patch (refIdx) is determined based on the reference frame index ([refFrmIdx]) corresponding to the patch and the decoded reference patch index when the patch type is a skip patch type. In one embodiment, the reference index, sometimes called the reference patch index (RefPatchIdx), is determined based on RefPatchIdx = predictorIdx + spdu_patch_index[frmIdx][p], where predictorIdx is the prediction index, spdu_patch_index is the skip type patch data unit index, frmIdx is the frame index, and p is the patch index. In one embodiment, the current patch frame data unit index (frmIdx) and pointers to the correct patch data unit in the patch index (p) point to the appropriate spot in a multidimensional array to find the actual reference index for the patch.

[0153] In block 1612, the patch is restored based on the reference index determined when the patch type is a skip patch type. In one embodiment, the patch is restored as follows: Restoration: Patch2dShiftU[frmIdx][p]=Patch2dShiftU[refIdx][RefPatchIdx]*ops_occupancy_packing_block_size Patch2dShiftV[frmIdx][p]=Patch2dShiftV[refIdx][RefPatchIdx]*ops_occupancy_packing_block_size Patch2dSizeU[frmIdx][p]=Patch2dSizeU[refIdx][RefPatchIdx]*ops_occupancy_packing_block_size Patch2dSizeV[frmIdx][p]=Patch2dSizeV[refIdx][RefPatchIdx]*ops_occupancy_packing_block_size Patch3dShiftTangentAxis[frmIdx][p]=Patch3dShiftTangentAxis[refIdx][RefPatchIdx] Patch3dShiftBiTangentAxis[frmIdx][p]=Patch3dShiftBiTangentAxis[refIdx][RefPatchIdx] Patch3dShiftNormalAxis[frmIdx][p]=Patch3dShiftNormalAxis[refIdx][RefPatchIdx] PatchNormalAxis[frmIdx][p]=PatchNormalAxis[refIdx][RefPatchIdx] PatchTangentAxis[frmIdx][p]=PatchTangentAxis[refIdx][RefPatchIdx] PatchBiTangentAxis[frmIdx][p]=PatchBiTangentAxis[refIdx][RefPatchIdx] PatchOrientationSwapFlag[frmIdx][p]=PatchOrientationSwapFlag[refIdx][RefPatchIdx] PatchLod[frmIdx][p]=PatchLod[refIdx][RefPatchIdx] PatchProjectionMode[frmIdx][p]=dpdu_projection_mode[frmIdx][p]

[0154] In one embodiment, the patch is restored as follows: Restoration: Patch2dShiftU[p]=pdu_2d_shift_u[refIdx] Patch2dShiftV[p]=pdu_2d_shift_v[refIdx] Patch2dSizeU[p]=Patch2dSizeU[refIdx] Patch2dSizeV[p]=Patch2dSizeV[refIdx] Patch3dShiftT[p]=Patch3dShiftT[refIdx] Patch3dShiftBT[p]=Patch3dShiftBT[refIdx] Patch3dShiftN[p]=Patch3dShiftN[refIdx] PatchNormalAxis[p]=PatchNormalAxis[refIdx] Orientation[p]=Orientation[refIdx] PatchLod[p]=PatchLod[refIdx]

[0155] In block 1650, the recovered patch information data is collected (for example, stored in memory), and process 1600 loops back to block 1602, where the new patch can be decoded.

[0156] In block 1614, it is determined whether the patch type for the patch data unit is an intra-patch type. If the patch type for the patch is an intra-patch type, process 1600 proceeds to block 1616. In block 1616, the reference patch data unit index (refIdx) for the patch data unit is determined based on the previous patch data unit (corresponding to the patch and decremented patch index) in the current frame index ([frmIdx]). In one embodiment, the current frame (e.g., the patch frame data unit) is used as the reference.

[0157] In block 1618, the patch is decoded using the two-dimensional (2D) component and the three-dimensional (3D) component corresponding to the patch. In one embodiment, the patch is decoded as follows: Decryption operation: u0(pdu_2d_shift_u) u1(pdu_2d_shift_v) d_size_u0(pdu_2d_delta_size_u) d_size_v0(pdu_2d_delta_size_v) u1(pdu_3d_shift_tangent_axis) v1(pdu_3d_shift_bitangent_axis) d1(pdu_3d_shift_normaI_axis) n(pdu_normI_axis) swap(pdu_orientation_swap_flag) L0D(pdu_Iod)

[0158] In block 1620, the patch is restored based on the decrypted patch. In one embodiment, the patch is decrypted as follows: Restoration: Patch2dShiftU[p]=pdu_2d_shift_u[p] Patch2dShiftV[p]=pdu_2d_shift_v[p] Patch2dSizeU[p]=pdu_2d_delta_size_u[p]+ +Patch2dSizeU[refldx] Patch2dSizeV[p]=pdu_2d_deIta_size_v[p]+ +Patch2dsizeV[refldx] Patch3dShiftT[p]=pdu_3d_shift_tan[p] Patch3dShiftBT[p]=pdu_3d_shift_bitan[p] Patch3dShiftN[p]=pdu_shift_norm[p] PatchNormalAxis[p]=pdu_norm_axis[p] Orientation[p]=pdu_orientation_swap_flag[p] PatchLod[p]=pdu_Iod[p]

[0159] In block 1650, the recovered patch information data is collected (for example, stored in memory), and process 1600 loops back to block 1602, where the new patch can be decoded.

[0160] In block 1622, it is determined whether the patch type for the patch is an interpatch type. If the patch type for the patch is an interpatch type, process 1600 proceeds to block 1624. In block 1624, the reference patch index (dpdu_patch_index) corresponding to the patch is decoded. That is, the value of the reference patch index is decoded. In block 1626, the reference index for the patch (refIdx) is determined based on the reference frame index ([refFrmIdx]) corresponding to the patch and the decoded reference patch index.

[0161] In block 1628, the patch is decoded using the two-dimensional (2D) component and the three-dimensional (3D) component corresponding to the patch. In one embodiment, the patch is decoded as follows: Decryption operation: d_u0(pdu_2d_shift_u) d_u1(pdu_2d_shift_v) d_size_u0(pdu_2d_delta_size_u) d_size_v0(pdu_2d_delta_size_v) d_u1(pdu_3d_shift_tangent_axis) d_v1(pdu_3d_shift_bitangent_axis) d_d1(pdu_3d_shift-normal_axis)

[0162] In block 1630, the patch is restored based on the decrypted patch. In one embodiment, the patch is restored as follows: Restoration: Patch2dShiftU[p]=pdu_2d_shift_u[p]+ +Patch2dShiftU[refldx] Patch2dShiftVf[p]=pdu_2d_shift_v[p]+ +Patch2dshiftV[refldx] Patch2dsizeU[p]=pdu_2d_delta_size_u[p]+ +Patch2dSizeU[refldx] Patch2dSizeV[p]=pdu_2d_delta_size_v[p]+ +Patch2dSizeV[refldx] Patch3dShiftT[p]=pdu_3d_shift_tan[p]+ +Patch3dshiftT[refldx] Patch3dShiftBT[p]=pdu_3d_shift_bitan[p]+ +patch3dshiftBT[refldx] Patch3dShiftN[p]=pdu_shift_norm[p]+ +Patch3dShiftN[refldx] PatchNormaIAxis[p]=PatchnormaIAxis[refldx] Orientation[p]=Orientation[refldx] PatchLod[p]=PatchLod[refldx]

[0163] In block 1650, the recovered patch information data is collected (for example, stored in memory), and process 1600 loops back to block 1602, where the new patch can be decoded.

[0164] In block 1632, it is determined whether the patch type for the patch is a PCM patch (or raw patch) type. If the patch type for the patch is a PCM patch type, process 1600 proceeds to block 1634. In block 1634, the reference index (refIdx) for the patch is determined based on the frame index ([frmIdx]) corresponding to the patch and the decremented patch index.

[0165] In block 1636, the patch is decoded using a two-dimensional (2D) component and a three-dimensional (3D) component corresponding to the patch. In one embodiment, the 2D and 3D components include the following: Decryption operation: separate_video_flag(ppdu_patch…) u0(ppdu_2d_shift_u) u1(ppdu_2d_shift_y) d_size_u0(ppdu_2d_delta_size_u) d_sze_v0(ppdu_2d_delta_size_v) PCM points (ppdu_pcm_points)

[0166] In block 1638, the patch is restored based on the decrypted patch. In one embodiment, the patch is restored as follows: Restoration: Patch2dShiftU[p]=pdu_2d_shift_u[p] Patch2dShiftV[p]=pdu_2d_shifty[p] Patch2dSizeU[p]=pdu_2d_deltasize_u[p]+ +Patch2dSizeU[refIdx] Patch2dSizeV[p]=pdu_2d_deIta_sizev[p]+ +Patch2dSizeV[refldx] PatchPcmPoints[p]=ppdu_pcm_points[p]

[0167] In block 1650, the recovered patch information data is collected (for example, stored in memory), and process 1600 loops back to block 1602, where the new patch can be decoded.

[0168] In block 1640, it is determined whether the patch type for the patch is the last patch type. If the patch type for the patch is the last patch type, process 1600 proceeds to block 1642. In block 1642, if the patch type is the last patch type, the recovery process corresponding to the encoded patch information data is terminated. That is, the recovery process is terminated. In one embodiment, any data contained in the patch having the last patch type may be decoded before the patch frame data unit decoding process 1600 is terminated.

[0169] Figure 17 shows one embodiment of the patch frame data unit decoding process 1700. The patch frame data unit decoding process 1700 may be used to decode (also known as decompress) a stereoscopic image encoded by an encoder. The frame patch data unit decoding process 1700 may be performed by an encoder (for example, an entropy decoding unit 70).

[0170] In block 1702, encoded patch information data is received and read. The encoded patch information data may be received by the decoder's receiver as part of a bitstream. The encoded patch information data contains the data and information necessary to decode the stereoscopic image encoded by the encoder. For example, the encoded patch information data is compressed and contains multiple patch data units (e.g., patch 502 or patch data unit 850) necessary to reconstruct a stereoscopic image (e.g., a 3D point cloud) from the patch. As part of block 1702, when process 1700 is started, a patch (e.g., the initial patch or the current patch) may be retrieved from multiple patches.

[0171] In block 1704, an input corresponding to encoded patch information data is received. In one embodiment, the input corresponds to one or more compressed patches. In one embodiment, the input comprises one or more of the following: patch_mode, patch index, reference index, frame index, and reference frame index, corresponding to the patch.

[0172] In block 1706, it is determined whether the patch type for the patch is a skip patch type. If the patch type for the patch is a skip patch type, process 1700 proceeds to block 1708. In block 1708, the reference patch index (spdu_patch_index) corresponding to the patch is decoded (or determined). That is, the value of the reference patch index is decoded or determined.

[0173] In block 1710, a reference index for a patch (refIdx) is determined based on a reference frame index ([refFrmIdx]) corresponding to the patch, and the reference patch index decoded when the patch type is a skip patch type. In one embodiment, the reference index, sometimes called the reference patch index (RefPatchIdx), is determined based on the following RefPatchIdx = predictorIdx + spdu_patch_index[frmIdx][p], where predictorIdx is the prediction index, spdu_patch_index is the skip type patch data unit index, frmIdx is the frame index, and p is the patch index.

[0174] In block 1712, the patch is restored based on the reference index determined when the patch type is a skip patch type. In one embodiment, the patch is restored as follows: Restoration: Patch2dShiftU[frmIdx][p]=Patch2dShiftU[refIdx][RefPatchIdx]*ops_occupancy_packing_block_size Patch2dShiftV[frmIdx][p]=Patch2dShiftV[refIdx][RefPatchIdx]*ops_occupancy_packing_block_size Patch2dSizeU[frmIdx][p]=Patch2dSizeU[refIdx][RefPatchIdx]*ops_occupancy_packing_block_size Patch2dSizeV[frmIdx][p]=Patch2dSizeV[refIdx][RefPatchIdx]*ops_occupancy_packing_block_size Patch3dShiftTangentAxis[frmIdx][p]=Patch3dShiftTangentAxis[refIdx][RefPatchIdx] Patch3dShiftBiTangentAxis[frmIdx][p]=Patch3dShiftBiTangentAxis[refIdx][RefPatchIdx] Patch3dShiftNormalAxis[frmIdx][p]=Patch3dShiftNormalAxis[refIdx][RefPatchIdx] PatchNormalAxis[frmIdx][p]=PatchNormalAxis[refIdx][RefPatchIdx] PatchTangentAxis[frmIdx][p]=PatchTangentAxis[refIdx][RefPatchIdx] PatchBiTangentAxis[frmIdx][p]=PatchBiTangentAxis[refIdx][RefPatchIdx] PatchOrientationSwapFlag[frmIdx][p]=PatchOrientationSwapFlag[refIdx][RefPatchIdx] PatchLod[frmIdx][p]=PatchLod[refIdx][RefPatchIdx] PatchProjectionMode[frmIdx][p]=dpdu_projection_mode[frmIdx][p]

[0175] In one embodiment, the patch is restored as follows: Restoration: Patch2dShiftU[p]=pdu_2d_shift_u[refIdx] Patch2dShiftV[p]=pdu_2d_shift_v[refIdx] Patch2dSizeU[p]=Patch2dSizeU[refIdx] Patch2dSizeV[p]=Patch2dSizeV[refIdx] Patch3dShiftT[p]=Patch3dShiftT[refIdx] Patch3dShiftBT[p]=Patch3dShiftBT[refIdx] Patch3dShiftN[p]=Patch3dShiftN[refIdx] PatchNormalAxis[p]=PatchNormalAxis[refIdx] Orientation[p]=Orientation[refIdx] PatchLod[p]=PatchLod[refIdx]

[0176] In block 1760, the flag is decoded to determine if there are any more patches to consider. In one embodiment, the flag is specified as more_patches_available_flag. In block 1762, the value of the flag is determined. If the flag has a first value (e.g., 1), process 1700 proceeds to block 1764. In block 1764, the recovered patch information data is stored, and the process loops back to block 1702 so that new patches can be decoded. If the flag has a second value (e.g., 0), process 1700 proceeds to block 1742. In block 1742, if the patch type is the last patch type, the recovery process corresponding to the encoded patch information data is terminated. That is, the recovery process is terminated.

[0177] In block 1714, it is determined whether the patch type for the patch data unit is an intra-patch type. If the patch type for the patch is an intra-patch type, process 1700 proceeds to block 1716. In block 1716, the reference patch data unit index (refIdx) for the patch data unit is determined based on the previous patch data unit (corresponding to the patch and decremented patch index) in the current frame index ([frmIdx]).

[0178] In block 1718, the patch is decoded using the two-dimensional (2D) component corresponding to the patch and the three-dimensional (3D) component corresponding to the patch. In one embodiment, the patch is decoded as follows: Decryption operation: u0(pdu_2d_shift_u) u1(pdu_2d_shift_v) d_size_u0(pdu_2d_delta_size_u) d_size_v0(pdu_2d_delta_size_v) u1(pdu_3d_shift_tangent_axis) v1(pdu_3d_shift_bitangent_axis) d1(pdu_3d_shift_normaI_axis) n(pdu_normI_axis) swap(pdu_orientation_swap_flag) L0D(pdu_Iod)

[0179] In block 1720, the patch is restored based on the decrypted patch. In one embodiment, the patch is decrypted as follows: Restoration: Patch2dShiftU[p]=pdu_2d_shift_u[p] Patch2dShiftV[p]=pdu_2d_shift_v[p] Patch2dSizeU[p]=pdu_2d_delta_size_u[p]+ +Patch2dSizeU[refldx] Patch2dSizeV[p]=pdu_2d_deIta_size_v[p]+ +Patch2dsizeV[refldx] Patch3dShiftT[p]=pdu_3d_shift_tan[p] Patch3dShiftBT[p]=pdu_3d_shift_bitan[p] Patch3dShiftN[p]=pdu_shift_norm[p] PatchNormalAxis[p]=pdu_norm_axis[p] Orientation[p]=pdu_orientation_swap_flag[p] PatchLod[p]=pdu_Iod[p]

[0180] In block 1760, the flag is decoded to determine if there are any more patches to consider. In one embodiment, the flag is specified as more_patches_available_flag. In block 1762, the value of the flag is determined. If the flag has a first value (e.g., 1), process 1700 proceeds to block 1764. In block 1764, the recovered patch information data is stored, and process 1700 loops back to block 1702 so that new patches can be decoded. If the flag has a second value (e.g., 0), process 1700 proceeds to block 1742. In block 1742, if the patch type is the last patch type, the recovery process corresponding to the encoded patch information data is terminated. That is, the recovery process is terminated.

[0181] In block 1722, it is determined whether the patch type for the patch is an interpatch type. If the patch type for the patch is an interpatch type, process 1700 proceeds to block 1724. In block 1724, the reference patch index (dpdu_patch_index) corresponding to the patch is decoded. That is, the value of the reference patch index is decoded. In block 1726, the reference index for the patch (refIdx) is determined based on the reference frame index ([refFrmIdx]) corresponding to the patch and the decoded reference patch index.

[0182] In block 1728, the patch is decoded using the two-dimensional (2D) component and the three-dimensional (3D) component corresponding to the patch. In one embodiment, the patch is decoded as follows: Decryption operation: d_u0(pdu_2d_shift_u) d_u1(pdu_2d_shift_v) d_size_u0(pdu_2d_delta_size_u) d_size_v0(pdu_2d_delta_size_v) d_u1(pdu_3d_shift_tangent_axis) d_v1(pdu_3d_shift_bitangent_axis) d_d1(pdu_3d_shift-normal_axis)

[0183] In block 1730, the patch is restored based on the decrypted patch. In one embodiment, the patch is restored as follows: Restoration: Patch2dShiftU[p]=pdu_2d_shift_u[p]+ +Patch2dShiftU[refldx] Patch2dShiftVf[p]=pdu_2d_shift_v[p]+ +Patch2dshiftV[refldx] Patch2dsizeU[p]=pdu_2d_delta_size_u[p]+ +Patch2dSizeU[refldx] Patch2dSizeV[p]=pdu_2d_delta_size_v[p]+ +Patch2dSizeV[refldx] Patch3dShiftT[p]=pdu_3d_shift_tan[p]+ +Patch3dshiftT[refldx] Patch3dShiftBT[p]=pdu_3d_shift_bitan[p]+ +patch3dshiftBT[refldx] Patch3dShiftN[p]=pdu_shift_norm[p]+ +Patch3dShiftN[refldx] PatchNormaIAxis[p]=PatchnormaIAxis[refldx] Orientation[p]=Orientation[refldx] PatchLod[p]=PatchLod[refldx]

[0184] In block 1760, the flag is decoded to determine if there are any more patches to consider. In one embodiment, the flag is specified as more_patches_available_flag. In block 1762, the value of the flag is determined. If the flag has a first value (e.g., 1), process 1700 proceeds to block 1764. In block 1764, the recovered patch information data is stored, and process 1700 loops back to block 1702 so that new patches can be decoded. If the flag has a second value (e.g., 0), process 1700 proceeds to block 1742. In block 1742, if the patch type is the last patch type, the recovery process corresponding to the encoded patch information data is terminated. That is, the recovery process is terminated.

[0185] In block 1732, it is determined whether the patch type for the patch is a PCM patch (or a raw patch). If the patch type for the patch is a PCM patch, process 1700 proceeds to block 1734. In block 1734, the reference index (refIdx) for the patch is determined based on the frame index ([frmIdx]) corresponding to the patch and the decremented patch index.

[0186] In block 1736, the patch is decoded using a two-dimensional (2D) component and a three-dimensional (3D) component corresponding to the patch. In one embodiment, the 2D and 3D components include the following: Decryption operation: separate_video_flag(ppdu_patch…) u0(ppdu_2d_shift_u) u1(ppdu_2d_shift_y) d_size_u0(ppdu_2d_delta_size_u) d_sze_v0(ppdu_2d_delta_size_v) PCM points (ppdu_pcm_points)

[0187] In block 1738, the patch is restored based on the decrypted patch. In one embodiment, the patch is restored as follows: Restoration: Patch2dShiftU[p]=pdu_2d_shift_u[p] Patch2dShiftV[p]=pdu_2d_shifty[p] Patch2dSizeU[p]=pdu_2d_deltasize_u[p]+ +Patch2dSizeU[refIdx] Patch2dSizeV[p]=pdu_2d_deIta_sizev[p]+ +Patch2dSizeV[refldx] PatchPcmPoints[p]=ppdu_pcm_points[p]

[0188] In block 1760, the flag is decoded to determine if there are any more patches to consider. In one embodiment, the flag is specified as more_patches_available_flag. In block 1762, the value of the flag is determined. If the flag has a first value (e.g., 1), process 1700 proceeds to block 1764. In block 1764, the recovered patch information data is stored, and process 1700 loops back to block 1702 so that new patches can be decoded. If the flag has a second value (e.g., 0), process 1700 proceeds to block 1742. In block 1742, if the patch type is the last patch type, the recovery process corresponding to the encoded patch information data is terminated. That is, the recovery process is terminated.

[0189] Figure 18 shows one embodiment of PCC method 1800 performed by a decoder (e.g., entropy decoding unit 70). Method 1800 may be used to decode an encoded bitstream to reconstruct a stereoscopic image. In block 1802, the decoder's receiver receives encoded patch information data. The encoded patch information may include information corresponding to multiple patches (e.g., patch 502). In block 1804, the decoder's processor retrieves the patch corresponding to the encoded patch information data. The patch has a patch type (patch_mode). The patch type may be, for example, skip, intra, inter, PCM, or last.

[0190] In block 1806, the decoder's processor determines whether the patch type for the patch is the last patch type. In block 1808, if the decoder's processor determines that the patch type is the last patch type, it terminates the decompression process corresponding to the encoded patch information data.

[0191] Figure 19 shows one embodiment of PCC method 1900 performed by a decoder (e.g., entropy decoding unit 70). Method 1900 may be used to decode an encoded bitstream to reconstruct a stereoscopic image. In block 1902, the decoder's receiver receives encoded patch information data. The encoded patch information may include information corresponding to multiple patches (e.g., patch 502). In block 1904, the decoder's processor retrieves the patch corresponding to the encoded patch information data. The patch has a patch type (patch_mode). The patch type may be, for example, skip, intra, inter, PCM, or last.

[0192] In block 1906, the decoder's processor determines whether the patch type for the patch is a skip patch type. In block 1908, if the patch type is a skip patch type, the decoder's processor decodes the reference patch index (spdu_patch_index) corresponding to the patch. In block 1910, the decoder's processor determines the reference index (refIdx) for the patch based on the reference frame index ([refFrmIdx]) corresponding to the patch and the reference patch index decoded when the patch type is a skip patch type. In block 1912, the decoder's processor reconstructs the 3D representation of the patch based on the reference index determined when the patch type is a skip patch type. Once reconstructed, the 3D image may be displayed for the user on the display of an electronic device (e.g., a smartphone, tablet, laptop computer, etc.).

[0193] Figure 20 shows one embodiment of PCC method 2000 performed by a decoder (e.g., entropy decoding unit 70). Method 2000 may be used to decode an encoded bitstream to reconstruct a stereoscopic image. In block 2002, the decoder's receiver receives encoded patch information data. The encoded patch information may include information corresponding to multiple patches (e.g., patch 502). In block 2004, the decoder's processor retrieves the patch corresponding to the encoded patch information data. The patch has a patch type (patch_mode). The patch type may be, for example, skip, intra, inter, PCM, or last.

[0194] In block 2006, the decoder's processor determines whether the patch type for the patch is a skip patch type. In block 2008, if the patch type is a skip patch type, the decoder's processor decodes the reference patch index (spdu_patch_index) corresponding to the patch. In block 2010, the decoder's processor determines the reference index (refIdx) for the patch based on the reference frame index ([refFrmIdx]) corresponding to the patch and the reference patch index decoded when the patch type is a skip patch type. In block 2012, the decoder's processor reconstructs the 3D representation of the patch based on the reference index determined when the patch type is a skip patch type.

[0195] In block 2014, the decoder's processor determines whether the "More Patches Available" flag has a first value or a second value. In block 2016, if the "More Patches Available" flag has a first value, the restored 3D representation is stored in the decoder's memory. Once all patches have been restored, the 3D image may be displayed for the user on the display of an electronic device (e.g., a smartphone, tablet, laptop computer).

[0196] In block 2018, the decoder's processor terminates the process of restoring the encoded patch information data when the More Patches Available flag has a second value.

[0197] Figure 21 shows one embodiment of PCC method 2100 performed by an encoder (e.g., an entropy coding unit 56). Method 2100 may be performed to encode a stereoscopic image into a bitstream for transmission to a decoder. In block 2102, the encoder's receiver obtains a patch frame data unit (pfdu) that identifies the patch type (pdfu_patch_mode) for each of a plurality of patches. In block 2104, the encoder's processor determines whether the patch type for a patch from the plurality of patches is the last patch type.

[0198] In block 2106, the encoder processor encodes patch information data for the patch when the patch type is not the last patch type, and the patch information data includes the patch type for the patch. In block 2108, the encoder processor encodes patch information data for the patch when the patch type is set to the last patch type. The patch information data includes the last patch type for the patch.

[0199] In one embodiment, patch information data and the final patch type are encoded in the bitstream sent to the decoder.

[0200] Figure 22 shows one embodiment of PCC method 2200 performed by an encoder (e.g., an entropy coding unit 56). Method 2200 may be performed to encode a stereoscopic image into a bitstream for transmission to a decoder. In block 2202, the encoder's receiver obtains a patch frame data unit (pfdu) for each of a plurality of patches. In block 2204, the encoder's processor adds a last patch flag to each of the plurality of patches.

[0201] In block 2206, the encoder processor determines, based on the value of the last patch flag, whether the patch type for a patch from multiple patches is the last patch type. In block 2208, if the patch type is not the last patch type, the encoder processor encodes patch information data for the patch, which includes the patch type and the last patch flag.

[0202] In one embodiment, patch information data, including the patch type and the last patch type, is encoded in a bitstream that is sent to the decoder.

[0203] Figure 23 is a schematic diagram of a coding device 2300 (e.g., an encoder 22, a decoder 28, etc.) according to one embodiment of the present disclosure. The coding device 2300 is suitable for carrying out the methods and processes disclosed herein. The coding device 2300 comprises an inlet port 2310 and a receiver unit (Rx) 2320 for receiving data, a processor, logic unit, or central processing unit (CPU) 2330 for processing the data, a transmitter unit (Tx) 2340 and an exit port 2350 for transmitting data, and a memory 2360 for storing data. The coding device 2300 may also comprise optical-electrical (OE) components and electrical-optical (EO) components coupled to the inlet port 2310, receiver unit 2320, transmitter unit 2340, and exit port 2350 for input and output of optical or electrical signals.

[0204] The processor 2330 is implemented by hardware and software. The processor 2330 may be implemented as one or more CPU chips, cores (for example, as a multi-core processor), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 2330 communicates with an inlet port 2310, a receiver unit 2320, a transmitter unit 2340, an exit port 2350, and memory 2360. The processor 2330 includes a coding module 2370. The coding module 2370 implements the embodiments disclosed above. In one embodiment, the coding module 2370 is a restoration module configured to project a restored stereoscopic image. Thus, including the coding module 2370 results in a substantial improvement to the functionality of the coding device 2300 and a transition of the coding device 2300 to different states. Alternatively, the coding module 2370 is implemented as instructions stored in memory 2360 and executed by the processor 2330.

[0205] The coding device 2300 may also include input and / or output (I / O) devices 2380 for exchanging data with the user. The I / O devices 2380 may include output devices such as a display for displaying video data and a speaker for outputting audio data. The I / O devices 2380 may also include input devices such as a keyboard, mouse, or trackball, and / or corresponding interfaces for interacting with such output devices.

[0206] Memory 2360 comprises one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device to store such programs when they are selected for execution, and to store instructions and data read during program execution. Memory 2360 may be volatile or non-volatile, and may be read-only memory (ROM), random access memory (RAM), tri-level content addressable memory (TCAM), and static random access memory (SRAM).

[0207] Figure 24 is a schematic diagram of one embodiment of the coding means 2400. In this embodiment, the coding means 2400 is implemented in a coding device 2402 (for example, an encoder 22 or a decoder 28). The coding device 2402 includes a receiving means 2401. The receiving means 2401 is configured to receive a picture to encode or a bitstream to decode. The coding device 2402 includes a transmitting means 2407 coupled to the receiving means 2401. The transmitting means 2407 is configured to transmit a bitstream to a decoder or transmit a decoded image to a display means (for example, one of the I / O devices 2380).

[0208] The coding device 2402 includes a memory means 2403. The memory means 2403 is coupled to at least one of the receiving means 2401 or the transmitting means 2407. The memory means 2403 is configured to store instructions. The coding device 2402 also includes a processing means 2405. The processing means 2405 is coupled to the memory means 2403. The processing means 2405 is configured to execute the instructions stored in the memory means 2403 for performing the methods disclosed herein.

[0209] In one embodiment, a syntax suitable for implementing the concepts disclosed herein is provided.

[0210] Patch frame data unit (pfdu). In one embodiment, a special end patch data unit type is introduced to indicate the end of the patch frame data unit.

[0211] [Table 1]

[0212] In one embodiment, a special 1-bit flag is added to each patch data unit to indicate the end of the patch frame data unit.

[0213] [[ID=2,5]][Table 2]

[0214] In one embodiment, the patch information data is as follows.

[0215] [Table 3] <\\

[0216] In one embodiment, the original delta_patch_data_unit syntax element is as follows.

[0217] [Table 4]

[0218] In one embodiment, the added skip_patch_data_unit syntax element is as follows:

[0219] [Table 5]

[0220] The introduction of the skip patch data unit type reduces the amount of overhead in the coded bitstream representation.

[0221] The semantics of the newly added elements are as follows:

[0222] spdu_patch_index[frmIdx][p] plus p specifies the index PredIdx of the patch in the patch frame that has the index RefIdx corresponding to the first patch frame in the reference patch frame list.

[0223] The decoding process for patch data units coded in skip prediction mode is as follows: When pfdu_patch_mode[frmIdx][p] is equal to P_SKIP, a process for decoding skip-coded patch data units is used, with frmIdx and p used as inputs to that process, and the output of that process (Patch2dShiftU, Patch2dShiftV, Patch2dSizeU, Patch2dSizeV, Patch3dShiftT, Patch3dShiftBT, Patch3dShiftN, PatchNormalAxis, Orientation, PatchLod) used as the output.

[0224] It should be recognized from the above that the orientation of the patch may be different from the default projection process. The orientation of the patch may be signaled in a simplified manner using a 1-bit flag. A mechanism for switching between the default orientation and the preferred orientation is introduced.

[0225] Although several embodiments are provided in the present disclosure, it will be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The examples of the present disclosure should be regarded as illustrative rather than limiting, and the intention should not be limited to the details given herein. For example, various elements or components may be combined or integrated in another system, or certain functions may be omitted or not implemented.

[0226] In addition, the techniques, systems, subsystems, and methods described and illustrated individually or separately in various embodiments may be combined or integrated with other systems, components, techniques, or methods without departing from the scope of the present disclosure. Other examples of changes, substitutions, and modifications can be elucidated by those skilled in the art and may be made without departing from the spirit and scope disclosed herein.

Description of Reference Numerals

[0227] 10 Coding system 12 Source device, video device 14 Destination device, video device 16 Computer-readable medium 18 Video source 20 Projection device 22 Video encoder 24 Output interface 26 Input interface 28 Video decoder 30 Restoration device 32 Display device 40 Mode selection unit 42 Motion Estimation Unit 44 Motion compensation unit 46 Intra Prediction Units 48 divided units 50 Adder 52 Conversion Processing Unit 54 Quantization Units 56 Entropy Coding Unit 58 Inverse Quantization Unit 60 Reverse Conversion Unit 62 Adder 64 Reference Frame Memory 70 Entropy Decoding Unit 72 Motion Compensation Unit 74 Intra Prediction Units 76 Inverse Quantization Unit 78 Reverse Conversion Unit 80 Adder 82 Reference Frame Memory 400-point cloud sequence 402 point cloud frames 404-point cloud frame 406-point cloud frame 408 point cloud 410-point cloud content 412 3D space 500 Boundary Box 502 patch 504 plane 600 Boundary Box 602 patch 610 Occupied Maps 612 Geometric Map 614 Attribute Map 700 V-PCC bitstream 702 V-PCC Unit 704 V-PCC Unit Header 706 V-PCC Unit Payload 708 Sequence Parameter Set 710 Patch Sequence Data 712 Exclusive video data 714 Geometric video data 716 Attribute Video Data 720 Sequence Parameter Set 722 Frame Parameter Set 724 Geometric Parameter Set 726 Attribute Parameter Set 728 Geometric Patch Parameter Set 730 Attribute Patch Parameter Set 732 Patch Frame Data Unit 734 Patch Data Unit 800-point cloud sequence 802 point cloud frames 810-point cloud content 812 Occupation Map 814 Geometric Map 816 Attribute Map 850 Auxiliary Information Patch Frame Data Unit 900 Patch Data Units 902 3D space 1000 Occupation Map 1002 Geometric Map 1004 Attribute Map 1006 Patch 1152 Patch Data Unit 1154 2D bounding box 1200 Schematic Diagram 1202 Point Cloud 3D Bounding Box 1204 Patch 3D Bounding Box 1206 Projection plane 1208 Patch 2D Projection Bounding Box 1252 Dataset Diagram 1254 Patch Data Unit 1300 Patch Frame Data Unit Encoding Process 1400 Patch Frame Data Unit Encoding Process 1500 Patch Information Data Frame 1502 Reference Patch Frame Data Unit 1504 Current patch frame data unit 1506 Patch Information Data Unit 1508 Intra-patch type data unit (PDU) 1510 PCM Patch Type Data Unit (PPDU) 1512 Interpatch Type Data Unit (DPDU) 1514 Skip Patch Type Data Unit (SPDU) 1600 Patch Frame Data Unit Decoding Process 1700 Patch Frame Data Unit Decoding Process PCC method implemented by 1800 decoder PCC method implemented by 1900 decoder 2000 Method of PCC implemented by decoder Method of PCC performed by 2100 encoders PCC method performed by 2200 encoders 2300 coding devices 2310 Entrance Port 2320 Receiver Unit (Rx) 2330 Processor 2340 Transmitter Unit (Tx) 2350 Exit Port 2360 memory 2370 Coding Modules 2380 Input and / or Output (I / O) Devices 2400 Means for coding 2401 Receiving means 2402 Video Coding Device 2403 Memory means 2405 Processing means 2407 Transmission method

Claims

1. A method for point cloud coding (PCC), A step of obtaining a patch flag corresponding to a patch, wherein the patch flag indicates whether the patch type (patch_mode) for the patch is a skip type, an intra type, an inter type, or the last patch type. When the patch type for the patch is the last patch type, the step of terminating the restoration process corresponding to the patch A method that includes [a certain feature].

2. A method for point cloud coding (PCC), The steps include determining that the patch type (patch_mode) for the patch is an interpatch type, The steps include: decrypting the reference patch index (dpdu_patch_index) corresponding to the aforementioned patch; The steps include determining a reference index (refIdx) for the patch based on the reference frame index ([refFrmIdx]) corresponding to the patch and the decoded reference patch index, A step of decoding the patch using a two-dimensional (2D) component and a three-dimensional (3D) component corresponding to the patch, wherein the two-dimensional (2D) component and the three-dimensional (3D) component are obtained based on the reference index (refIdx) for the patch. A step of restoring the 3D representation based on the decoded patch and A method that includes [a certain feature].

3. The steps include determining that the patch type for the patch is an intra-patch type, The steps include determining the reference index (refIdx) for the patch based on the frame index ([frmIdx]) corresponding to the patch and the decremented patch index, The steps include decoding the patch using a two-dimensional (2D) component corresponding to the patch and a three-dimensional (3D) component corresponding to the patch, A step of restoring the 3D representation based on the decoded patch. The method according to claim 2, further comprising:

4. The steps include determining that the patch type for the patch is a skip patch type, When the patch type is the skip patch type, the steps include: decrypting the reference patch index (spdu_patch_index) corresponding to the patch; The steps include determining the reference index (refIdx) for the patch based on the reference frame index ([refFrmIdx]) corresponding to the patch and the reference patch index decoded when the patch type is the skip patch type, A step of restoring the three-dimensional representation of the patch based on the reference index determined when the patch type is the skip patch type. The method according to claim 2 or 3, further comprising:

5. The steps include determining that the patch type for the patch is a pulse code modulation (PCM) patch type, The steps include determining the reference index (refIdx) for the patch based on the frame index ([frmIdx]) corresponding to the patch and the decremented patch index, The steps of decoding the patch using a two-dimensional (2D) component for an independent point corresponding to the patch and a three-dimensional (3D) component for the independent point corresponding to the patch, A step of restoring the 3D representation based on the decoded patch. The method according to any one of claims 2 to 4, further comprising:

6. The method according to any one of claims 2 to 5, further comprising the step of receiving an input corresponding to the patch, wherein the input comprises one or more of the patch_mode, patch index, reference index, frame index, and reference frame index.

7. The method according to any one of claims 2 to 6, further comprising the step of displaying an image generated based on the restored three-dimensional representation on a display of an electronic device.

8. The steps include determining whether the flag for more available patches has a first value or a second value, When more than the above-mentioned patch availability flags have the first value, the steps include storing the restored 3D representation, When more patch availability flags than those mentioned above have the second value, the patch restoration process is terminated. The method according to any one of claims 2 to 6, further comprising:

9. The steps include determining that the patch type for the patch is an intra-patch type, The steps include determining the reference index (refIdx) for the patch based on the frame index ([frmIdx]) corresponding to the patch and the decremented patch index, The steps include decoding the patch using a two-dimensional (2D) component corresponding to the patch and a three-dimensional (3D) component corresponding to the patch, A step of restoring the 3D representation based on the decoded patch. The method according to claim 8, further comprising:

10. A method for point cloud coding (PCC), A step of encoding a patch flag corresponding to a patch into a bitstream, wherein the patch flag indicates whether the patch type (patch_mode) for the patch is a skip type, an intra type, an inter type, or the last patch type. The steps of encoding the patch into the bitstream based on the patch type for the patch, A method that includes [a certain feature].

11. The method according to claim 10, wherein the bitstream includes a frame index (frmIdx), a two-dimensional (2D) component corresponding to the patch, and a three-dimensional (3D) component corresponding to the patch.

12. The method according to claim 10 or 11, further comprising the steps of determining whether the patch type for the patch is the last patch type, and if it is determined that the patch has the last patch type, not encoding patch information data for subsequent patches following the patch.

13. A decoding device, A receiver configured to receive a bitstream containing encoded point cloud data, A memory coupled to the receiver, wherein the memory stores instructions, A processor coupled to the memory, wherein the processor is configured to execute the instruction causing the decoding device to perform the method according to claim 1. A decoding device equipped with the following features.

14. A decoding device, A receiver configured to receive a bitstream containing encoded point cloud data, A memory coupled to the receiver, wherein the memory stores instructions, A processor coupled to the memory, wherein the processor is configured to execute the instruction causing the decoding device to perform the method according to any one of claims 2 to 9. A decoding device equipped with the following features.

15. An encoding device, A receiver configured to receive three-dimensional (3D) images, A memory coupled to the receiver, wherein the memory includes instructions, A processor coupled to the memory, wherein the processor is configured to implement the instruction causing the encoding device to perform the method according to any one of claims 10 to 12. An encoding device equipped with the following features.

Images