Information processing apparatus and method

Abstract

The present disclosure relates to an information processing device and method capable of suppressing an increase in transmission capacity. An atlas image of alpha components is generated by atlas transformation of an alpha image of 3D data, and the atlas image of alpha components is encoded. An atlas image of an RGB component is generated by atlas transformation of an RGB image of 3D data, and the atlas image of the RGB component is encoded. Further, the encoded data is decoded to generate an atlas image of the alpha component, and an inverse atlas transform is performed on the atlas image of the alpha component to generate an alpha image of the 3D data. The encoded data is decoded to generate an atlas image of the RGB components, and an inverse atlas transform is performed on the atlas image of the RGB components to generate an RGB image of the 3D data. The present disclosure can be applied to, for example, an information processing apparatus, an electronic device, an information processing method, or a program.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an information processing apparatus and method, and more particularly, to an information processing apparatus and method capable of suppressing an increase in transmission capacity. BACKGROUND

[0002] From the past, as a method of expressing a three-dimensional shape of an object, there is NeRF (Neural Radiance Field representing a scene as a neural radiance field for view synthesis) that generates a radiance field corresponding to a space including an object and approximates the radiance field by a neural network (for example, see Non-Patent Literatures 1 to 3). In NeRF, spatial information of a 3D object is expressed by an MLP (Multi-Layer Perceptron).

[0003] There is also a method called MPI (Multi-Plane Image) in which expression is performed by dividing a three-dimensional space into a plurality of layers (2D images). In order to speed up the rendering process of NeRF, a method called NeX that transforms an MLP into this MPI to perform processing is designed (see, for example, Non-Patent Literature 4). MPI is a group of images in which a plurality of 2D images are arranged by layers, and it is possible to process 3D data as 2D images.

[0004] LIST OF CITATIONS

[0005] NON-PATENT LITERATURE

[0006] Non-Patent Literature 1: Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng, “NeRF: Representing scenes as neural radiance fields for view synthesis”, ECCV 2020, 2020 / 3 / 19

[0007] Non-Patent Literature 2: Thomas Muller, Alex Evans, Christoph Schied, Alexander Keller, “Instant neural graphics primitives with a multiresolution hash encoding”, arXiv preprint arXiv:2201.05989, 2022 / 1 / 16

[0008] Non-Patent Literature 3: Ricardo Martin-Brualla, Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, Daniel Duckworth, “NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections”, https: / / arxiv.org / abs / 2008.02268, 2020 / 8 / 5

[0009] Non-Patent Literature 4: Suttisak Wizadwongsa, Pakkapon Phongthawee, Jiraphon Yenphraphai, “NeX: Real-time View Synthesis with Neural Basis Expansion”, arXiv: 2103.05606v2 [cs.CV] 12 Apr 2021 SUMMARY

[0010] TECHNICAL PROBLEM

[0011] However, the 3D data using MPI as in NeX has a large amount of data, and thus there is a concern that its transmission capacity will increase when transmitting the 3D data.

[0012] The present disclosure is made in view of the above-described situation, and aims to enable suppression of an increase in transmission capacity.

[0013] SOLUTION TO THE PROBLEM

[0014] An information processing apparatus according to one aspect of the present technology is an information processing apparatus including: a first atlas transformation unit that performs atlas transformation on an alpha image of 3D data and generates an atlas image of an alpha component; a first encoding unit that encodes the atlas image of the alpha component; a second atlas transformation unit that performs atlas transformation on an RGB image of the 3D data and generates an atlas image of an RGB component; and a second encoding unit that encodes the atlas image of the RGB component.

[0015] An information processing method according to one aspect of the present technology is an information processing method including: performing atlas transformation on an alpha image of 3D data and generating an atlas image of an alpha component; encoding the atlas image of the alpha component; performing atlas transformation on an RGB image of the 3D data and generating an atlas image of an RGB component; and encoding the atlas image of the RGB component.

[0016] An information processing apparatus according to another aspect of the present technology is an information processing apparatus including: a first decoding unit that decodes encoded data and generates an atlas image of an alpha component; a first atlas inverse transformation unit that performs atlas inverse transformation on the atlas image of the alpha component and generates an alpha image of 3D data; a second decoding unit that decodes the encoded data and generates an atlas image of an RGB component; and a second atlas inverse transformation unit that performs atlas inverse transformation on the atlas image of the RGB component and generates an RGB image of the 3D data.

[0017] An information processing method according to another aspect of the present technology is an information processing method including: decoding encoded data to generate an atlas image of an alpha component; performing atlas inverse transformation on the atlas image of the alpha component and generating an alpha image of 3D data; decoding the encoded data and generating an atlas image of an RGB component; and performing atlas inverse transformation on the atlas image of the RGB component and generating an RGB image of the 3D data.

[0018] In the information processing apparatus and method according to aspects of the present technology, atlas transformation is performed on an alpha image of 3D data to generate an atlas image of an alpha component, the atlas image of the alpha component is encoded, atlas transformation is performed on an RGB image of the 3D data to generate an atlas image of an RGB component, and the atlas image of the RGB component is encoded.

[0019] In the information processing apparatus and method according to another aspect of the present technology, encoded data is decoded to generate an atlas image of an alpha component, atlas inverse transformation is performed on the atlas image of the alpha component to generate an alpha image of 3D data, the encoded data is decoded to generate an atlas image of an RGB component, and atlas inverse transformation is performed on the atlas image of the RGB component to generate an RGB image of the 3D data. BRIEF DESCRIPTION OF DRAWINGS

[0020] [ Figure 1 ] for explaining a graph of 3D data using MPI.

[0021] [ Figure 2 ] for explaining a graph of 3D data using MPI.

[0022] [ Figure 3 ] A diagram for illustrating NeX.

[0023] [ Figure 4 ] A diagram for illustrating NeX.

[0024] [ Figure 5 ] A diagram for illustrating NeX.

[0025] [ Figure 6 ] A diagram for illustrating NeX.

[0026] [ Figure 7 ] A diagram showing an example of a 3D data transmission method.

[0027] [ Figure 8 ] A diagram for illustrating the use of atlas transformation.

[0028] [ Figure 9 ] A diagram showing an example of atlas transformation of NeX data.

[0029] [ Figure 10 ] A diagram showing an example of a state of generating a mask image.

[0030] [ Figure 11 ] A diagram showing an example of a state of atlas transformation using an RGB_base image of a mask image.

[0031] [ Figure 12 ] A diagram showing an example of a state of atlas transformation using an RGB_view image of a mask image.

[0032] [ Figure 13 ] A diagram showing a configuration example of a bitstream.

[0033] [ Figure 14 ] A diagram showing an example of atlas transformation of NeX data.

[0034] [ Figure 15 ] A diagram showing a configuration example of a bitstream.

[0035] [ Figure 16 ] A diagram for illustrating an example of a state of generating atlas transformation information.

[0036] [ Figure 17 ] A diagram showing a configuration example of a bitstream.

[0037] [ Figure 18 ] A system diagram showing a main configuration example of an information processing system.

[0038] [ Figure 19 ] A flowchart for illustrating an example of a flow of a transmission process.

[0039] [Figure 20 A flowchart for explaining an example of a flow of the reproduction process.

[0040] [ Figure 21 A block diagram showing a main configuration example of the encoding device.

[0041] [ Figure 22 A flowchart for explaining an example of a flow of the encoding process.

[0042] [ Figure 23 A block diagram showing a main configuration example of the decoding device.

[0043] [ Figure 24 A flowchart for explaining an example of a flow of the decoding process.

[0044] [ Figure 25 A block diagram showing a main configuration example of the encoding device.

[0045] [ Figure 26 A flowchart for explaining an example of a flow of the encoding process.

[0046] [ Figure 27 A block diagram showing a main configuration example of the decoding device.

[0047] [ Figure 28 A flowchart for explaining an example of a flow of the decoding process.

[0048] [ Figure 29 A block diagram showing a main configuration example of the encoding device.

[0049] [ Figure 30 A block diagram showing a main configuration example of the decoding device.

[0050] [ Figure 31 A flowchart for explaining an example of a flow of the decoding process.

[0051] [ Figure 32 A block diagram showing a main configuration example of the computer. DETAILED DESCRIPTION

[0052] Hereinafter, a manner for carrying out the present disclosure (hereinafter, will be referred to as an embodiment) will be described. Note that the description will be given in the following order.

[0053] 1. Literature and the like supporting technical contents and technical terms

[0054] 2. Transmission capacity of 3D data using MPI

[0055] 3. Encoding / decoding using atlas transform

[0056] 4. First embodiment (information processing system)

[0057] 5. Supplementary notes

[0058] <1. Literature and the like supporting technical contents and technical terms>

[0059] The scope of the present technology includes not only the contents described in the embodiments, but also the contents described in the following non-patent literature and the like known at the time of filing, the contents of other literature cited in the following non-patent literature, and the like.

[0060] Non-Patent Literature 1: (as described above)

[0061] Non-Patent Literature 2: (as described above)

[0062] Non-Patent Literature 3: (as described above)

[0063] Non-Patent Literature 4: (as described above)

[0064] In other words, the contents described in the above-described non-patent literature, the contents of other literature cited in the above-described non-patent literature, and the like also become a basis for determining the support conditions.

[0065] <2. Transmission capacity of 3D data using MPI>

[0066] <nerf>

[0067] As a method of expressing a three-dimensional shape as an object of expression from the past, there is NeRF (Neural Radiance Fields) that generates a radiance field corresponding to a space including an object and approximates the radiance field by a neural network.

[0068] A radiance field represents a scene in a space including an object by color of each direction at each position in the space and opacity at each position in the space. Here, the opacity is an index indicating the presence of some object, and can also be considered as a density. In other words, by obtaining a radiance field in which the density of coordinates where an object is present becomes high, it is possible to express the shape of a three-dimensional space. NeRF approximates such a radiance field by a neural network. In other words, NeRF is a technique that generates a radiance field corresponding to a 3D object, approximates the radiance field by a neural network, and performs rendering using the neural network.

[0069] In NeRF, an object in a three-dimensional space is captured from different positions, camera poses of captured images are calculated, and the captured images and the camera poses are used for learning of a neural network, thereby generating a neural network (MLP (Multi-Layer Perceptron)) that approximates a radiance field representing a three-dimensional shape of the object. During inference, pose information of a new desired viewpoint (will also be referred to as viewpoint information) is input to the neural network in order to obtain a rendered image of the new viewpoint.

[0070] <nex>

[0071] There is also a method called MPI (Multi-Plane Image) in which a three-dimensional space is represented by being divided into a plurality of layers (2D images). In order to speed up the rendering process of NeRF, Non-Patent Literature 4 proposes a method called NeX of transforming an MLP into this MPI to be processed. The MPI is a group of images in which a plurality of 2D images are arranged by layer, and 3D data can be processed like a 2D image.

[0072] In the MPI, a plurality of layers (2D planes) are formed in a three-dimensional space, for example, like layers 11 to 14 shown in Figure 1 , and an object is expressed in each of the layers. The MPI is composed of a plurality of RGBA images. This RGBA image is generated by learning a plurality of view images using a DNN (Deep Neural Network). In a rendered image at a viewpoint position, a synthesis result (superimposition result) of an object in the corresponding layer as viewed from that viewpoint is shown. For example, as shown in Figure 1 , it is assumed that an object 21 exists in the layer 11, an object 22 exists in the layer 12, an object 23 exists in the layer 13, and an object 24 exists in the layer 14. In the rendered image, these objects are superimposed in a positional relationship as viewed from the viewpoint position. Figure 2 The rendered image 41 in Figure 1 shows an example of a rendered image at the viewpoint 31 in Figure 1 , and the rendered image 42 shows a rendered image at the viewpoint 32 in . The way in which the objects 21 to 24 overlap changes according to the viewpoint position in this way, exhibiting a stereoscopic effect of a three-dimensional space (three-dimensional shape of an object).

[0073] In NeX, the MLP of NeRF is transformed into such an MPI (also to be called a NeX MPI) to be processed. In other words, a three-dimensional space is represented by a plurality of layers. Figure 3 An example of a layer configuration of NeX is shown in . In the case of the NeX MPI, a plurality of main layers 51 are formed as the illustrated main layers 51-1, 51-2, …, and in addition, a plurality of sub-layers 52 are formed as the illustrated sub-layers 52-1 to 52-11 and sub-layers 52-21 to … between the main layers 51. In other words, in a group of layers representing a three-dimensional space, some of the layers set at a predetermined number of layers are formed as the main layers 51, and the remaining layers between the main layers 51 are formed as the sub-layers 52.

[0074] The number of layers in the layer group, the number of main layers, and the number of sub-layers between the main layers are all arbitrary. For example, a three-dimensional space at a certain time (a scene) can be represented by 192 layers, in which one of the 12 layers (i.e., a total of 16 layers) is set as a main layer, and the remaining 176 layers are set as sub-layers. In this case, one of the 12 consecutive layers in the layer group is set as a main layer, and the remaining 11 layers are set as sub-layers. Alternatively, for example, the number of main layers can be set to 32. In this case, the number of layers in the layer group representing a scene becomes 384, although of course is not limited to these examples.

[0075] In the case of NeX, as shown in Figure 4 The data is composed of an alpha image 71, an RGB_base image 72, an RGB_view image 73, and a base image 74. The alpha image 71 is distribution information of an alpha component indicating the presence of an object in a single layer in the layer group representing a three-dimensional space. In other words, the alpha image 71 indicates the position at which the object is present in the layer. This alpha image 71 is generated for each sub-layer. That is, this alpha image 71 is generated for all layers (main layers 51 and sub-layers 52 shown in Figure 3 ).

[0076] The RGB_base image 72 and the RGB_view image 73 are each information indicating the distribution of an RGB component in a plurality of consecutive layers in the layer group, the RGB component indicating color information of an object. Note that the RGB_base image and the RGB_view image are also collectively referred to as an RGB image.

[0077] The RGB_base image 72 is color distribution information that does not depend on the viewpoint angle. The RGB_base image 72 is composed of a single image. In other words, the RGB_base image 72 is color distribution information in which the color component is strong when viewed from the front of the image.

[0078] The RGB_view image 73 is color distribution information that depends on the viewpoint angle. The RGB_view image 73 is composed of a plurality of images. The plurality of images are color distribution information in which the color component is strong when viewed from different viewpoints. For example, the RGB_view image 73 is composed of four images: an image k0 having a strong color component when viewed from above, an image having a strong color component when viewed from below, an image having a strong color component when viewed from the left, and an image having a strong color component when viewed from the right.

[0079] The RGB_base image 72 and the RGB_view image 73 are generated for each of the main layers (i.e., for each of the plurality of consecutive layers in the layer group). For example, in Figure 3 In this case, one RGB_base image 72 and one RGB_view image 73 are generated with respect to the main layer 51-1 and the sub-layers 52-1 to 52-11. That is, the RGB_base image 72 indicates color distribution information for 12 consecutive layers and generates one image for the 12 layers. Similarly, the RGB_view image 73 indicates color distribution information for 12 consecutive layers and generates four images for the 12 layers.

[0080] The base images 74 are constituted by the same number of images as the RGB_view images 73. For example, when four RGB_view images 73 are formed as described above, the number of base images 74 to be formed is also four. The base images 74 respectively correspond to different RGB_view images 73 and each indicates a mixing ratio for each viewpoint direction for the corresponding RGB_view image 73. That is, the pixel position of the base image 74 is determined in accordance with the viewpoint direction, and the RGB_view image 73 corresponding to the base image 74 is mixed in accordance with the mixing ratio expressed as a pixel value. For example, as shown in FIG. 8, when the viewpoint is front, the RGB_view images 73 (k1 to k4) are respectively multiplied by the mixing ratio expressed as the center pixel value of the base image 74 (H1 to H4) corresponding thereto, and the multiplication results are added to the RGB_base image 72 (k0). Further, as shown in FIG. 9, when the viewpoint is upper left, the RGB_view images 73 (k1 to k4) are respectively multiplied by the mixing ratio expressed as the upper left pixel value of the base image 74 (H1 to H4) corresponding thereto, and the multiplication results are added to the RGB_base image 72 (k0). Figure 5 Figure 6

[0081] By changing the RGB portion in accordance with the viewpoint direction in this way, it is possible to improve the expressiveness of the NeX MPI. Note that by generating an alpha image for each of the sub-layers and an RGB image for each of the main layers, it is possible to suppress an increase in the amount of data while improving the expressiveness, and achieve efficient learning.

[0082] <MPI transmission>

[0083] As an MPI transmission method, there is an MPEG Immersive Video (MIV) encoding standard that is being standardized by MPEG (Moving Picture Experts Group). In MPEG-I Part 12: Immersive Video (MIV), after performing preprocessing for viewpoint reduction and / or data reduction such as pixel pruning, standardization of metadata required for decoding and synthesis accompanying the preprocessing is performed in a framework for encoding and transmitting a multi-view video constituted by a large number of textures and depths. ​​

[0084] There is also MPEG-I Part 5, 2nd edition (V3C (coding based on visual volumetric video)), which describes details of MIV atlas processing and video processing. Immersive video has several degrees of freedom of view points, and there are several expressions to realize this.

[0085] As described above, MPI has a configuration of slicing three-dimensional space into a plurality of layers (images). Therefore, each layer is sparse data. Therefore, the amount of data can be reduced by reducing pixels (atlas) before transmission. Information related to atlas (cutout position, etc.) is specified in MIV / V3C.

[0086] In atlas transformation, a portion for which transparency (alpha component) is zero is deleted. The remaining portion (non-zero region) is cut into a rectangle and arranged in an atlas image. Data transformed into an atlas in this way is transmitted as atlas transformation information indicating the content of atlas transformation and an atlas image (i.e., an atlas image in which a non-zero region is arranged) indicating the transformation result of the atlas transformation.

[0087] In the case of MPI, basic information of atlas transformation information is stored in V3C_VPS. In addition, data of atlas transformation information is stored in V3C_AD. Further, atlas images (an atlas image of an alpha component and an atlas image of an RGB component) are stored in V3C_AVD.

[0088] V3C_VPS can transmit up to 63 pieces of atlas transformation information (basic information). This basic information includes accompanying information of an atlas image, such as identification information (ID), the size of an atlas image in the width direction (horizontal direction), and the size of an atlas image in the height direction (vertical direction).

[0089] Note that identification information (vuh_atlas_id) is used in v3c_unit_header() to perform association of an atlas image with an alpha image and / or an RGB image.

[0090] In addition, an atlas transmission method is standardized in MPEG-I Part 5. A non-zero region can be rotated to be arranged in an atlas image.

[0091] <Amount of data of NeX MPI>

[0092] The data amount of NeX MPI is large. As an example, assume that 3D moving image data of 60 FPS is used, the resolution of an alpha image and an RGB image is 4K, the main layer constituting a scene is 32 layers, one main layer is set every 12 layers, and the number of Basis is 4. Also assume that in this case the data of NeX MPI (4K32L) is transmitted and reproduced immediately (in real time) at the transmission destination. In this case, the required transmission capacity is 524 GB / s. Further, as another example, assume that 3D moving image data of 60 FPS is used, the resolution of an alpha image and an RGB image is HD, the main layer constituting a scene is 16 layers, one main layer is set every 12 layers, and the number of Basis is 4. Also assume that in this case the data of NeX MPI (HD16L) is transmitted and reproduced immediately (in real time) at the transmission destination. In this case, the required transmission capacity is 83 GB / s. Note that, in the present specification, the transmission capacity indicates the amount of data that can be transmitted per second through a transmission path (which can include a communication medium, a data bus, an information processing unit, and the like). It is difficult to secure such a transmission capacity, and thus there is a concern that the reproduction processing will fail. Therefore, there is a demand to reduce the transmission capacity of NeX MPI.

[0093] Note that, not only NeX, but also any 3D data using an MPI similar to NeX has a similar problem. That is, 3D data using an MPI similar to NeX has a large amount of data, and there is a demand to reduce the transmission capacity.

[0094] <3. Encoding / decoding using atlas transformation>

[0095] <Method 1>

[0096] In this regard, as shown on the top row of the table in Figure 7 , an alpha image and an RGB image smaller than the alpha image in 3D data are subjected to atlas transformation for encoding, and then transmitted (Method 1).

[0097] For example, the first information processing apparatus includes: a first atlas transformation unit that performs atlas transformation on an alpha image of 3D data to generate an atlas image of alpha components; a first encoding unit that encodes the atlas image of alpha components; a second atlas transformation unit that performs atlas transformation on an RGB image of 3D data to generate an atlas image of RGB components; and a second encoding unit that encodes the atlas image of RGB components.

[0098] Further, the first information processing method includes: performing atlas transformation on an alpha image of the 3D data to generate an atlas image of an alpha component; encoding the atlas image of the alpha component; performing atlas transformation on an RGB image of the 3D data to generate an atlas image of an RGB component; and encoding the atlas image of the RGB component.

[0099] Further, the first information processing apparatus includes: a first decoding unit that decodes the encoded data to generate an atlas image of an alpha component; a first atlas inverse transformation unit that performs atlas inverse transformation on the atlas image of the alpha component to generate an alpha image of the 3D data; a second decoding unit that decodes the encoded data to generate an atlas image of an RGB component; and a second atlas inverse transformation unit that performs atlas inverse transformation on the atlas image of the RGB component to generate an RGB image of the 3D data.

[0100] Further, the second information processing method includes: decoding the encoded data to generate an atlas image of an alpha component; performing atlas inverse transformation on the atlas image of the alpha component to generate an alpha image of the 3D data; decoding the encoded data to generate an atlas image of an RGB component; and performing atlas inverse transformation on the atlas image of the RGB component to generate an RGB image of the 3D data.

[0101] Note that the alpha image is distribution information of an alpha component indicating the presence of an object in a single layer in a layer group representing a three-dimensional space. The RGB image is information indicating the distribution of an RGB component in a plurality of consecutive layers in the layer group, the RGB component indicating color information of the object. For the three-dimensional space, the 3D data includes an alpha image for each of the single layers and an RGB image for each of the plurality of consecutive layers. Further, the atlas transformation is a process of arranging a non-zero region included in a single alpha image or a non-zero region included in an RGB image (which corresponds to a non-zero region included in one or more alpha images) in an atlas image, and the atlas inverse transformation is an inverse process of the atlas transformation.

[0102] For example, as shown in Figure 8 extracts a non-zero region from each layer of 3D data 111 (alpha image and / or RGB image) composed of a plurality of layers, and arranges it in an atlas image 112. In this specification, such a process will be referred to as atlas transformation. Further, an inverse process of such a process will be referred to as atlas inverse transformation. Like the above-described MPI, the alpha image and / or the RGB image of each layer of the NeX MPI is generally sparse. Therefore, a plurality of non-zero regions can be arranged in one atlas image.

[0103] The first atlas transformation unit of the first information processing apparatus performs such an atlas transformation on the alpha image to generate an atlas image of the alpha component (atlas image (alpha)) as shown in Figure 9 Then, the first encoding unit of the first information processing apparatus encodes the atlas image (alpha) using a 2D encoding method to generate encoded data. Similarly, the second atlas transformation unit of the first information processing apparatus performs such an atlas transformation on the RGB image to generate an atlas image of the RGB component (atlas image (RGB)) as shown in Figure 9 Then, the second encoding unit of the first information processing apparatus encodes the atlas image (RGB) using a 2D encoding method to generate encoded data. These encoded data pieces are transmitted to the second information processing apparatus.

[0104] Then, the first decoding unit of the second information processing apparatus decodes the encoded data of the atlas image (alpha) to generate (restore) the atlas image (alpha). Then, the first atlas inverse transformation unit of the second information processing apparatus performs an atlas inverse transformation on the atlas image (alpha) to generate (restore) the alpha image. Similarly, the second decoding unit of the second information processing apparatus decodes the encoded data of the atlas image (RGB) to generate (restore) the atlas image (RGB). Then, the second atlas inverse transformation unit of the second information processing apparatus performs an atlas inverse transformation on the atlas image (RGB) to generate (restore) the RGB image.

[0105] With such a configuration, the transmission capacity required for data transmission of NeX MPI (4K32L) can be reduced from 524 GB / s to 7 GB / s (to about 1 / 75). Further, the transmission capacity required for data transmission of NeX MPI (HD16L) can be reduced from 83 GB / s to 1.1 GB / s (to about 1 / 75). In other words, an increase in the transmission capacity required for transmission of 3D data using MPI such as NeX can be suppressed.

[0106] Note that although NeX MPI is taken as an example in the present specification, the present technology (Method 1) can be applied to any 3D data as long as it is 3D data using MPI. In other words, the application of the present technology is not limited to NeX MPI. Further, the encoding method to which the present technology (Method 1) is applied only needs to be an encoding method involving atlas transformation, and can be V3C or other encoding methods.

[0107] <Method 1-1>

[0108] When this Method 1 is applied, as from Figure 7 As shown on the second row from the top of the table in FIG. 10, the intermediate data (masks) in which all sub-layers and the main layer of the alpha image are synthesized can be generated, and the masks can be used to perform the atlas transformation of one RGB image corresponding to the "continuous multiple layers" (Method 1-1).

[0109] For example, the first information processing apparatus described above can further include a mask image generation unit that generates a mask image of a non-zero region of an alpha image in which a plurality of continuous layers are synthesized, and the second atlas transformation unit can perform the atlas transformation of the RGB image using the mask image.

[0110] Since the alpha component is information indicating the presence of an object, only the non-zero region of the alpha image needs to be extracted and atlas-transformed. The portion of the RGB component corresponding to the object corresponds to the non-zero region of the alpha component. In other words, the second atlas transformation unit basically only needs to extract the same region as the alpha image from the RGB image and perform the atlas transformation. However, as shown in FIG. 10, an alpha image is formed for each of the sub-layers (for each single layer), and an RGB image is formed for each of the main layers (each continuous multiple layer). In other words, one RGB image includes color distribution information for a plurality (e.g., 12) of continuous layers. In other words, one RGB image corresponds to a plurality (e.g., 12) of continuous alpha images. Figure 9

[0111] Therefore, there is a concern that if the same region as the non-zero region of one alpha image is simply extracted from the RGB image and subjected to the atlas transformation, the color distribution information of the other layers in the continuous multiple layers (e.g., the information of the remaining 11 layers in the case where an RGB image is formed every 12 layers) will be lost. In other words, there is a concern that in the case of 3D data using MPI similar to NeX MPI and having fewer RGB images than alpha images, the information of the RGB image will be lost due to the atlas transformation, and the quality of the 3D data (the image quality of the rendered image) will thus be reduced.

[0112] In this regard, the mask image generation unit of the first information processing apparatus synthesizes all layers of the "continuous multiple layers" corresponding to one RGB image to generate intermediate data (a mask image), as shown in FIG. 10. Figure 10 ​The mask images are generated as shown in FIG. 6. For example, when an RGB image is formed every 12 layers, the alpha images are synthesized (superimposed) every 12 layers, and a mask image in which the alpha images for the 12 layers are superimposed is generated therefrom. Thus, the same number of mask images as the number of layers of the RGB image is generated. For example, when a three-dimensional space is expressed by 192 layers, 16 mask images are generated.

[0113] Then, as shown in Figure 11 and Figure 12 , the atlas transformation is performed on the RGB image using the mask image to generate an atlas image (RGB). Each of the mask images is obtained by superimposing 12 alpha images, and thus includes non-zero regions of 12 layers (non-zero regions of 12 layers are superimposed). Thus, by applying the mask image, the color distribution information of the objects of 12 layers can be extracted from the RGB image, and the loss of information as described above is suppressed. In other words, the reduction in the quality (image quality of the rendered image) of the 3D data due to the atlas transformation can be suppressed.

[0114] <Method 1-1-1>

[0115] When this method 1-1 is applied, the atlas transformation information can be made common to the RGB_base image and the RGB_view image, as shown on the third row from the top of the table in Figure 7 (Method 1-1-1).

[0116] For example, in the first information processing apparatus described above, the RGB image can include an RGB_base image indicating the distribution of an RGB_base component among the RGB components, and an RGB_view image indicating the distribution of an RGB_view component among the RGB components. Further, the second atlas transformation unit can perform atlas transformation on the RGB_base image and the RGB_view image to generate atlas images thereof, and generate atlas transformation information common to the RGB_base component and the RGB_view component. Note that the atlas transformation information can be information indicating the content of the atlas transformation.

[0117] For example, in the second information processing apparatus described above, the RGB image can include: an RGB_base image indicating a distribution of an RGB_base component among the RGB components; and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the second decoding unit can decode the encoded data to generate an atlas image of the RGB_base component and an atlas image of the RGB_view component. Further, the second atlas inverse transformation unit can perform atlas inverse transformation on the atlas image of the RGB_base component and the atlas image of the RGB_view component using atlas transformation information common to the RGB_base component and the RGB_view component. Further, the atlas transformation information can be information indicating a content of the atlas transformation.

[0118] As Figure 8 indicated in the first information processing apparatus, the second atlas transformation unit generates the atlas transformation information 113 indicating the content of the atlas transformation together with the atlas image 112. For example, the "content of the atlas transformation" can include information on what regions (e.g., regions 121 to 124 in Figure 8 ) extracted from what layers are to be arranged in what positions in the atlas image and how to arrange, and the like. Such atlas transformation information 113 is used for the atlas inverse transformation by the second atlas inverse transformation unit of the second information processing apparatus. The atlas inverse transformation is inverse processing of the atlas transformation. In other words, the atlas inverse transformation is processing of extracting each of the regions (e.g., regions 121 to 124 in Figure 8 ) arranged in the atlas image by the atlas transformation, and generating (restoring) the original alpha image and / or the RGB image. Therefore, by performing such atlas inverse transformation based on the atlas transformation information indicating the content of the atlas transformation, the second atlas inverse transformation unit can correctly generate (restore) the alpha image and / or the RGB image.

[0119] When the RGB_base image and the RGB_view image exist as the RGB image, as in the example shown in Figure 9 , the atlas transformation unit performs the atlas transformation on each of the RGB_base image and the RGB_view image. In these atlas transformations, the atlas transformation unit can extract the same regions from the RGB_base image and the RGB_view image, and arrange the regions at the same positions in the atlas image. In this case, the atlas transformation unit can generate the atlas transformation information (atlas transformation information (RGB)) common to the RGB_base component and the RGB_view component, as in Figure 11 and Figure 12 As shown in the diagram, this configuration reduces redundancy between atlas transformation information when generating both RGB_base and RGB_view component atlas transformation information. Therefore, it suppresses the increase in data volume and the increase in transmission capacity required for data transfer.

[0120] In this case, such as Figure 13 As shown, for example, atlas transformation information (RGB) can be stored in V3C_AD(1), which is shared by the RGB_base component and the RGB_view component in the V3C bitstream. In addition, the atlas image (RGB_base) and the atlas image (RGB_view) can be stored in V3C_AVD associated with the V3C_AD(1).

[0121] <Method 1-2>

[0122] When applying method 1 described above, such as from Figure 7 As shown in the fourth row from the top of the table, the alpha image can be transformed to the same size as the RGB image to generate a mask (Method 1-2).

[0123] The dimensions (resolution) of alpha images and RGB images can differ. Figure 14 In the example shown, the size (resolution) of the alpha image and the RGB_view image is a so-called 1K size, and the size (resolution) of the RGB_base image is a so-called 2K size. There is a concern that when the size (resolution) of the alpha image and the RGB image differs in this way, the size (resolution) of the mask image generated from the alpha image as described above will be different from the size (resolution) of the RGB image, and therefore will not be applicable.

[0124] In this respect, atlas transformation can be performed correctly by transforming the alpha image to the same size as the RGB image. Therefore, the increase in transmission capacity required for data transmission can be suppressed.

[0125] <Method 1-2-1>

[0126] When applying method 1-2, such as from Figure 7 As shown in the fifth row from the top of the table, atlas transformations can be performed independently on RGB_base and RGB_view images of different sizes (Method 1-2-1).

[0127] For example, in the first information processing apparatus described above, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the second atlas transformation unit can perform the atlas transformation independently of each other on the RGB_base image and the RGB_view image having different sizes.

[0128] For example, in the second information processing apparatus described above, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the RGB_base image and the RGB_view image can have different sizes. Further, the second decoding unit can decode each of the encoded data of the atlas image of the RGB_base component and the encoded data of the atlas image of the RGB_view component. The second atlas inverse transformation unit can perform the atlas inverse transformation independently of each other on the atlas image of the RGB_base component and the atlas image of the RGB_view component.

[0129] Further, as in the example shown in Figure 14 , the sizes (resolutions) between the RGB_base image and the RGB_view image can be different. In such a case, it can be difficult to generate the common atlas transformation information in the atlas transformation. It can also be difficult to correctly perform the atlas inverse transformation using the common atlas transformation information.

[0130] In this regard, as described above, the atlas transformation can be performed independently of each other on the RGB_base image and the RGB_view image. In addition, the atlas inverse transformation can be performed independently of each other on the atlas image of the RGB_base component and the atlas image of the RGB_view component.

[0131] By such a configuration, the atlas transformation and the atlas inverse transformation of the RGB_base component and the RGB_view component can be correctly performed.

[0132] <Method 1-2-1-1>

[0133] When this method 1-2-1 is applied, the atlas transformation information for the RGB_base image and the atlas transformation information for the RGB_view image can be generated, as shown in the sixth row from the top of the table in Figure 7 (Method 1-2-1-1).

[0134] For example, in the first information processing apparatus described above, the second atlas transformation unit can perform atlas transformation on the RGB_base image to generate atlas transformation information of the RGB_base component. In addition, the second atlas transformation unit can perform atlas transformation on the RGB_view image to generate atlas transformation information of the RGB_view component. Further, the atlas transformation information can be information indicating the content of the atlas transformation.

[0135] For example, in the second information processing apparatus described above, the second atlas inverse transformation unit can perform atlas inverse transformation on the atlas image of the RGB_base component using the atlas transformation information of the RGB_base component. In addition, the second atlas inverse transformation unit can perform atlas inverse transformation on the atlas image of the RGB_view component using the atlas transformation information of the RGB_view component. Further, the atlas transformation information can be information indicating the content of the atlas transformation.

[0136] For example, as shown in Figure 14 , the atlas transformation information (RGB_base) and the atlas transformation information (RGB_view) can be generated by atlas transformation. With such a configuration, the atlas transformation of the RGB_base component and the atlas transformation of the RGB_view component can be performed independently of each other. In addition, the atlas inverse transformation can be performed using the atlas transformation information (RGB_base) and the atlas transformation information (RGB_view). The atlas inverse transformation of the RGB_base component and the atlas inverse transformation of the RGB_view component can be performed independently of each other.

[0137] In this case, for example, as shown in Figure 15 , in the V3C bitstream, the atlas transformation information (RGB_base) can be stored in the V3C_AD (1) for the RGB_base component, and the atlas transformation information (RGB_view) can be stored in the V3C_AD (2) for the RGB_view component (may be stored in different V3C_ADs). In addition, the atlas image (RGB_base) can be stored in the V3C_AVD associated with the V3C_AD (1). In addition, the atlas image (RGB_view) can be stored in the V3C_AVD associated with the V3C_AD (2).

[0138] <Method 1-2-1-2>

[0139] When the above-described method 1-2-1 is applied, the intermediate data (mask) can be generated for each of the RGB_base image and the RGB_view image, as shown in the seventh row from the top of the table in Figure 7 (Method 1-2-1-2).

[0140] For example, the first information processing apparatus described above can further include a mask image generation unit that generates a mask image in which a non-zero region of an alpha image in which a plurality of layers that are continuous are synthesized is generated with respect to each of the RGB_base image and the RGB_view image. Further, in the first information processing apparatus, the second atlas transformation unit can perform atlas transformation on the RGB_base image using the mask image for the RGB_base image. Further, the second atlas transformation unit can perform atlas transformation on the RGB_view image using the mask image for the RGB_view image.

[0141] When the RGB_base image and the RGB_view image have different sizes as in the example shown in FIG. 8, at least one of the sizes is different from the size of the alpha image. Therefore, there is a concern that the mask image generated from the alpha image will not be able to be applied in its original size. In this regard, intermediate data (masks) can be generated for each of the RGB_base image and the RGB_view image. By generating the mask images individually in this way, mask images that match the sizes of both the RGB_base image and the RGB_view image can be generated. Therefore, atlas transformation can be performed correctly. Figure 14

[0142] <Method 1-2-1-3>

[0143] Further, when the method 1-2-1 described above is applied, the alpha and the RGB_view can be down-scaled, as shown in the eighth row from the top of the table in FIG. 9 (Method 1-2-1-3). Figure 7

[0144] For example, the first information processing apparatus described above can further include a down-scaling processing unit that down-scales the alpha image and the RGB_view image. In addition, in the first information processing apparatus, the first atlas transformation unit can perform atlas transformation on the down-scaled alpha image. Further, the second atlas transformation unit can perform atlas transformation on the RGB_base image and the down-scaled RGB_view image independently of each other.

[0145] Further, the second information processing apparatus described above can further include an up-scaling processing unit that up-scales the alpha image and the RGB_view image.

[0146] As in the example shown in FIG. 10, Figure 9 For the alpha image, the RGB_base image, and the RGB_view image that have the same size (e.g., 2 K), the size of the alpha image and the RGB_view image can be down-scaled (e.g., to 1 K) to perform atlas transformation for encoding, as in FIG. 11.​​ Figure 14 In this way, by reducing the images for transmission and enlarging the images after transmission to restore the images to their original size, the amount of data during transmission can be reduced. Thus, an increase in transmission capacity required for data transmission can additionally be suppressed.

[0147] Note that since the alpha image is information indicating the presence of an object, even if the resolution of the alpha image is reduced, the influence on the subjective image quality of the rendered image is relatively small. Further, since the RGB_view image is mixed with a direction-dependent mixing ratio, the influence on the subjective image quality of the rendered image is relatively small compared to the RGB_base image. Thus, by reducing the alpha image and the RGB_view image as described above, an increase in transmission capacity required for data transmission can additionally be suppressed while suppressing a decrease in the subjective image quality of the rendered image.

[0148] <Method 1-2-2>

[0149] Further, when the above-described method 1-2 is applied, if the sizes of the RGB_base and view are different, the atlas transform information of one of the sizes of the RGB can be transmitted, as shown in the ninth row from the top of the table in Figure 7 Additionally, the atlas transform information of the other size can also not be transmitted (method 1-2-2).

[0150] For example, in the above-described first information processing apparatus, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the RGB_base image and the RGB_view image can have different sizes. Further, in the first information processing apparatus, the second atlas transform unit can perform atlas transformation on the RGB_base image and the RGB_view image to generate atlas transform information of the RGB_base component or the RGB_view component. Note that the atlas transform information can be information indicating the content of the atlas transformation.

[0151] For example, in the second information processing apparatus described above, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the RGB_base image and the RGB_view image can have different sizes. Further, the second decoding unit can decode each of the encoded data of the atlas image of the RGB_base component and the encoded data of the atlas image of the RGB_view component. Further, the second atlas inverse transformation unit can perform the atlas inverse transformation on the atlas image of the RGB_base component and the atlas image of the RGB_view component using the atlas transformation information of the RGB_base component or the RGB_view component.

[0152] For example, the second information processing apparatus described above can further include a reduction processing unit that reduces the atlas transformation information of the RGB_base component or the RGB_view component.

[0153] Even when the sizes of the RGB_base image and the RGB_view image are different, as in the example shown in Figure 14 the atlas transformation of the two images is performed using the mask image generated from the same alpha image, and thus the arrangement of one atlas image and the arrangement of the other atlas image are similar, as in the example shown in Figure 16 That is, the size of one of the atlas transformation information can be transformed so as to match the size of the other atlas transformation information. Thus, the size of one of the atlas transformation information can be transformed to generate the other atlas transformation information. That is, one of the atlas transformation information can be transmitted (without transmitting the other atlas transformation information), and the size of the transmitted one of the atlas transformation information is transformed (scaling transformation) during the atlas inverse transformation of the other atlas transformation information to generate and apply the other atlas transformation information. With such a configuration, it is possible to reduce the amount of data during transmission. Thus, the increase in transmission capacity required for data transmission can be additionally suppressed.

[0154] Note that the atlas transformation information to be transmitted can be for the RGB_base component or can be for the RGB_view component. Further, the atlas transformation information for the larger image size can be transmitted, or the atlas transformation information for the smaller image size can be transmitted.

[0155] When the atlas transform information for a larger image size is transmitted, the atlas transform information for a smaller image size can be generated by reducing the transmitted atlas transform information at the transmission destination. With such a configuration, the atlas transform information with higher accuracy can be transmitted, and the atlas inverse transform can be performed more accurately.

[0156] Further, when the atlas transform information for a smaller image size is transmitted, the atlas transform information for a larger image size can be generated by enlarging the transmitted atlas transform information at the transmission destination. With such a configuration, the amount of data during transmission can be additionally reduced. Thus, the increase in the transmission capacity required for data transmission can be additionally suppressed.

[0157] <Method 1-2-2-1>

[0158] Further, when the above-described method 1-2 is applied, the atlas transform information to be transferred, the reduction instruction, the reduction ratio, and the like can be defined in a file, as shown in the 10th row from the top of the table in Figure 7

[0159] For example, assume that the RGB_view image is smaller than the RGB_base image, and the atlas transform information (RGB_base) is reduced to generate the atlas transform information (RGB_view), as in the example shown in Figure 14 In this case, as shown in Figure 17 , for example, in the V3C bitstream, flag information indicating whether or not the reduction processing is performed, information indicating the reduction ratio at the time of performing the reduction processing, and the like can be stored in the V3C_AVD storing the atlas image (RGB_base) and the atlas image (RGB_view). That is, these pieces of information can be stored in the bitstream and transmitted. Then, when the reduction processing is performed, the V3C_AD associated with this V3C_AVD can be generated by reducing another V3C_AD (V3C_AD(1)).

[0160] <Method 1-3 and Method 1-3-1>

[0161] When the above-described method 1 is applied, the identification information for the RGB image can be used to identify a plurality of RGB images, as shown in the 11th row from the top of the table in Figure 7 (Method 1-3). Further, when this method 1-3 is applied, the attribute_id can be used to identify a plurality of RGB images, as shown in the bottom row of the table in Figure 7 (Method 1-3-1).

[0162] ​For example, in the first information processing apparatus described above, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the atlas transformation unit can perform atlas transformation on the RGB_base image and the RGB_view image to generate atlas images thereof. Further, the first information processing apparatus can further include a multiplexing unit that multiplexes the encoded data of the atlas image of the alpha component, the encoded data of the atlas image of the RGB_base component, and the encoded data of the atlas image of the RGB_view component using the identification information to generate the bitstream. Further, in the first information processing apparatus described above, the identification information can be information for identifying the RGB_base component and the RGB_view component.

[0163] For example, in the second information processing apparatus described above, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the second information processing apparatus can further include a demultiplexing unit that demultiplexes the bitstream to generate the encoded data of the atlas image of the alpha component, the encoded data of the atlas image of the RGB_base component, and the encoded data of the atlas image of the RGB_view component.

[0164] For example, in the second information processing apparatus described above, the demultiplexing unit can demultiplex the bitstream using identification information for identifying the RGB_base component and the RGB_view component.

[0165] When there are a plurality of RGB images as in NeX MPI, it is necessary to identify each of the RGB images. In this regard, as described above, the plurality of RGB images can be identified using identification information for the RGB images. Note that any specification can be used for the identification information. For example, identification information for identifying an attribute (attribute_id) can be used as the identification information. For example, in the case of a V3C bitstream, vuh_attribute_index stored in a V3C unit header can be used as the identification information.

[0166] By using such identification information, it becomes easy to identify each RGB, and an increase in the load of the reproduction processing (decoding processing) can be suppressed.

[0167] <About the method>

[0168] Each of the methods described above can be applied in combination with other methods as long as no contradiction arises. For example, Figure 7 Methods 1-3 shown in the table of Figure 7 A plurality of methods among Methods 1-2-1-1 to 1-2-1-3 shown in the table of Figure 7 The methods shown in

[0169] Note that, in the description of the present specification, a higher-order method can include a lower-order method. In other words, the application of Method 1 can include the application of one or more of Methods 1-1 to 1-3, Method 1-1-1, Methods 1-2-1 and 1-2-2, Method 1-3-1, Methods 1-2-1-1 to 1-2-1-3, and Method 1-2-2-1. Similarly, the application of Method 1-1 can include the application of Method 1-1-1. Similarly, the application of Method 1-2 can include the application of one or more of Methods 1-2-1 and 1-2-2, Methods 1-2-1-1 to 1-2-1-3, and Method 1-2-2-1. Similarly, the application of Method 1-2-1 can include the application of one or more of Methods 1-2-1-1 to 1-2-1-3. Similarly, the application of Method 1-2-2 can include the application of Method 1-2-2-1. Similarly, the application of Method 1-3 can include the application of Method 1-3-1.

[0170] Thus, for example, a case described as "application of Method 1" can include a case in which one or more of Methods 1-1 to 1-3, Method 1-1-1, Methods 1-2-1 and 1-2-2, Method 1-3-1, Methods 1-2-1-1 to 1-2-1-3, and Method 1-2-2-1 are applied.

[0171] <4. First Embodiment>

[0172] <Information Processing System>

[0173] Figure 18 is a block diagram showing a main configuration example of an information processing system to which the present technology described in <3. Encoding / decoding using atlas transformation> above is applied. Figure 18 The information processing system 300 shown in

[0174] As Figure 18 As the device of the transmission side (the device of the side that provides NeX data), the NeX learning device 311, the NeX MPI generation device 312, the post-processing device 313, the packing device 314, and the encoding device 315 perform a transmission process and perform processing relating to the provision of NeX data. The flow of the transmission process will be described with reference to a flowchart shown in FIG. 3.

[0175] <Flow of transmission process>

[0176] As the device of the transmission side (the device of the side that provides NeX data), the NeX learning device 311, the NeX MPI generation device 312, the post-processing device 313, the packing device 314, and the encoding device 315 perform a transmission process and perform processing relating to the provision of NeX data. The flow of the transmission process will be described with reference to a flowchart shown in FIG. 3. Figure 19 An example of the flow of the transmission process will be described with reference to a flowchart shown in FIG. 3.

[0177] When the transmission process is started, in step S301, the NeX learning device 311 performs learning of NeX and sets MLP parameters. In step S302, the NeX MPI generation device 312 generates a NeX MPI using the MLP. In step S303, the post-processing device 313 performs post-processing on the NeX MPI. In step S304, the packing device 314 packs the post-processed NeX MPI. In step S304, the encoding device 315 performs an encoding process to encode the packed NeX MPI and generates a bitstream. At this time, the encoding device 315 applies the method 1 described above to perform atlas transformation on an alpha image and / or an RGB image to encode. In step S305, the encoding device 315 transmits the generated bitstream.

[0178] <Flow of reproduction process>

[0179] As the device of the transmission side (the device of the side that provides NeX data), the NeX learning device 311, the NeX MPI generation device 312, the post-processing device 313, the packing device 314, and the encoding device 315 perform a transmission process and perform processing relating to the provision of NeX data. The flow of the transmission process will be described with reference to a flowchart shown in FIG. 3. Figure 20 An example of the flow of the reproduction process will be described with reference to a flowchart shown in FIG. 4.

[0180] When the reproduction processing starts, in step S351, the decoding device 316 receives the bitstream transmitted from the encoding device 315. In step S352, the decoding device 316 performs decoding processing to decode the received bitstream, and generates (recovers) an alpha image and / or an RGB image. In step S353, the unpacking device 317 unpacks the alpha image and / or the RGB image. In step S354, the rendering device 318 performs rendering using the unpacked alpha image and / or the RGB image to generate a rendered image of a desired viewpoint. In step S355, the display preprocessing device 319 performs display preprocessing on the rendered image. In step S356, the display device 320 displays the rendered image.

[0181] By performing each processing in this way, the information processing system 300 can suppress an increase in transmission capacity required for transmission of 3D data using MPI as in NeX.

[0182] <Encoding device>

[0183] Figure 21 is a block diagram showing a main configuration example of the encoding device 315. As shown in Figure 21 The encoding device 315 includes a non-zero region extraction unit 511, a atlas transformation unit 512, an encoding unit 513A, an encoding unit 513B, a mask image generation unit 514, an atlas transformation unit 515, an encoding unit 516A, an encoding unit 516B, and a multiplexing unit 517.

[0184] The non-zero region extraction unit 511 performs processing related to extraction of a non-zero region of an alpha image. For example, the non-zero region extraction unit 511 can acquire an alpha image supplied to the encoding device 315. Further, the non-zero region extraction unit 511 can extract a non-zero region from the alpha image. Further, the non-zero region extraction unit 511 can supply the extracted non-zero region to the atlas transformation unit 512. Further, the non-zero region extraction unit 511 can supply the alpha image to the mask image generation unit 514. Note that the non-zero region extraction unit 511 can perform such processing for each of layers in a layer group representing a three-dimensional space.

[0185] The atlas conversion unit 512 performs processing relating to atlas conversion of the alpha component. For example, the atlas conversion unit 512 can acquire the non-zero region of the alpha component supplied from the non-zero region extraction unit 511. The atlas conversion unit 512 can perform atlas conversion of the alpha component, and arrange the non-zero region in an atlas image (alpha). Further, the atlas conversion unit 512 can generate atlas conversion information for the atlas conversion. Further, the atlas conversion unit 512 can supply the atlas image (alpha), which is the generated atlas image of the alpha component, to the encoding unit 513A. The atlas conversion unit 512 can also supply the atlas conversion information (alpha), which is the generated atlas conversion information of the alpha component, to the encoding unit 513B. Note that the atlas conversion unit 512 can perform such processing on each of the layers of the layer group representing the three-dimensional space.

[0186] The encoding units 513A and 513B perform processing relating to encoding of the alpha component. For example, the encoding unit 513A can acquire the atlas image (alpha) supplied from the atlas conversion unit 512. The encoding unit 513A can encode the atlas image (alpha) using a 2D encoding method (e.g., JPEG (Joint Photographic Experts Group), MPEG, or the like) to generate encoded data. The encoding unit 513A can supply the encoded data of the generated atlas image (alpha) (will also be referred to as encoded data (alpha)) to the multiplexing unit 517. Further, the encoding unit 513B can acquire the atlas conversion information (alpha) supplied from the atlas conversion unit 512. The encoding unit 513B can encode the atlas conversion information (alpha) to generate encoded data. Any method can be used as the encoding method of this atlas conversion information (alpha). For example, the encoding unit 513B can encode the atlas conversion information (alpha) in accordance with the MIV / V3C standard. The encoding unit 513B can supply the encoded data of the generated atlas conversion information (alpha) (may be included in the encoded data (alpha)) to the multiplexing unit 517. Note that the encoding units 513A and 513B can perform such processing on each of the layers of the layer group representing the three-dimensional space. Note that the encoding units 513A and 513B will also be referred to as the encoding unit 513.

[0187] The mask image generation unit 514 performs processing relating to generation of a mask image. For example, the mask image generation unit 514 can acquire the alpha image supplied from the non-zero region extraction unit 511. Further, the mask image generation unit 514 can generate a mask image using the acquired alpha image. At this time, the mask image generation unit 514 can generate intermediate data (mask image) by compositing the alpha image of the corresponding layer for each of the plurality of layers corresponding to a single RGB image in succession. Further, the mask image generation unit 514 can supply the generated mask image to the atlas conversion unit 515.

[0188] The atlas conversion unit 515 performs processing relating to atlas conversion of RGB components. For example, the atlas conversion unit 515 can acquire the RGB images (RGB_base image and RGB_view image) supplied to the encoding device 315. Further, the atlas conversion unit 515 can acquire the mask image supplied from the mask image generation unit 514. Further, the atlas conversion unit 515 can perform atlas conversion of RGB components using the mask image and the RGB images. For example, the atlas conversion unit 515 can extract a region corresponding to the non-zero region of the mask image from the RGB_base image, and arrange the region in the atlas image (RGB_base). Further, the atlas conversion unit 515 can extract a region corresponding to the non-zero region of the mask image from the RGB_view image, and arrange the region in the atlas image (RGB_view). Further, the atlas conversion unit 515 can generate atlas conversion information (RGB) which is atlas conversion information common to those atlas conversions. Further, the atlas conversion unit 515 can supply the atlas image (RGB_base) which is the atlas image of the generated RGB_base component and the atlas image (RGB_view) which is the atlas image of the generated RGB_view component to the encoding unit 516A. The atlas conversion unit 515 can also supply the atlas conversion information (RGB) which is the atlas conversion information of the generated RGB components to the encoding unit 516B. Note that the atlas conversion unit 515 can perform such processing for each of the "plurality of layers in succession corresponding to a single RGB image" in the layer group representing a three-dimensional space.

[0189] The encoding unit 516A and the encoding unit 516B perform processing relating to encoding of the RGB_base component and the RGB_view component (may also be referred to as RGB components). For example, the encoding unit 516A can acquire the atlas image (RGB_base) supplied from the atlas conversion unit 515. The encoding unit 516A can encode the atlas image (RGB_base) using a 2D encoding method (for example, JPEG, MPEG, or the like) to generate encoded data. The encoding unit 516A can supply the generated encoded data of the atlas image (RGB_base) (will also be referred to as encoded data (RGB_base)) to the multiplexing unit 517. Further, the encoding unit 516A can acquire the atlas image (RGB_view) supplied from the atlas conversion unit 515. The encoding unit 516A can encode the atlas image (RGB_view) using a 2D encoding method (for example, JPEG, MPEG, or the like) to generate encoded data. The encoding unit 516A can supply the generated encoded data of the atlas image (RGB_view) (will also be referred to as encoded data (RGB_view)) to the multiplexing unit 517. Further, the encoding unit 516B can acquire the atlas conversion information (RGB) supplied from the atlas conversion unit 515. The encoding unit 516B can encode the atlas conversion information (RGB) to generate encoded data. Any method can be used as an encoding method for this atlas conversion information (RGB). For example, the encoding unit 516B can encode the atlas conversion information (RGB) in accordance with the MIV / V3C standard. The encoding unit 516B can supply the generated encoded data of the atlas conversion information (RGB) (will also be referred to as encoded data (RGB)) to the multiplexing unit 517. Note that the encoding unit 516A and the encoding unit 516B can perform processing on each of "a plurality of layers continuous corresponding to a single RGB image" in a layer group representing a three-dimensional space. Note that the encoding unit 516A and the encoding unit 516B are also referred to as the encoding unit 516.

[0190] The multiplexing unit 517 performs processing relating to multiplexing of data. For example, the multiplexing unit 517 can acquire the encoded data (alpha) (encoded data of the atlas image (alpha) and the atlas conversion information (alpha)) supplied from the encoding unit 513A and the encoding unit 513B. The multiplexing unit 517 can also acquire the encoded data (RGB_base) and the encoded data (RGB_view) supplied from the encoding unit 516A. The multiplexing unit 517 can also acquire the encoded data (encoded data (RGB)) of the atlas conversion information (RGB) from the encoding unit 516B. Further, the multiplexing unit 517 can multiplex these acquired information pieces to generate a bitstream. Further, the multiplexing unit 517 can transmit the generated bitstream.

[0191] In the encoding device 315 having such a configuration, the present technology described above in <3. Encoding / decoding using atlas transformation> can be applied. For example, the method 1 described above can be applied to the encoding device 315. For example, the atlas transformation unit 512 can perform atlas transformation on the alpha image of the 3D data to generate an atlas image of the alpha component. Further, the encoding unit 513A can encode the atlas image of the alpha component. Further, the atlas transformation unit 515 can perform atlas transformation on the RGB image of the 3D data to generate an atlas image of the RGB component. Further, the encoding unit 516A can encode the atlas image of the RGB component.

[0192] Further, the method 1-1 described above can be applied to the encoding device 315. For example, the mask image generation unit 514 can generate a mask image in which non-zero regions of the alpha images of the plurality of layers in succession are synthesized. Further, the atlas transformation unit 515 can perform atlas transformation on the RGB image using the mask image.

[0193] Further, the method 1-1-1 described above can be applied to the encoding device 315. For example, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the atlas transformation unit 515 can perform atlas transformation on the RGB_base image and the RGB_view image to generate atlas images thereof, and generate atlas transformation information common to the RGB_base component and the RGB_view component.

[0194] Further, the method 1-3 described above can be applied to the encoding device 315. Further, the method 1-3-1 described above can be applied to the encoding device 315. For example, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the atlas transformation unit 515 can perform atlas transformation on the RGB_base image and the RGB_view image to generate atlas images thereof. Further, the multiplexing unit 517 can multiplex the encoded data of the atlas image of the alpha component, the encoded data of the atlas image of the RGB_base component, and the encoded data of the atlas image of the RGB_view component using identification information to generate a bitstream. Further, the identification information can be information for identifying the RGB_base component and the RGB_view component.

[0195] With such a configuration, the encoding device 315 can suppress an increase in transmission capacity required for transmission of 3D data using MPI as in NeX. Note that when the post-processed NeX MPI is packed, processing up to atlas transformation can be performed. In other words, the packing device 314 and the encoding device 315 can be integrated.

[0196] <Flow of encoding processing>

[0197] An example of a flow of encoding processing performed by the encoding device 315 will be described with reference to a flowchart illustrated in FIG. 6. Figure 22

[0198] When the encoding processing starts, in step S501, the non-zero region extraction unit 511 extracts a non-zero region from the alpha image. In step S502, the atlas transformation unit 512 performs atlas transformation on the alpha component to generate an atlas image (alpha) and atlas transformation information (alpha). In step S503, the encoding unit 513A encodes the atlas image of the alpha component to generate encoded data (alpha). In addition, the encoding unit 513B encodes the atlas transformation information of the alpha component to generate encoded data of the atlas transformation information (alpha) (encoded data (alpha)).

[0199] In step S504, the mask image generation unit 514 generates a mask image using the non-zero regions of the main layer and the sub layer of the alpha image.

[0200] In step S505, the atlas transformation unit 515 performs atlas transformation on RGB_base using the mask image to generate an atlas image (RGB_base). In step S506, the encoding unit 516A encodes the atlas image of the RGB_base component to generate encoded data (RGB_base).

[0201] In step S507, the atlas transformation unit 515 performs atlas transformation on RGB_view using the mask image to generate an atlas image (RGB_view). In step S508, the encoding unit 516A encodes the atlas image of the RGB_view component to generate encoded data (RGB_view).

[0202] Note that in the processing of step S505 or the processing of step S507, the atlas transformation unit 515 generates atlas transformation information (RGB). In step S509, the encoding unit 516B encodes the atlas transformation information of the RGB component to generate encoded data of the atlas transformation information (RGB) (encoded data (RGB)).

[0203] ​In step S510, the multiplexing unit 517 multiplexes the encoded data (alpha) (encoded data of the atlas image (alpha) and the atlas transform information (alpha)), the encoded data (RGB_base), the encoded data (RGB_view), and the encoded data (RGB) to generate a bitstream.

[0204] When the processing of step S509 ends, the encoding processing ends, and the processing returns to Figure 19 .

[0205] In this way, in each processing, the present technology described above in <3. Encoding / decoding using atlas transform> can be applied. For example, the method 1 described above can be applied. Further, the method 1-1 described above can be applied. Further, the method 1-1-1 described above can be applied. Further, the method 1-3 described above can be applied. Further, the method 1-3-1 described above can be applied.

[0206] By performing each processing in this way, the encoding device 315 can suppress an increase in transmission capacity required for transmission of 3D data using MPI as in NeX.

[0207] <Decoding device>

[0208] Figure 23 is a block diagram illustrating a main configuration example of the decoding device 316. As Figure 23 illustrated in <2. Encoding device>, the decoding device 316 includes a demultiplexing unit 551, a decoding unit 552A, a decoding unit 552B, an atlas inverse transform unit 553, a decoding unit 554A, a decoding unit 554B, and an atlas inverse transform unit 555.

[0209] The demultiplexing unit 551 performs a process related to demultiplexing. For example, the demultiplexing unit 551 can acquire a bitstream supplied to the decoding apparatus 316. Further, the demultiplexing unit 551 can demultiplex the bitstream to generate encoded data of the atlas image (alpha) (encoded data (alpha)), encoded data of the atlas transform information (alpha) (encoded data (alpha)), encoded data of the atlas image (RGB_base) (encoded data (RGB_base)), encoded data of the atlas image (RGB_view) (encoded data (RGB_view)), and encoded data of the atlas transform information (RGB) (encoded data (RGB)). Further, the demultiplexing unit 551 can supply the encoded data of the atlas image (alpha) (encoded data (alpha)) to the decoding unit 552A. The demultiplexing unit 551 can also supply the encoded data of the atlas transform information (alpha) (encoded data (alpha)) to the decoding unit 552B. The demultiplexing unit 551 can also supply the encoded data of the atlas image (RGB_base) (encoded data (RGB_base)) and the encoded data of the atlas image (RGB_view) (encoded data (RGB_view)) to the decoding unit 554A. The demultiplexing unit 551 can also supply the encoded data of the atlas transform information (RGB) (encoded data (RGB)) to the decoding unit 554B.

[0210] The decoding units 552A and 552B perform a process related to decoding of the alpha component. For example, the decoding unit 552A can acquire the encoded data of the atlas image (alpha) (encoded data (alpha)) supplied from the demultiplexing unit 551. The decoding unit 552A can decode the encoded data (alpha) by a 2D decoding method (e.g., JPEG, MPEG, or the like) to generate the atlas image (alpha). The decoding unit 552A can supply the atlas image (alpha) to the atlas inverse transform unit 553. Further, the decoding unit 552B can acquire the encoded data of the atlas transform information (alpha) (encoded data (alpha)) supplied from the demultiplexing unit 551. The decoding unit 552B can decode the encoded data (alpha) by a 2D decoding method (e.g., JPEG, MPEG, or the like) to generate the atlas transform information (alpha). The decoding method can be any method. For example, the decoding unit 552B can decode the encoded data (alpha) in accordance with the MIV / V3C standard. Further, the decoding unit 552B can supply the atlas transform information (alpha) to the atlas inverse transform unit 553. Note that the decoding units 552A and 552B will also be referred to as the decoding unit 552.

[0211] The atlas inverse transform unit 553 performs processing related to an atlas inverse transform of the alpha component. For example, the atlas inverse transform unit 553 can acquire the atlas image (alpha) supplied from the decoding unit 552A. The atlas inverse transform unit 553 can also acquire the atlas transform information (alpha) supplied from the decoding unit 552B. Further, the atlas inverse transform unit 553 can perform an atlas inverse transform on the atlas image (alpha) using the atlas transform information (alpha) to generate (restore) the alpha image. The atlas inverse transform unit 553 can supply the generated alpha image to the depacketizer 317.

[0212] The decoding units 554A and 554B perform processing related to decoding of the RGB components. For example, the decoding unit 554A can acquire the encoded data (RGB_base) and the encoded data (RGB_view) provided from the demultiplexing unit 551. The decoding unit 554A can decode the encoded data (RGB_base) and the encoded data (RGB_view) by a 2D decoding method (e.g., JPEG, MPEG, etc.) to generate the atlas image (RGB_base) and the atlas image (RGB_view). The decoding unit 554A can supply the atlas image (RGB_base) and the atlas image (RGB_view) to the atlas inverse transform unit 555. The decoding unit 554B can acquire the encoded data (RGB) supplied from the demultiplexing unit 551. The decoding unit 554B can decode the encoded data (RGB) by a 2D decoding method (e.g., JPEG, MPEG, etc.) to generate the atlas transform information (RGB). The decoding method can be any method. For example, the decoding unit 554B can decode the encoded data (RGB) according to the MIV / V3C standard. Further, the decoding unit 554B can provide the atlas transform information (RGB) to the atlas inverse transform unit 555. Note that the decoding units 554A and 554B will also be referred to as the decoding unit 554.

[0213] The atlas inverse transform unit 555 performs processing relating to the atlas inverse transform of the RGB components. For example, the atlas inverse transform unit 555 can acquire the atlas image (RGB_base) and the atlas image (RGB_view) supplied from the decoding unit 554A. The atlas inverse transform unit 555 can also acquire the atlas transform information (RGB) supplied from the decoding unit 554B. Further, the atlas inverse transform unit 555 can perform the atlas inverse transform on the atlas image (RGB_base) and the atlas image (RGB_view) using the atlas transform information (RGB) to generate (restore) the RGB images (RGB_base image and RGB_view image). The atlas inverse transform unit 553 can supply the generated RGB images (RGB_base image and RGB_view image) to the depacketizing device 317.

[0214] In the decoding device 316 having such a configuration, the present technology described above in <3. Encoding / decoding using atlas transform> can be applied. For example, the method 1 described above can be applied to the decoding device 316. For example, the decoding unit 552A can decode the encoded data to generate the atlas image of the alpha component. Further, the atlas inverse transform unit 553 can perform the atlas inverse transform on the atlas image of the alpha component to generate the alpha image of the 3D data. Further, the decoding unit 554A can decode the encoded data to generate the atlas images of the RGB components. Further, the atlas inverse transform unit 555 can perform the atlas inverse transform on the atlas images of the RGB components to generate the RGB images of the 3D data.

[0215] Further, the method 1-1-1 described above can be applied to the decoding device 316. For example, the RGB images can include the RGB_base image indicating the distribution of the RGB_base component among the RGB components and the RGB_view image indicating the distribution of the RGB_view component among the RGB components. Further, the decoding unit 554A can decode the encoded data to generate the atlas image of the RGB_base component and the atlas image of the RGB_view component. Further, the atlas inverse transform unit 555 can perform the atlas inverse transform on the atlas image of the RGB_base component and the atlas image of the RGB_view component using the atlas transform information common to the RGB_base component and the RGB_view component.

[0216] Further, the methods 1-3 described above can be applied to the decoding device 316. Further, the methods 1-3-1 described above can be applied to the decoding device 316. For example, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the demultiplexing unit 551 can demultiplex the bitstream to generate the encoded data of the atlas image of the alpha component, the encoded data of the atlas image of the RGB_base component, and the encoded data of the atlas image of the RGB_view component. Further, the demultiplexing unit 551 can demultiplex the bitstream using the identification information for identifying the RGB_base component and the RGB_view component.

[0217] With such a configuration, the decoding device 316 can suppress an increase in transmission capacity required for transmission of 3D data using MPI as in NeX. Note that the atlas inverse transform can be performed at the time of unpacking the alpha image and / or the RGB image (RGB_base image and RGB_view image). In other words, the decoding device 316 and the unpacking device 317 can be integrated.

[0218] <Flow of decoding processing>

[0219] An example of the flow of the decoding processing performed by the decoding device 316 will be described with reference to the flowchart illustrated in FIG. 17. Figure 24 An example of the flow of the decoding processing performed by the decoding device 316 will be described with reference to the flowchart illustrated in FIG. 17.

[0220] When the decoding processing starts, in step S551, the demultiplexing unit 551 demultiplexes the bitstream to generate the encoded data of the atlas image (alpha), the encoded data of the atlas transform information (alpha), the encoded data of the atlas image (RGB_base), the encoded data of the atlas image (RGB_view), and the encoded data of the atlas transform information (RGB).

[0221] In step S552, the decoding unit 552A decodes the encoded data of the alpha component to generate the atlas image (alpha). Further, the decoding unit 552B decodes the encoded data of the atlas transform information (alpha) (encoded data (alpha)) to generate the atlas transform information (alpha). In step S553, the atlas inverse transform unit 553 performs the atlas inverse transform on the atlas image (alpha) using the atlas transform information (alpha) to generate the alpha image.

[0222] In step S554, the decoding unit 554B decodes the encoded data (RGB) to generate atlas transform information (RGB). In step S555, the decoding unit 554A decodes the encoded data of the RGB_base component to generate an atlas image (RGB_base). In step S556, the atlas inverse transform unit 555 performs atlas inverse transform on the atlas image (RGB_base) using the atlas transform information (RGB) to generate an RGB_base image.

[0223] In step S557, the decoding unit 554A decodes the encoded data of the RGB_view component to generate an atlas image (RGB_view). In step S558, the atlas inverse transform unit 555 performs atlas inverse transform on the atlas image (RGB_view) using the atlas transform information (RGB) to generate an RGB_view image.

[0224] When the processing of step S558 ends, the decoding processing ends, and the processing returns to Figure 20 .

[0225] In this way, in each processing, the present technology described above in <3. Encoding / decoding using atlas transform> can be applied. For example, the method 1 described above can be applied. Further, the method 1-1 described above can be applied. Further, the method 1-1-1 described above can be applied. Further, the method 1-3 described above can be applied. Further, the method 1-3-1 described above can be applied.

[0226] By performing each processing in this way, the decoding device 316 can suppress an increase in transmission capacity required for transmission of 3D data using MPI like in NeX.

[0227] <Encoding device>

[0228] Further, the method 1-2 described above in <3. Encoding / decoding using atlas transform> can be applied to the encoding device 315. In Figure 25 a main configuration example of the encoding device 315 in this case is shown.

[0229] In this case, the encoding device 315 also basically has a configuration similar to that of the example shown in Figure 21 However, in Figure 25 In the case of the example shown in FIG. 6, the non-zero region extraction unit 511 includes a non-zero region extraction unit 611 and a non-zero region extraction unit 612. Further, the mask image generation unit 514 includes a mask image generation unit 613 and a mask image generation unit 614. Further, the atlas conversion unit 515 includes an atlas conversion unit 615 and an atlas conversion unit 616. Further, the encoding unit 516 includes an encoding unit 617A, an encoding unit 617B, an encoding unit 618A, and an encoding unit 618B. Further, in the case of this example, the encoding apparatus 315 further includes a reduction processing unit 621 and a reduction processing unit 622.

[0230] The non-zero region extraction unit 611 can acquire the reduced alpha image supplied from the reduction processing unit 621 (also to be referred to as a reduced alpha image). The non-zero region extraction unit 611 can further extract a non-zero region from the reduced alpha image. Further, the non-zero region extraction unit 611 can supply the extracted non-zero region to the atlas conversion unit 512. The non-zero region extraction unit 611 can further supply the reduced alpha image to the mask image generation unit 613.

[0231] The non-zero region extraction unit 612 can acquire the alpha image supplied to the encoding apparatus 315. Further, the non-zero region extraction unit 612 can extract a non-zero region from the alpha image. Further, the non-zero region extraction unit 612 can supply the extracted non-zero region and the alpha image to the mask image generation unit 614.

[0232] The mask image generation unit 613 can acquire the reduced alpha image supplied from the non-zero region extraction unit 611. Further, the mask image generation unit 613 can generate a mask image (also to be referred to as a reduced mask image) using the acquired reduced alpha image. At this time, the mask image generation unit 613 can generate intermediate data (reduced mask image) by synthesizing the reduced alpha image of the corresponding layer for each of the plurality of layers corresponding to a single RGB image. Further, the mask image generation unit 613 can supply the generated reduced mask image to the atlas conversion unit 615.

[0233] The mask image generation unit 614 can acquire the alpha image and its non-zero region supplied from the non-zero region extraction unit 612. Further, the mask image generation unit 614 can generate a mask image using the acquired alpha image and its non-zero region. At this time, the mask image generation unit 614 can generate intermediate data (mask image) by synthesizing the alpha image of the corresponding layer for each of the plurality of layers corresponding to a single RGB image. Further, the mask image generation unit 614 can supply the generated mask image to the atlas conversion unit 616.

[0234] The atlas conversion unit 615 can acquire the reduced RGB_view image supplied from the reduction processing unit 622 (also to be referred to as a reduced RGB_view image). Further, the atlas conversion unit 615 can acquire the reduced mask image supplied from the mask image generating unit 613. Further, the atlas conversion unit 615 can perform atlas conversion of the RGB_view component using the reduced mask image and the reduced RGB_view image. For example, the atlas conversion unit 615 can extract a region corresponding to a non-zero region of the reduced mask image from the reduced RGB_view image, and arrange the region in an atlas image (RGB_view). Further, the atlas conversion unit 615 can generate atlas conversion information (RGB_view) relating to this atlas conversion. Further, the atlas conversion unit 615 can supply the atlas image (RGB_view) that is the atlas image of the generated RGB_view component to the encoding unit 617A. Further, the atlas conversion unit 615 can supply the generated atlas conversion information (RGB_view) to the encoding unit 617B. Note that the atlas conversion unit 615 can perform such processing on each of "a plurality of layers continuous corresponding to a single RGB image" in a layer group representing a three-dimensional space.

[0235] The atlas conversion unit 616 can acquire the RGB_base image supplied to the encoding apparatus 315. The atlas conversion unit 616 can also acquire the mask image supplied from the mask image generating unit 614. Further, the atlas conversion unit 616 can perform atlas conversion of the RGB_base component using the mask image and the RGB_base image. For example, the atlas conversion unit 616 can extract a region corresponding to a non-zero region of the mask image from the RGB_base image, and arrange the region in an atlas image (RGB_base). Further, the atlas conversion unit 616 can generate atlas conversion information (RGB_base) relating to the atlas conversion. Further, the atlas conversion unit 616 can supply the atlas image (RGB_base) that is the atlas image of the generated RGB_base component to the encoding unit 618A. Further, the atlas conversion unit 616 can supply the generated atlas conversion information (RGB_base) to the encoding unit 618B. Note that the atlas conversion unit 616 can perform such processing on each of "a plurality of layers continuous corresponding to a single RGB image" in a layer group representing a three-dimensional space.

[0236] The encoding unit 617A and the encoding unit 617B perform processing relating to encoding of the RGB_view component. For example, the encoding unit 617A can acquire the atlas image (RGB_view) provided from the atlas conversion unit 615. Further, the encoding unit 617A can encode the atlas image (RGB_view) using a 2D encoding method (for example, JPEG, MPEG, or the like) to generate encoded data. Further, the encoding unit 617A can supply the generated encoded data of the atlas image (RGB_view) (encoded data (RGB_view)) to the multiplexing unit 517. Further, the encoding unit 617B can acquire the atlas conversion information (RGB_view) provided from the atlas conversion unit 515. The encoding unit 617B can encode the atlas conversion information (RGB_view) to generate encoded data. Any method can be used as an encoding method for this atlas conversion information (RGB_view). For example, the encoding unit 617B can encode the atlas conversion information (RGB_view) in accordance with the MIV / V3C standard. The encoding unit 617B can supply the generated encoded data of the atlas conversion information (RGB_view) (also referred to as encoded data (RGB_view)) to the multiplexing unit 517. Note that the encoding unit 617A and the encoding unit 617B can perform processing on each "continuous multiple layers corresponding to a single RGB image" in the layer group representing a three-dimensional space. Note that the encoding unit 617A and the encoding unit 617B are also referred to as the encoding unit 617.

[0237] The encoding unit 618A and the encoding unit 618B perform processing relating to encoding of the RGB_base component. For example, the encoding unit 618A can acquire the atlas image (RGB_base) provided from the atlas conversion unit 616. The encoding unit 618A can encode the atlas image (RGB_base) using a 2D encoding method (for example, JPEG, MPEG, or the like) to generate encoded data. The encoding unit 618A can supply the generated encoded data of the atlas image (RGB_base) (encoded data (RGB_base)) to the multiplexing unit 517. Further, the encoding unit 618B can acquire the atlas conversion information (RGB_base) provided from the atlas conversion unit 515. The encoding unit 618B can encode the atlas conversion information (RGB_base) to generate encoded data. Any method can be used as an encoding method for this atlas conversion information (RGB_base). For example, the encoding unit 618B can encode the atlas conversion information (RGB_base) in accordance with the MIV / V3C standard. The encoding unit 618B can supply the generated encoded data of the atlas conversion information (RGB_base) (also referred to as encoded data (RGB_base)) to the multiplexing unit 517. Note that the encoding unit 618A and the encoding unit 618B can perform processing on each of "a plurality of layers that are continuous and correspond to a single RGB image" in a layer group that represents a three-dimensional space. Note that the encoding unit 618A and the encoding unit 618B are also referred to as the encoding unit 618.

[0238] In this case, the multiplexing unit 517 can acquire the encoded data (alpha) (encoded data of the atlas image (alpha)) provided from the encoding unit 513A. The multiplexing unit 517 can also acquire the encoded data (alpha) (encoded data of the atlas conversion information (alpha)) provided from the encoding unit 513B. The multiplexing unit 517 can also acquire the encoded data (RGB_view) (encoded data of the atlas image (RGB_view)) provided from the encoding unit 617A. The multiplexing unit 517 can also acquire the encoded data (RGB_view) (encoded data of the atlas conversion information (RGB_view)) provided from the encoding unit 617B. The multiplexing unit 517 can also acquire the encoded data (RGB_base) (encoded data of the atlas image (RGB_base)) provided from the encoding unit 618A. The multiplexing unit 517 can also acquire the encoded data (RGB_base) (encoded data of the atlas conversion information (RGB_base)) provided from the encoding unit 617B. Further, the multiplexing unit 517 can multiplex these acquired information pieces to generate a bitstream. Further, the multiplexing unit 517 can transmit the generated bitstream.

[0239] The downscaling processing unit 621 can downscale the alpha image supplied to the encoding apparatus 315 to generate a down-scaled alpha image. The downscaling processing unit 621 can supply the generated down-scaled alpha image to the non-zero region extraction unit 611.

[0240] The downscaling processing unit 622 can downscale the RGB_view image supplied to the encoding apparatus 315 to generate a down-scaled RGB_view image. The downscaling processing unit 622 can supply the generated down-scaled RGB_view image to the atlas transformation unit 615.

[0241] In the encoding apparatus 315 having such a configuration, the present technology described above in <3. Encoding / decoding using atlas transformation> can be applied. For example, the method 1-2 described above can be applied to this encoding apparatus 315. For example, the alpha image can be transformed to the same size as the RGB image.

[0242] Further, the method 1-2-1 described above can be applied to this encoding apparatus 315. For example, the RGB image can include an RGB_base image indicating a distribution of the RGB_base component among the RGB components and an RGB_view image indicating a distribution of the RGB_view component among the RGB components. Then, the atlas transformation unit 515 (the atlas transformation unit 615 and the atlas transformation unit 616) can perform the atlas transformation on the RGB_base image and the RGB_view image having different sizes independently of each other.

[0243] Further, the method 1-2-1-1 described above can be applied to this encoding apparatus 315. For example, the atlas transformation unit 616 can perform the atlas transformation on the RGB_base image to generate the atlas transformation information of the RGB_base component. The atlas transformation unit 615 can further perform the atlas transformation on the RGB_view image to generate the atlas transformation information of the RGB_view component.

[0244] Further, the method 1-2-1-2 described above can be applied to this encoding apparatus 315. For example, the mask image generation unit 514 (the mask image generation unit 613 and the mask image generation unit 614) can generate a mask image in which a non-zero region of the alpha image in which a plurality of layers that are continuous are synthesized for each of the RGB_base image and the RGB_view image. Then, the atlas transformation unit 616 can perform the atlas transformation on the RGB_base image using the mask image for the RGB_base image. The atlas transformation unit 615 can further perform the atlas transformation on the RGB_view image using the mask image for the RGB_view image.

[0245] Further, the above-described method 1-2-1-3 can be applied to the encoding apparatus 315. For example, the down-sampling processing unit 621 can down-sample the alpha image. Further, the down-sampling processing unit 622 can down-sample the RGB_view image. Further, the atlas conversion unit 512 can perform atlas conversion on the down-sampled alpha image. Further, the atlas conversion units 515 (the atlas conversion unit 615 and the atlas conversion unit 616) can perform atlas conversion on the RGB_base image and the down-sampled RGB_view image independently of each other.

[0246] With such a configuration, the encoding apparatus 315 can suppress an increase in transmission capacity required for transmission of 3D data using MPI as in NeX. In this case, it is also possible to perform processing up to atlas conversion at the time of packing the post-processed NeX MPI. In other words, the packing apparatus 314 and the encoding apparatus 315 can be integrated.

[0247] <Flow of encoding processing>

[0248] An example of a flow of encoding processing performed by the encoding apparatus 315 will be described with reference to the flowchart illustrated in FIG. 17. Figure 26

[0249] When the encoding processing starts, in step S601, the down-sampling processing unit 621 down-samples the alpha image. In step S602, the non-zero region extraction unit 611 extracts a non-zero region from the down-sampled alpha image. In step S603, the atlas conversion unit 512 performs atlas conversion on the alpha component to generate an atlas image (alpha) and atlas conversion information (alpha). In step S604, the encoding unit 513A encodes the atlas image of the alpha component to generate encoded data (alpha). In addition, the encoding unit 513B encodes the atlas conversion information of the alpha component to generate encoded data (alpha).

[0250] In step S605, the mask image generation unit 613 generates a down-sampled mask image using the non-zero regions of the main layer and the sub-layers of the down-sampled alpha image. In step S606, the down-sampling processing unit 622 down-samples the RGB_view image.

[0251] In step S607, the atlas conversion unit 615 performs atlas conversion on the RGB_view using the down-sampled mask image to generate an atlas image (RGB_view). In step S608, the encoding unit 617A encodes the atlas image of the RGB_view component to generate encoded data (RGB_view). In addition, the encoding unit 617B encodes the atlas conversion information of the RGB_view component to generate encoded data (RGB_view).​

[0252] In step S609, the non-zero region extraction unit 612 extracts a non-zero region from the alpha image. In step S610, the mask image generation unit 614 generates a mask image using the non-zero regions of the main layer and the sub-layers of the alpha image.

[0253] In step S611, the atlas transformation unit 616 performs atlas transformation on RGB_base using the mask image to generate an atlas image (RGB_base). In step S612, the encoding unit 618A encodes the atlas image of the RGB_base component to generate encoded data (RGB_base). In addition, the encoding unit 618B encodes the atlas transformation information of the RGB_base component to generate encoded data (RGB_base).

[0254] In step S613, the multiplexing unit 517 multiplexes the encoded data of the atlas image (alpha), the encoded data of the atlas transformation information (alpha), the encoded data of the atlas image (RGB_base), the encoded data of the atlas image (RGB_view), the encoded data of the atlas transformation information (RGB_base), and the encoded data of the atlas transformation information (RGB_view) to generate a bitstream.

[0255] When the processing of step S613 ends, the encoding processing ends, and the processing returns to Figure 19 .

[0256] In this way, in each processing, the present technology described above in <3. Encoding / Decoding using Atlas Transformation> can be applied. For example, the method 1-2 described above can be applied. In addition, the method 1-2-1 described above can be applied. Furthermore, the method 1-2-1-1 described above can be applied. Furthermore, the method 1-2-1-2 described above can be applied. Furthermore, the method 1-2-1-3 described above can be applied.

[0257] By performing each processing in this way, the encoding device 315 can suppress an increase in transmission capacity required for transmission of 3D data using MPI as in NeX.

[0258] <Decoding Device>

[0259] In addition, the method 1-2 described above in <3. Encoding / Decoding using Atlas Transformation> can be applied to the decoding device 316. In Figure 27 A main configuration example of the decoding device 316 in this case is shown in FIG. 34.

[0260] In this case, the decoding device 316 also basically has the same configuration as Figure 23 The configuration shown in the example is similar to the configuration. However, in Figure 27 In the example shown, decoding device 316 includes decoding units 652A, 652B, 654A, and 654B replacing decoding unit 554. Additionally, decoding device 316 includes atlas inverse transformation units 653 and 655 replacing atlas inverse transformation unit 555. Furthermore, decoding device 316 also includes amplification processing unit 651 and amplification processing unit 656. In this case, demultiplexing unit 551 can be used with... Figure 23 The bitstream is acquired and demultiplexed in a similar manner to the case shown. Demultiplexing unit 551 can supply the encoded data (encoded data (alpha)) of the generated atlas image (alpha) to decoding unit 552A. Demultiplexing unit 551 can supply the encoded data (encoded data (alpha)) of the generated atlas transformation information (alpha) to decoding unit 552B. Demultiplexing unit 551 can supply the encoded data (encoded data (RGB_base)) of the generated atlas image (RGB_base) to decoding unit 652A. Demultiplexing unit 551 can provide the encoded data (encoded data (RGB_base)) of the generated atlas transformation information (RGB_base) to decoding unit 652B. Demultiplexing unit 551 can supply the encoded data (encoded data (RGB_view)) of the generated atlas image (RGB_view) to decoding unit 654A. The demultiplexing unit 551 can supply the encoded data (encoded data (RGB_view)) of the generated atlas transformation information (RGB_view) to the decoding unit 654B. The decoding unit 552A can... Figure 23 The encoded data (alpha) is decoded in a similar manner to the case shown, and the resulting atlas image (alpha) is provided to the atlas inverse transform unit 553. The decoding unit 552B can be configured to... Figure 23 The encoded data (alpha) is decoded in a similar manner to the case shown, and the generated atlas transformation information (alpha) is provided to the atlas inverse transformation unit 553.

[0261] In this case, the atlas inverse conversion unit 553 can perform an atlas inverse conversion on the atlas image (alpha) to generate the reduced alpha image. That is, the atlas inverse conversion unit 553 can acquire the atlas image (alpha) supplied from the decoding unit 552A. Further, the atlas inverse conversion unit 553 can acquire the atlas conversion information (alpha) supplied from the decoding unit 552B. Further, the atlas inverse conversion unit 553 can perform an atlas inverse conversion on the atlas image (alpha) using the atlas conversion information (alpha) to generate (restore) the reduced alpha image. The atlas inverse conversion unit 553 can supply the reduced alpha image to the enlargement processing unit 651. The enlargement processing unit 651 performs processing relating to enlargement of the reduced alpha image. For example, the enlargement processing unit 651 can acquire the reduced alpha image supplied from the atlas inverse conversion unit 553. The enlargement processing unit 651 can enlarge the reduced alpha image to generate (restore) the alpha image. The enlargement processing unit 651 can supply the generated alpha image to the unpacking device 317.

[0262] The decoding unit 652A and the decoding unit 652B perform processing relating to decoding of the RGB_base component. For example, the decoding unit 652A can acquire the encoding data (encoding data (RGB_base)) of the atlas image (RGB_base) supplied from the demultiplexing unit 551. The decoding unit 652A can decode the encoding data (RGB_base) by a 2D decoding method (e.g., JPEG, MPEG, or the like) to generate the atlas image (RGB_base). The decoding unit 652A can supply the atlas image (RGB_base) to the atlas inverse conversion unit 653. The decoding unit 652B can acquire the encoding data (encoding data (RGB_base)) of the atlas conversion information (RGB_base) supplied from the demultiplexing unit 551. The decoding unit 652B can decode the encoding data (RGB_base) to generate the atlas conversion information (RGB_base). The decoding method can be any method. For example, the decoding unit 652B can decode the encoding data (RGB_base) in accordance with the MIV / V3C standard. Further, the decoding unit 652B can supply the atlas conversion information (RGB_base) to the atlas inverse conversion unit 653. Note that the decoding unit 652A and the decoding unit 652B are also referred to as the decoding unit 652.

[0263] The atlas inverse transform unit 653 performs processing related to the atlas inverse transform of the RGB_base component. For example, the atlas inverse transform unit 653 can acquire the atlas image (RGB_base) supplied from the decoding unit 652A. The atlas inverse transform unit 653 can also acquire the atlas transform information (RGB_base) supplied from the decoding unit 652B. Further, the atlas inverse transform unit 653 can perform the atlas inverse transform on the atlas image (RGB_base) using the atlas transform information (RGB_base) to generate (restore) the RGB_base image. The atlas inverse transform unit 653 can supply the generated RGB_base image to the depacketizer 317.

[0264] The decoding units 654A and 654B perform processing related to the decoding of the RGB_view component. For example, the decoding unit 654A can acquire the encoded data of the atlas image (RGB_view) (encoded data (RGB_view)) supplied from the demultiplexing unit 551. The decoding unit 654A can decode the encoded data (RGB_view) by a 2D decoding method (e.g., JPEG, MPEG, etc.) to generate the atlas image (RGB_view). The decoding unit 654A can supply the atlas image (RGB_view) to the atlas inverse transform unit 655. The decoding unit 654B can acquire the encoded data of the atlas transform information (RGB_view) (encoded data (RGB_view)) supplied from the demultiplexing unit 551. The decoding unit 654B can decode the encoded data (RGB_view) to generate the atlas transform information (RGB_view). The decoding method can be any method. For example, the decoding unit 654B can decode the encoded data (RGB_view) according to the MIV / V3C standard. Further, the decoding unit 654B can supply the atlas transform information (RGB_view) to the atlas inverse transform unit 655. Note that the decoding units 654A and 654B are also referred to as the decoding unit 654.

[0265] For example, the atlas inverse transform unit 655 can acquire the atlas image (RGB_view) supplied from the decoding unit 654A. The atlas inverse transform unit 655 can also acquire the atlas transform information (RGB_view) supplied from the decoding unit 654B. Further, the atlas inverse transform unit 655 can perform the atlas inverse transform on the atlas image (RGB_view) using the atlas transform information (RGB_view) to generate (restore) the downsized RGB_view image. The atlas inverse transform unit 655 can supply the generated downsized RGB_view image to the upscaling processing unit 656.

[0266] The up-scaling processing unit 656 can acquire the down-scaled RGB_view image supplied from the atlas inverse conversion unit 655. The up-scaling processing unit 656 can up-scale the down-scaled RGB_view image to generate (restore) the RGB_view image. The up-scaling processing unit 656 can supply the generated RGB_view image to the depacketizing device 317.

[0267] In the decoding device 316 having such a configuration, the present technology described above in <3. Encoding / decoding using atlas transformation> can be applied. For example, the method 1-2 described above can be applied to this decoding device 316. That is, the alpha image can be transformed to the same size as the RGB image to generate a mask.

[0268] Further, the method 1-2-1 described above can be applied to this decoding device 316. For example, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Further, the RGB_base image and the RGB_view image can have different sizes. Further, the decoding unit 652A and the decoding unit 654A can decode the encoded data of the atlas image of the RGB_base component and the encoded data of the atlas image of the RGB_view component, respectively. Then, the atlas inverse conversion unit 653 and the atlas inverse conversion unit 655 can perform the atlas inverse transformation on the atlas image of the RGB_base component and the atlas image of the RGB_view component independently of each other.

[0269] Further, the method 1-2-1-1 described above can be applied to this decoding device 316. For example, the atlas inverse conversion unit 653 can perform the atlas inverse transformation on the atlas image of the RGB_base component using the atlas transformation information of the RGB_base component. Further, the atlas inverse conversion unit 655 can perform the atlas inverse transformation on the atlas image of the RGB_view component using the atlas transformation information of the RGB_view component. In addition, the atlas transformation information can be information indicating a content of the atlas transformation.

[0270] Further, the method 1-2-1-3 described above can be applied to this decoding device 316. For example, the up-scaling processing unit 651 can up-scale the down-scaled alpha image. Further, the up-scaling processing unit 656 can up-scale the down-scaled RGB_view image.

[0271] With this configuration, the decoding device 316 can suppress the increase in transmission capacity required for transmitting 3D data using MPI, such as in NeX. In this case, inverse atlas transformation can also be performed during unpacking of the alpha image, RGB_base image, and RGB_view image. In other words, the decoding device 316 and the unpacking device 317 can be integrated.

[0272] <Decoding Process>

[0273] Reference Figure 28 The flowchart shown illustrates an example of the decoding process performed by the decoding device 316.

[0274] When the decoding process begins, in step S651, the demultiplexing unit 551 demultiplexes the bitstream to generate encoded data of the atlas image (alpha), encoded data of the atlas transformation information (alpha), encoded data of the atlas image (RGB_base), encoded data of the atlas transformation information (RGB_base), encoded data of the atlas image (RGB_view), and encoded data of the atlas transformation information (RGB_view).

[0275] In step S652, decoding unit 552A decodes the encoded data of the atlas image of the alpha component to generate an atlas image (alpha). Additionally, decoding unit 552B decodes the encoded data of the atlas transformation information of the alpha component to generate atlas transformation information (alpha). In step S653, inverse atlas transformation unit 553 performs an inverse atlas transformation on the atlas image (alpha) using the atlas transformation information (alpha) to generate a reduced alpha image. In step S654, magnification processing unit 651 magnifies the reduced alpha image.

[0276] In step S655, decoding unit 652A decodes the encoded data of the atlas image of the RGB_base component to generate an atlas image (RGB_base). Additionally, decoding unit 652B decodes the encoded data of the atlas transformation information of the RGB_base component to generate atlas transformation information (RGB_base). In step S656, inverse atlas transformation unit 653 performs an inverse atlas transformation on the atlas image (RGB_base) using the atlas transformation information (RGB_base) to generate an RGB_base image.

[0277] In step S657, the decoding unit 654A decodes the encoded data of the atlas image of the RGB_view component to generate the atlas image (RGB_view). In addition, the decoding unit 654B decodes the encoded data of the atlas transform information of the RGB_view component to generate the atlas transform information (RGB_view). In step S658, the atlas inverse transform unit 655 performs atlas inverse transform on the atlas image (RGB_view) using the atlas transform information (RGB_view) to generate the reduced RGB_view image. In step S659, the upscaling processing unit 656 upscales the reduced RGB_view image.

[0278] When the processing of step S659 ends, the decoding processing ends, and the processing returns to Figure 20 .

[0279] In this way, in each processing, the present technology described above in <3. Encoding / decoding using atlas transform> can be applied. For example, the method 1-2 described above can be applied. In addition, the method 1-2-1 described above can be applied. Furthermore, the method 1-2-1-1 described above can be applied. Furthermore, the method 1-2-1-2 described above can be applied. Furthermore, the method 1-2-1-3 described above can be applied.

[0280] By performing each processing in this way, the decoding device 316 can suppress an increase in transmission capacity required for transmission of 3D data using MPI like in NeX.

[0281] <Encoding device>

[0282] In addition, the method 1-2-2 described above in <3. Encoding / decoding using atlas transform> can be applied to the encoding device 315. In this case, the encoding device 315 can perform the processing of step S651, and then perform the processing of step S652. Figure 29 A main configuration example of the encoding device 315 in this case is shown in FIG. 17.

[0283] As shown in Figure 29 , in this case, the encoding device 315 also basically has a configuration similar to that of the example shown in Figure 25 . However, in the case of the example shown in Figure 29 , the atlas transform unit 615 does not generate the atlas transform information (RGB_view) (or does not supply the atlas transform information (RGB_view) to the encoding unit 617B).

[0284] That is, as in Figure 29 As shown in the example, encoding unit 617B can be omitted. In this case, multiplexing unit 517 can acquire encoded data (alpha), atlas transformation information (alpha), encoded data (RGB_view), encoded data (RGB_base), and atlas transformation information (RGB_base), and multiplex the encoded data to generate a bitstream.

[0285] In the encoding device 315 with such a configuration, the technique described above in <3. Encoding / Decoding using atlas transformation> can be applied. For example, method 1-2-2 described above can be applied to this encoding device 315. For example, an RGB image may include an RGB_base image indicating the distribution of the RGB_base component in the RGB components and an RGB_view image indicating the distribution of the RGB_view component in the RGB components. Furthermore, the RGB_base image and the RGB_view image may have different sizes. Furthermore, the atlas transformation unit 515 (atlas transformation unit 615 and atlas transformation unit 616) can perform atlas transformation on the RGB_base image and the RGB_view image to generate atlas transformation information for the RGB_base component or the RGB_view component. Note that the atlas transformation information may be information indicating the content of the atlas transformation.

[0286] Furthermore, the method 1-2-2-1 described above can be applied to the encoding device 315. For example, the atlas transformation information to be transferred, reduction instructions, reduction ratios, etc., can be defined in a file.

[0287] With this configuration, the encoding device 315 can suppress the increase in transmission capacity required for transmitting 3D data using MPI such as NeX. In this case, processing up to atlas transformation can also be performed when packing post-processed NeX MPI. In other words, the packing device 314 and the encoding device 315 can be integrated.

[0288] In this case, the encoding process is basically the same as Figure 26 The process shown in the example is similar. However, in step S608, only the atlas image (RGB_view) needs to be encoded by the encoding unit 617A, and the encoding of the atlas transformation information (RGB_view) is omitted (skipped). Then, in step S613, the multiplexing unit 517 only needs to multiplex the encoded data of the atlas image (alpha), the encoded data of the atlas transformation information (alpha), the encoded data of the atlas image (RGB_base), the encoded data of the atlas image (RGB_view), and the encoded data of the atlas transformation information (RGB_base) to generate a bitstream.

[0289] <Decoding device>

[0290] Further, the method 1-2-2 described above in <3. Encoding / decoding using atlas transformation> can be applied to the decoding device 316. In this case, the decoding device 316 is configured as shown in Fig. 34. Figure 30 A main configuration example of the decoding device 316 in this case is shown in Fig. 35.

[0291] In this case, the decoding device 316 basically has a configuration similar to that of the example shown in Fig. 34. However, in the case of the example shown in Fig. 35, the decoding device 316 further includes a downscaling processing unit 681. Figure 27 Figure 30

[0292] In the case of this example, the demultiplexing unit 551 demultiplexes the bitstream and supplies the generated atlas transformation information (RGB_base) to the decoding unit 652B and the decoding unit 654B.

[0293] In this case, similarly to the decoding unit 652B, the decoding unit 654B acquires the encoded data of the atlas transformation information (RGB_base) from the demultiplexing unit 551 and decodes the encoded data. Then, the decoding unit 654B supplies the generated atlas transformation information (RGB_base) by decoding to the downscaling processing unit 681. Note that this decoding unit 654B can be omitted, and the atlas transformation information (RGB_base) can be supplied from the decoding unit 652B to the downscaling processing unit 681. The downscaling processing unit 681 can acquire the atlas transformation information (RGB_base) supplied from the decoding unit 654B (or the decoding unit 652B). Further, the downscaling processing unit 681 can downscale (downsize) the atlas transformation information (RGB_base) and supply the downscaled atlas transformation information (RGB_base) to the atlas inverse transformation unit 655.

[0294] The atlas inverse transformation unit 655 can perform atlas inverse transformation on the atlas image (RGB_view) using the downscaled atlas transformation information (RGB_base) to generate a downscaled RGB_view image.

[0295] ​​In the decoding apparatus 316 having such a configuration, the present technology described above in <3. Encoding / decoding using atlas transform> can be applied. For example, the method 1-2-2 described above can be applied to this decoding apparatus 316. For example, the RGB image can include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components. Furthermore, the RGB_base image and the RGB_view image can have different sizes. Furthermore, the decoding unit 652A can decode the encoded data of the atlas image of the RGB_base component, and the decoding unit 654A can decode the encoded data of the atlas image of the RGB_view component. Furthermore, the atlas inverse transform unit 653 can perform atlas inverse transform on the atlas image of the RGB_base component using the atlas transform information of the RGB_base component, and the atlas inverse transform unit 655 can perform atlas inverse transform on the atlas image of the RGB_view component using the atlas transform information of the RGB_view component. Furthermore, the down-sampling processing unit 681 can down-sample the atlas transform information of the RGB_base component or the RGB_view component.

[0296] Furthermore, the method 1-2-2-1 described above can be applied to this decoding apparatus 316. For example, the atlas transform information to be transferred, the down-sampling instruction, the down-sampling ratio, and the like can be set in accordance with the definition of the file.

[0297] With such a configuration, the decoding apparatus 316 can suppress an increase in transmission capacity required for transmission of 3D data using MPI as in NeX. In this case, it is also possible to perform atlas inverse transform at the time of unpacking the alpha image, the RGB_base image, and the RGB_view image. In other words, it is possible to integrate the decoding apparatus 316 and the unpacking apparatus 317.

[0298] <Flow of decoding processing>

[0299] An example of the flow of the decoding processing performed by the decoding apparatus 316 will be described with reference to the flowchart illustrated in FIG. 68. Figure 31

[0300] When the decoding processing starts, in step S681, the demultiplexing unit 551 demultiplexes the bitstream.

[0301] ​In step S682, the decoding unit 552A decodes the encoded data of the atlas image of the alpha component to generate the atlas image (alpha). In addition, the decoding unit 552B decodes the encoded data of the atlas transform information of the alpha component to generate the atlas transform information (alpha). In step S683, the atlas inverse transform unit 553 performs atlas inverse transform on the atlas image (alpha) using the atlas transform information (alpha) to generate the down-sampled alpha image. In step S684, the up-sampling processing unit 651 up-samples the down-sampled alpha image.

[0302] In step S685, the decoding unit 652A decodes the encoded data of the atlas image of the RGB_base component to generate the atlas image (RGB_base). In addition, the decoding unit 652B decodes the encoded data of the atlas transform information of the RGB_base component to generate the atlas transform information (RGB_base). In step S686, the atlas inverse transform unit 653 performs atlas inverse transform on the atlas image (RGB_base) using the atlas transform information (RGB_base) to generate the RGB_base image.

[0303] In step S687, the decoding unit 654A decodes the encoded data of the atlas image of the RGB_view component to generate the atlas image (RGB_view). In addition, the decoding unit 654B decodes the encoded data of the atlas transform information of the RGB_base component to generate the atlas transform information (RGB_base). In step S688, the down-sampling processing unit 681 down-samples the atlas transform information (RGB_base) to generate the atlas transform information for the RGB_view component (down-sampled atlas transform information (RGB_base)).

[0304] In step S689, the atlas inverse transform unit 655 performs atlas inverse transform on the atlas image (RGB_view) using the atlas transform information for the RGB_view component (down-sampled atlas transform information (RGB_base)) to generate the down-sampled RGB_view image. In step S690, the up-sampling processing unit 656 up-samples the down-sampled RGB_view image.

[0305] When the processing of step S690 ends, the decoding processing ends, and the processing returns to Figure 20 .

[0306] In this way, in each processing, the present technology described above in <3. Encoding / decoding using atlas transform> can be applied. For example, the method 1-2-2 described above can be applied. Further, the method 1-2-2-1 described above can be applied.

[0307] By performing each process in this way, the decoding device 316 can suppress an increase in transmission capacity required for transmission of 3D data using MPI as in NeX.

[0308] <5. Supplementary notes>

[0309] <Computer>

[0310] The series of processes described above can be executed by hardware or can be executed by software. When the series of processes is executed by software, a program configuring the software is installed in a computer. In this context, the computer includes a computer incorporated into dedicated hardware, a computer such as, for example, a general-purpose personal computer capable of installing various programs therein to execute various functions, and the like.

[0311] Figure 32 is a block diagram showing a hardware configuration example of a computer that executes the series of processes described above by a program.

[0312] In Figure 32 In the computer 900 shown in FIG. 9, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Acess Memory) 903 are connected to each other via a bus 904.

[0313] An input / output interface 910 is also connected to the bus 904. An input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected to the input / output interface 910.

[0314] The input unit 911 is constituted by, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like. The output unit 912 is constituted by, for example, a display, a speaker, an output terminal, and the like. The storage unit 913 is constituted by, for example, a hard disk, a RAM disk, a nonvolatile memory, and the like. The communication unit 914 is constituted by, for example, a network interface. The drive 915 drives a removable medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0315] In the computer configured as described above, the CPU 901 loads a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904, and executes the program to execute the series of processes described above, for example. Data required for the CPU 901 to execute various types of processing and the like is also appropriately stored in the RAM 903.

[0316] For example, a program executed by a computer can be applied by being recorded on a removable medium 921 or the like as a package medium. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by loading the removable medium 921 into the drive 915.

[0317] Further, the program can also be provided via wired or wireless transmission medium such as a local area network, the Internet, and digital satellite broadcasting. In this case, the program can be received by the communication unit 914 to be installed in the storage unit 913.

[0318] Alternatively, the program can be pre-installed in the ROM 902 or the storage unit 913.

[0319] <Applications of the Present Technology>

[0320] The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic apparatuses.

[0321] Further, for example, the present technology can be implemented as a partial configuration of an apparatus such as a processor (for example, a video processor) as a system LSI (Large Scale Integration) or the like, a module (for example, a video module) using a plurality of processors or the like, a unit (for example, a video unit) using a plurality of modules or the like, or a device (for example, a video device) obtained by adding other functions to the unit.

[0322] Further, for example, the present technology can also be applied to a network system configured by a plurality of apparatuses. For example, the present technology can be implemented as cloud computing in which a plurality of apparatuses share and cooperate for processing via a network. For example, the present technology can be implemented in, for example, a cloud service that provides a service related to an image (moving image) for any terminal such as a computer, an AV (Audio Visual) apparatus, a mobile information processing terminal, and an IoT (Internet of Things) device.

[0323] Note that, in the present specification, a system refers to a collection of a plurality of constituent elements (apparatuses, modules (parts), and the like), regardless of whether all the constituent elements are within the same housing. Therefore, both a plurality of apparatuses housed in different housings and connected via a network and a single apparatus in which a plurality of modules are housed in a single housing are systems.

[0324] <Fields / Uses to Which the Present Technology Can be Applied>

[0325] The system, apparatus, processing unit, and the like to which the present technology is applied can be used in any field such as traffic, medical care, crime prevention, agriculture, animal husbandry, mining, beauty care, factories, household electrical appliances, weather, natural monitoring, and the like. In addition, the use thereof is arbitrary.

[0326] <Other>

[0327] Note that the "flag" used in this specification is information for identifying a plurality of states, and includes not only information used when identifying two states, that is, true (1) and false (0), but also information with which three or more states can be identified. Therefore, the value that the "flag" can take can be, for example, binary such as 1 / 0 or ternary or more. In other words, the number of bits configuring the "flag" is arbitrary, and can be one bit or a plurality of bits. Furthermore, since the identification information (including the flag) can take not only a form in which the identification information is incorporated in a bit stream but also a form in which difference information with respect to certain information that becomes a reference is incorporated in a bit stream, in this specification, the "flag" or the "identification information" includes not only the information but also the difference information with respect to the information that becomes the reference.

[0328] In addition, various types of information (metadata and the like) on coded data (bit stream) can be transmitted or recorded in any form as long as the information is associated with the coded data. The term "associated" herein means that, when one of data is processed, the other data can be used (linked). In other words, data associated with each other can be compiled as one piece of data, or can each be regarded as a separate piece of data. For example, information associated with coded data (image) can be transmitted on a transmission path different from that of the coded data (image). In addition, for example, information associated with coded data (image) can be recorded on a recording medium different from that of the coded data (image) (or in a different recording area in the same recording medium). Note that the "association" can be applied not only to the entire data but also to a part of the data. For example, an image and information corresponding to the image can be associated with each other in any unit including a plurality of frames, one frame, a part of a frame, and the like.

[0329] Note that in this specification, the terms "synthesis", "multiplexing", "adding", "integrating", "incorporating", "storing", "putting", "introducing", "inserting", and the like mean compiling a plurality of objects into one, such as compiling coded data and metadata into one piece of data, and refer to one method of the above-described "association".

[0330] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present technology.

[0331] For example, a configuration described as one device (or processing unit) can be divided and configured as a plurality of devices (or processing units). Conversely, a configuration described above as a plurality of devices (or processing units) can be collectively configured as one device (or processing unit). Furthermore, a configuration other than the above-described configurations can of course be added to the configuration of the corresponding device (or the corresponding processing unit). In addition, a part of the configuration of a certain device (or processing unit) can be incorporated in the configuration of another device (or another processing unit) as long as the configuration and operation of the entire system are substantially the same.

[0332] Furthermore, for example, the above-described program can be executed in any device. In this case, the device only needs to have the necessary functions (functional blocks and the like) to be able to acquire the necessary information.

[0333] Furthermore, for example, the respective steps of a single flowchart can be executed by a single device, or can be shared by a plurality of devices to be executed. In addition, when a plurality of processes are included in a single step, the plurality of processes can be executed by a single device, or can be shared by a plurality of devices to be executed. In other words, the plurality of processes included in a single step can be executed as the processes of a plurality of steps. Conversely, the processes described as a plurality of steps can be executed as a single step as a whole.

[0334] Furthermore, for example, with regard to a program executed by a computer, the processes of the steps of the program can be executed in a time series in the order described in this specification, or can be executed individually in parallel or at necessary timing such as when called. In other words, the processes of the respective steps can be executed in an order different from the above-described order as long as no contradiction arises. In addition, the processes of the steps of the program can be executed in parallel with the processes of other programs, or can be executed in combination with the processes of other programs.

[0335] Furthermore, for example, a plurality of technologies related to the present technology can each be executed independently and individually as long as no contradiction arises. Of course, a plurality of any of the present technology can be executed in combination. For example, a part or all of the present technology described in any one of the embodiments can be executed in combination with a part or all of the present technology described in other embodiments. Furthermore, a part or all of any of the above-described present technology can be executed in combination with other technologies not described above.

[0336] Note that the present technology can also adopt the following configurations.

[0337] (1) An information processing apparatus comprising:

[0338] a first atlas transformation unit that performs atlas transformation on an alpha image of 3D data and generates an atlas image of an alpha component;

[0339] a first encoding unit that encodes atlas images of the alpha component;

[0340] a second atlas conversion unit that performs atlas conversion on RGB images of the 3D data and generates atlas images of RGB components; and

[0341] a second encoding unit that encodes the atlas images of the RGB components.

[0342] (2) The information processing apparatus according to (1), further comprising:

[0343] a mask image generation unit that generates a mask image obtained by synthesizing non-zero regions of a plurality of layers of alpha images in succession,

[0344] the second atlas conversion unit is configured to perform the atlas conversion on the RGB images using the mask image.

[0345] (3) The information processing apparatus according to (2), in which

[0346] the RGB images include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components,

[0347] the second atlas conversion unit is configured to perform atlas conversion on the RGB_base image and the RGB_view image to generate an atlas image of each of the RGB_base image and the RGB_view image, and generate atlas conversion information common to the RGB_base component and the RGB_view component, and

[0348] the atlas conversion information is information indicating a content of atlas conversion.

[0349] (4) The information processing apparatus according to (1), in which

[0350] the RGB images include an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components, and

[0351] the second atlas conversion unit is configured to perform atlas conversion on the RGB_base image and the RGB_view image independently of each other with different sizes from each other.

[0352] (5) The information processing apparatus according to (4), wherein

[0353] the second atlas transformation unit is configured to perform atlas transformation on the RGB_base image to generate atlas transformation information of the RGB_base component, and perform atlas transformation on the RGB_view image to generate atlas transformation information of the RGB_view component, and

[0354] the atlas transformation information is information indicating contents of atlas transformation.

[0355] (6) The information processing apparatus according to (4) or (5), further comprising:

[0356] a mask image generation unit that generates, for each of the RGB_base image and the RGB_view image, a mask image obtained by synthesizing non-zero regions of a plurality of layers of alpha images in succession, wherein

[0357] the second atlas transformation unit is configured to perform atlas transformation on the RGB_base image using the mask image for the RGB_base image, and perform atlas transformation on the RGB_view image using the mask image for the RGB_view image.

[0358] (7) The information processing apparatus according to any one of (4) to (6), further comprising:

[0359] a down-sampling processing unit that down-samples the alpha images and the RGB_view image, wherein

[0360] the first atlas transformation unit is configured to perform atlas transformation on the down-sampled alpha images, and

[0361] the second atlas transformation unit is configured to perform atlas transformation on the RGB_base image and the down-sampled RGB_view image independently of each other.

[0362] (8) The information processing apparatus according to (1), wherein

[0363] the RGB image includes an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components,

[0364] the RGB_base image and the RGB_view image have mutually different sizes,

[0365] the second atlas transformation unit is configured to perform atlas transformation on the RGB_base image and the RGB_view image to generate atlas transformation information of the RGB_base component or the RGB_view component, and

[0366] The atlas transformation information is information indicating the content of the atlas transformation.

[0367] (9) The information processing apparatus according to any one of (1) to (8), in which

[0368] The RGB image includes an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components,

[0369] The second atlas transformation unit is configured to perform atlas transformation on the RGB_base image and the RGB_view image to generate an atlas image of each of the RGB_base image and the RGB_view image,

[0370] The information processing apparatus further includes:

[0371] a multiplexing unit that multiplexes the encoded data of the atlas image of the alpha component, the encoded data of the atlas image of the RGB_base component, and the encoded data of the atlas image of the RGB_view component using identification information to generate a bitstream, and

[0372] The identification information is information for identifying the RGB_base component and the RGB_view component.

[0373] (10) An information processing method including:

[0374] performing atlas transformation on an alpha image of 3D data and generating an atlas image of an alpha component;

[0375] encoding the atlas image of the alpha component;

[0376] performing atlas transformation on an RGB image of the 3D data and generating an atlas image of an RGB component; and

[0377] encoding the atlas image of the RGB component, in which

[0378] The alpha image is distribution information of the alpha component indicating the presence of an object in a single layer among a layer group representing a three-dimensional space,

[0379] the RGB image is information indicating a distribution of the RGB components representing color information of the object in a plurality of consecutive layers in the layer group,

[0380] for the three-dimensional space, the 3D data includes the alpha image for each of the single layers and the RGB image for each of the plurality of consecutive layers, and

[0381] the atlas transform is a process for arranging, in the atlas image, a non-zero region included in the single alpha image or a non-zero region included in the RGB image corresponding to a non-zero region included in one or more alpha images.

[0382] (11) An information processing apparatus comprising:

[0383] a first decoding unit that decodes encoded data and generates an atlas image of an alpha component;

[0384] a first atlas inverse transform unit that performs an atlas inverse transform on the atlas image of the alpha component and generates an alpha image of 3D data;

[0385] a second decoding unit that decodes encoded data and generates an atlas image of an RGB component; and

[0386] a second atlas inverse transform unit that performs an atlas inverse transform on the atlas image of the RGB component and generates an RGB image of the 3D data, wherein

[0387] the alpha image is distribution information of the alpha component indicating presence of an object in a single layer in a layer group representing a three-dimensional space,

[0388] the RGB image is information indicating a distribution of the RGB components representing color information of the object in a plurality of consecutive layers in the layer group,

[0389] for the three-dimensional space, the 3D data includes the alpha image for each of the single layers and the RGB image for each of the plurality of consecutive layers, and

[0390] the atlas inverse transform is an inverse process of an atlas transform for arranging, in the atlas image, a non-zero region included in the single alpha image or a non-zero region included in the RGB image corresponding to a non-zero region included in one or more alpha images.

[0391] (12) The information processing apparatus according to (11), wherein

[0392] The RGB image includes an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components,

[0393] The second decoding unit is configured to decode the encoded data and generate an atlas image of the RGB_base component and an atlas image of the RGB_view component,

[0394] The second atlas inverse transform unit is configured to perform atlas inverse transform on the atlas image of the RGB_base component and the atlas image of the RGB_view component using atlas transform information common to the RGB_base component and the RGB_view component, and

[0395] The atlas transform information is information indicating contents of atlas transform.

[0396] (13) The information processing apparatus according to (11), in which

[0397] The RGB image includes an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components,

[0398] The RGB_base image and the RGB_view image have mutually different sizes,

[0399] The second decoding unit is configured to decode each of encoded data of the atlas image of the RGB_base component and encoded data of the atlas image of the RGB_view component, and

[0400] The second atlas inverse transform unit is configured to perform atlas inverse transform on the atlas image of the RGB_base component and the atlas image of the RGB_view component independently of each other.

[0401] (14) The information processing apparatus according to (13), in which

[0402] The second atlas inverse transform unit is configured to perform atlas inverse transform on the atlas image of the RGB_base component using atlas transform information of the RGB_base component and perform atlas inverse transform on the atlas image of the RGB_view component using atlas transform information of the RGB_view component, and

[0403] The atlas transform information is information indicating contents of atlas transform.

[0404] (15) The information processing apparatus according to (13) or (14), further comprising:

[0405] an upscaling processing unit that upscales the alpha image and the RGB_view image.

[0406] (16) The information processing apparatus according to (11), in which

[0407] the RGB image includes an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components,

[0408] the RGB_base image and the RGB_view image have different sizes,

[0409] the second decoding unit is configured to decode each of encoded data of an atlas image of the RGB_base component and encoded data of an atlas image of the RGB_view component, and

[0410] the second atlas inverse transform unit is configured to perform the atlas inverse transform on the atlas image of the RGB_base component and the atlas image of the RGB_view component using atlas transform information of the RGB_base component or the RGB_view component.

[0411] (17) The information processing apparatus according to (16), further comprising:

[0412] a downsizing processing unit that downsizes the atlas transform information of the RGB_base component or the RGB_view component.

[0413] (18) The information processing apparatus according to any one of (11) to (17), in which

[0414] the RGB image includes an RGB_base image indicating a distribution of an RGB_base component among the RGB components and an RGB_view image indicating a distribution of an RGB_view component among the RGB components, and

[0415] the information processing apparatus further comprises:

[0416] a demultiplexing unit that demultiplexes a bitstream to generate encoded data of an atlas image of the alpha component, encoded data of an atlas image of the RGB_base component, and encoded data of an atlas image of the RGB_view component.

[0417] (19) The information processing apparatus according to (18), in which

[0418] The demultiplexing unit demultiplexes the bitstream using identification information for identifying the RGB_base component and the RGB_view component.

[0419] (20) An information processing method comprising:

[0420] decoding encoded data and generating a sprite image of an alpha component;

[0421] performing a sprite inverse transform on the sprite image of the alpha component and generating an alpha image of 3D data;

[0422] decoding encoded data and generating a sprite image of an RGB component; and

[0423] performing a sprite inverse transform on the sprite image of the RGB component and generating an RGB image of the 3D data, wherein

[0424] the alpha image is distribution information of the alpha component indicating presence of an object in a single layer of a layer group representing a three-dimensional space,

[0425] the RGB image is information indicating distribution of the RGB component representing color information of the object in a plurality of consecutive layers of the layer group,

[0426] for the three-dimensional space, the 3D data includes the alpha image for each of the single layer and the RGB image for each of the plurality of consecutive layers, and

[0427] the sprite inverse transform is an inverse process of a sprite transform for arranging, in the sprite image, a non-zero region included in the single alpha image or a non-zero region included in the RGB image corresponding to a non-zero region included in one or more alpha images.

[0428] List of Reference Signs

[0429] 300 information processing system

[0430] 311 NeX learning device

[0431] 312 NeX MPI generation device

[0432] 313 post-processing device

[0433] 314 packing device

[0434] 315 encoding device

[0435] 316 decoding device

[0436] 317 unpacking device

[0437] 318 rendering device

[0438] 319 display pre-processing device

[0439] 320 display device

[0440] 511 non-zero region extraction unit

[0441] 512 atlas transformation unit

[0442] 513 encoding unit

[0443] 514 mask image generation unit

[0444] 515 atlas transformation unit

[0445] 516 encoding unit

[0446] 517 multiplexing unit

[0447] 551 demultiplexing unit

[0448] 552 decoding unit

[0449] 553 atlas inverse transformation unit

[0450] 554 decoding unit

[0451] 555 atlas inverse transformation unit

[0452] 611 non-zero region extraction unit

[0453] 612 non-zero region extraction unit

[0454] 613 mask image generation unit

[0455] 614 mask image generation unit

[0456] 615 atlas transformation unit

[0457] 616 atlas transformation unit

[0458] 617 encoding unit

[0459] 618 encoding unit

[0460] 651 upscaling processing unit

[0461] 652 decoding unit

[0462] 653 atlas inverse transformation unit

[0463] 654 decoding unit

[0464] 655 atlas inverse transform unit

[0465] 656 upscaling processing unit

[0466] 681 downscaling processing unit

[0467] 900 computer< / nex> < / nerf>

Claims

1. An information processing apparatus, comprising: The first atlas transformation unit performs atlas transformation on the alpha image of the 3D data and generates an atlas image of the alpha component; A first encoding unit encodes the atlas image of the alpha component; The second atlas transformation unit performs atlas transformation on the RGB image of the 3D data and generates an atlas image of the RGB components; as well as The second encoding unit encodes the atlas image of the RGB components.

2. The information processing apparatus according to claim 1, further comprising: A mask image generation unit generates a mask image by synthesizing non-zero regions of alpha images from multiple consecutive layers, wherein... The second atlas transformation unit is configured to perform atlas transformation on the RGB image using the mask image.

3. The information processing apparatus according to claim 2, wherein, The RGB image includes an RGB_base image indicating the distribution of the RGB_base components in the RGB components and an RGB_view image indicating the distribution of the RGB_view components in the RGB components. The second atlas transformation unit is configured to perform atlas transformation on the RGB_base image and the RGB_view image to generate an atlas image for each of the RGB_base image and the RGB_view image, and to generate atlas transformation information shared by the RGB_base component and the RGB_view component. The atlas transformation information is information indicating the content of the atlas transformation.

4. The information processing apparatus according to claim 1, wherein, The RGB image includes an RGB_base image indicating the distribution of the RGB_base components in the RGB components and an RGB_view image indicating the distribution of the RGB_view components in the RGB components, and The second atlas transformation unit is configured to perform atlas transformations independently on the RGB_base image and the RGB_view image, which have different sizes.

5. The information processing apparatus according to claim 4, wherein, The second atlas transformation unit is configured to perform atlas transformation on the RGB_base image to generate atlas transformation information for the RGB_base components, and to perform atlas transformation on the RGB_view image to generate atlas transformation information for the RGB_view components. The atlas transformation information is information indicating the content of the atlas transformation.

6. The information processing apparatus according to claim 4, further comprising: A mask image generation unit generates a mask image for each of the RGB_base image and the RGB_view image by synthesizing non-zero regions of an alpha image from multiple consecutive layers. The second atlas transformation unit is configured to perform atlas transformation on the RGB_base image using a mask image for the RGB_base image, and to perform atlas transformation on the RGB_view image using a mask image for the RGB_view image.

7. The information processing apparatus according to claim 4, further comprising: A scaling down unit reduces the size of the alpha image and the RGB_view image, wherein... The first atlas transformation unit is configured to perform atlas transformation on the scaled-down alpha image, and The second atlas transformation unit is configured to perform atlas transformations independently on the RGB_base image and the scaled-down RGB_view image.

8. The information processing apparatus according to claim 1, wherein, The RGB image includes an RGB_base image indicating the distribution of the RGB_base components in the RGB components and an RGB_view image indicating the distribution of the RGB_view components in the RGB components. The RGB_base image and the RGB_view image have different sizes. The second atlas transformation unit is configured to perform atlas transformation on the RGB_base image and the RGB_view image to generate atlas transformation information for the RGB_base component or the RGB_view component, and The atlas transformation information is information indicating the content of the atlas transformation.

9. The information processing apparatus according to claim 1, wherein, The RGB image includes an RGB_base image indicating the distribution of the RGB_base components in the RGB components and an RGB_view image indicating the distribution of the RGB_view components in the RGB components. The second atlas transformation unit is configured to perform atlas transformation on the RGB_base image and the RGB_view image to generate an atlas image for each of the RGB_base image and the RGB_view image. The information processing device further includes: A multiplexing unit uses identification information to multiplex the encoded data of the atlas image of the alpha component, the encoded data of the atlas image of the RGB_base component, and the encoded data of the atlas image of the RGB_view component to generate a bitstream. The identification information is used to identify the RGB_base component and the RGB_view component.

10. An information processing method, comprising: Perform atlas transformation on the alpha image of 3D data and generate atlas images of alpha components; The atlas image of the alpha component is encoded; Perform atlas transformation on the RGB image of the 3D data and generate an atlas image of RGB components; as well as The atlas image of the RGB components is encoded.

11. An information processing apparatus, comprising: The first decoding unit decodes the encoded data and generates an atlas image of the alpha component; The first inverse atlas transformation unit performs an inverse atlas transformation on the atlas image of the alpha component and generates an alpha image of 3D data; The second decoding unit decodes the encoded data and generates an atlas image of RGB components; as well as The second inverse atlas transformation unit performs an inverse atlas transformation on the atlas image of the RGB components and generates an RGB image of the 3D data.

12. The information processing apparatus according to claim 11, wherein, The RGB image includes an RGB_base image indicating the distribution of the RGB_base components in the RGB components and an RGB_view image indicating the distribution of the RGB_view components in the RGB components. The second decoding unit is configured to decode the encoded data and generate an atlas image of the RGB_base component and an atlas image of the RGB_view component. The second inverse atlas transformation unit is configured to perform an inverse atlas transformation on the atlas images of the RGB_base component and the RGB_view component using atlas transformation information shared by the RGB_base component and the RGB_view component, and The atlas transformation information is information indicating the content of the atlas transformation.

13. The information processing apparatus according to claim 11, wherein, The RGB image includes an RGB_base image indicating the distribution of the RGB_base components in the RGB components and an RGB_view image indicating the distribution of the RGB_view components in the RGB components. The RGB_base image and the RGB_view image have different sizes. The second decoding unit is configured to decode each of the encoded data of the atlas image of the RGB_base component and the encoded data of the atlas image of the RGB_view component, and The second atlas inverse transformation unit is configured to perform atlas inverse transformation independently on the atlas image of the RGB_base component and the atlas image of the RGB_view component.

14. The information processing apparatus according to claim 13, wherein, The second inverse atlas transformation unit is configured to perform an inverse atlas transformation on the atlas image of the RGB_base component using the atlas transformation information of the RGB_base component, and to perform an inverse atlas transformation on the atlas image of the RGB_view component using the atlas transformation information of the RGB_view component. The atlas transformation information is information indicating the content of the atlas transformation.

15. An information processing method, comprising: Decode the encoded data and generate an atlas image of the alpha component; Perform an inverse atlas transform on the atlas image of the alpha component and generate an alpha image of 3D data; Decode the encoded data and generate an atlas image with RGB components; as well as Perform an inverse atlas transform on the atlas image of the RGB components and generate an RGB image of the 3D data.

Images