Decoder-side depth and color alignment with metadata for trancoding of volume video

Abstract

Methods, apparatus, and data streams are provided to encode an unaligned multi-view plus depth image associated with assistance metadata for generating a depth map aligned with an unaligned color view. At the encoding stage, for misaligned color views, the contribution of each depth map is evaluated and the represented object is mapped. The picture patch is cut to be homogeneous in terms of object and contribution views. Metadata describing this homogeneous cut is encoded with the atlas. At a decoding phase, in order to generate a depth map aligned with the unaligned color view, only the depth map referenced in the assistance metadata is warped.

Description

1. Technical Field

[0001] The present principles generally relate to the field of multi-view plus depth (MVD) content, and in particular, the general principles relate to MVD content acquired by unaligned color and depth sensors.This document is also understood in the context of encoding and formatting of metadata associated with the encoded MVD content. 2. Background technology

[0002] This section is intended to introduce the reader to various aspects of the technology, which may be related to various aspects of the present principles described and / or claimed below. It is believed that this discussion is helpful in providing the reader with background information to facilitate a better understanding of various aspects of the present principles. Therefore, it should be understood that these statements are to be read in this light, and not as admissions of prior art.

[0003] Multi-view plus depth (MVD) images are acquired by a set of color and depth sensors located at different points of a 3D scene and facing the 3D scene. The result of such acquisition is a set of color views and depth views. The color and depth sensors can be aligned (i.e., the depth sensor and the color camera have a common focus and capture a common frustrum), or they can be misaligned. The number of depth sensors can be different from the number of color cameras. Since such a set of color images and depth views represents a large amount of data, MVD images are usually encoded and compressed according to methods that reduce the size of the data. For example, encoding an MVD image as a pair of patch atlases is a technique that allows the removal of redundant information between (color and / or depth) views and the compression of the atlas using image or video compression methods. Unaligned MVD images are easier to capture and more lightweight to encode than aligned MVD images, which require high-tech camera equipment and generate more depth maps.

[0004] MVD images are used, for example, to synthesize virtual views from viewpoints of a scene that do not correspond to one of the camera's viewpoints. View synthesis requires depth map warping and color blending operations. Encoded unaligned MVD images can also be converted into aligned MVD images. The process includes synthesizing a depth view (also called a depth map) for each color view (also called a color image or color map), that is, processing the distortion of each unaligned depth map for each color image. However, depending on the position of the sensor's viewpoint and the position of the view to be synthesized, each depth map does not contribute to view synthesis at the same level. In addition, the volumetric scene captured as an MVD image may include identified objects that can be managed independently when synthesizing virtual views (e.g., inserting 3D objects in different backgrounds). Therefore, when MVD images are unaligned, it is necessary to associate the object of interest of the color view with a distorted depth view that contributes to its geometry in order to reduce the complexity, computational burden, and memory usage of transcoding. 3. Summary of the invention

[0005] A simplified summary of the present principles is presented below to provide a basic understanding of some aspects of the present principles. This summary is not an extensive review of the present principles. It is not intended to identify the key or decisive elements of the present principles. The following summary merely presents some aspects of the present principles in a simplified form as a preface to the more detailed description provided below.

[0006] The present principles relate to a method for encoding a 3D scene in a data stream. The method includes obtaining a multi-view plus depth image, the multi-view plus depth image including a color picture with color picture position information and a depth picture with depth picture position information, wherein at least one depth picture is non-collocated with at least one color picture. Then, for at least one color picture, a depth view allocation map is generated. A color patch atlas and a depth patch atlas are generated. For both of them, the patch is part of a unique color picture or a unique depth picture selected according to the depth view allocation map. The color patch atlas, the depth patch atlas and metadata are encoded in the data stream, and for each color patch, the metadata includes information indicating which depth patch is used to un-project the color patch. In an embodiment, for at least one color picture, an object map is generated, and color patches and depth patches belonging to a unique object are generated according to the object map. In another embodiment, the metadata includes information indicating whether a given depth patch must be selected to un-project at least one color patch.

[0007] The present principles also relate to an apparatus comprising a memory associated with a processor, the processor being configured to implement the above-mentioned method.

[0008] The present principles also relate to a method for rendering a 3D scene from a data stream. The method comprises decoding a color patch atlas, a depth patch atlas, and metadata from the data stream, the metadata comprising, for each color patch, information indicating which depth patch to use to unproject the color patch. The 3D scene is rendered by unprojecting the color patch according to the corresponding depth patch.

[0009] The present principles also relate to an apparatus comprising a memory associated with a processor, the processor being configured to implement the above-mentioned method. 4. Description of the drawings

[0010] The present disclosure will be better understood, and other particular features and advantages will emerge, upon reading the following description, which refers to the accompanying drawings, in which:

[0011] - Figure 1shows an example camera array layout including four depth sensors placed around a three-by-three color camera rig;

[0012] - Figure 2A The diagram shows Figure 1 The contribution of the depth map and color image captured by the sensor to the synthesized view;

[0013] - Figure 2B Picture shows Figure 1 The potential contribution of the depth map of to the depth maps associated with the nine color views;

[0014] - Figure 3 An example architecture of a processing engine that may be configured to implement methods according to the present principles is shown;

[0015] - Figure 4 An example of an embodiment of the syntax of a data stream encoding an unaligned MVD image or a sequence of unaligned MVD images according to the present principles is shown;

[0016] - Figure 5 illustrates an example of an MVD scene segmented into four objects input in an encoder according to the present principles;

[0017] - Figure 6 A method for preparing metadata to assist in the alignment process of unaligned MVD images in accordance with the present principles is illustrated. 5. Specific implementation methods

[0018] The present principle will be described more fully below with reference to the accompanying drawings, in which examples of the present principle are shown. However, the present principle can be implemented in a variety of alternative forms and should not be interpreted as being limited to the examples set forth herein. Therefore, although the present principle is susceptible to various modifications and alternative forms, its specific examples are shown by the examples in the accompanying drawings and will be described in detail herein. However, it should be understood that there is no intention to limit the present principle to the specific form disclosed, but on the contrary, the present disclosure will cover all modifications, equivalents and alternatives that fall within the spirit and scope of the present principle as defined by the claims.

[0019] The terms used herein are only used for the purpose of describing specific examples and are not intended to be a limitation of this principle. As used herein, the singular forms "one", "an" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will be further understood that when the terms "include", "include", "contain" and / or "contain" are used in this specification, the existence of the features, integers, steps, operations, elements and / or components is specified, but the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups is not excluded. In addition, when an element is referred to as "in response to" or "connected to" another element, it can directly respond to or be connected to another element, or there can be an intermediate element. In contrast, when an element is referred to as "directly responding to" or "directly connected to" another element, there is no intermediate element. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items, and can be abbreviated as " / ".

[0020] It will be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of the present principles, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0021] Although some of the figures include arrows on communication paths to illustrate a primary direction of communication, it is to be understood that communication can occur in the opposite direction to the depicted arrows.

[0022] Some examples are described with respect to block diagrams and operational flow charts, wherein each block represents a portion of a circuit element, module, or code, which includes one or more executable instructions for implementing a specified (one or more) logical function. It should also be noted that in other implementations, the (one or more) functions indicated in the blocks may not appear in the order indicated. For example, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the functions involved.

[0023] References herein to "according to an example" or "in an example" mean that a particular feature, structure, or characteristic described in connection with the example may be included in at least one implementation of the present principles. The phrases "according to an example" or "in an example" appearing in various places in the specification are not necessarily all referring to the same example, nor are they necessarily separate or alternative examples that are mutually exclusive of other examples.

[0024] Reference signs appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.The present examples and variations may be used in any combination or sub-combination although not explicitly described.

[0025] Figure 1 An example camera array layout is shown including four depth sensors Vd1 to Vd4 placed around a three by three color camera setup. The present principles are valid for any other arbitrary number and relative positions of color and depth sensors. Figure 1 In the example of , color cameras Vc1 to Vc9 are not aligned with depth sensors Vd1 to Vd4. Figure 1 In the example above, the number of depth sensors is lower than the number of color cameras. Figure 1 The positions and orientations of a set of color cameras (each of which captures a color view) and a set of depth sensors (each of which captures a depth view) are schematically shown. Each color or depth sensor can have a different orientation, and even in practice it is more convenient to organize them into rigs that point in a unique direction. In various embodiments, the rig has a spherical shape, and the sensors (depth and color) point in each direction.

[0026] Figure 2A The diagram shows Figure 1 The contribution of the depth map and color image captured by the sensor to the synthesized view SV. In order to synthesize the virtual view SV, the renderer reversely projects the (u, v) pixels of each decoded depth view to their (x, y, z) 3D positions, and reprojects the corresponding color attributes of the color view of the corresponding pixel coordinates (u', v') into the virtual viewport reference system (i.e., the pixel array of the view to be synthesized). When the depth view is not aligned with the color view, such a reverse projection plus reprojection process is not simple. In fact, the color value of the point (x, y, z) (obtained from the reverse projection of the pixel of at least one depth view) does not directly correspond to the pixel of the color view. Existing view rendering algorithms use a two-stage visibility and coloring method. First, a viewport depth view (also called a visibility map) is generated by warping all depth views from the MVD image to the viewport, selecting the best depth candidate for each pixel, and filtering the resulting viewport depth view. Then, according to the visibility map, the color of the viewport is generated by mixing the colors of the source view. Such a two-stage method for view rendering does not require the depth view and the color source view to be spatially aligned. However, blending colors on an unaligned visibility map is a more complex process than blending colors on an aligned visibility map.

[0027] In some use cases, the MVD source must be regenerated at the decoder side and transcoded into an MVD signal with aligned color and depth views. For example, such transcoding may be required when an autostereoscopic multi-view display is used. When transcoding is required, for example, when transcoding of an MIV (MPEG Immersive Video) signal must be performed to another 3D compression format that assumes spatially aligned color and depth components (such as 3D-HEVC (High Efficiency Video Coding)), it may also be necessary to generate aligned MVD images. The generation of a depth map aligned with the decoded color view involves depth warping, selection and filtering operations similar to those in the visibility stage of the rendering algorithm. However, since it is not a single viewport depth view that is to be calculated (but a depth view corresponding to each source color view), the computational requirements increase proportionally with the number of color cameras.

[0028] Figure 2B The potential contribution of the depth view from the top left depth sensor (Vd1) to the depth views associated with the nine color views is illustrated. The top left depth sensor Vd1 can contribute primarily to four nearby color views (Vc1, Vc2, Vc4, Vc5) instead of the other five color views, and the number of depth warp operations can be reduced to the closest color views. Depending on the 3D geometry of the captured scene and the position and orientation of the depth sensor, the operation may be more complex.

[0029] Figure 5 An example of an MVD scene segmented into four objects (background and three foreground characters) is illustrated, which is input into an encoder according to the present principles, wherein a segmentation mask (also called entity map) is associated with each source view. In a patch atlas representation of the 3D scene (not represented in the figure, e.g., such as in the MIV standard), each patch can be assigned a unique entity ID at the encoder side, and at the decoder side, possible filtering of a group of objects can be enabled, e.g., for partial reconstruction. For example, in Figure 5 In the example of , the background entities can be discarded and three characters can be rendered for compositions with different backgrounds. According to the present principles, for applications that need to transcode decoded unaligned color and depth views into color-depth spatially aligned representations on only a subset of given objects rather than on the entire scene, metadata is provided at the object level to assist the alignment process.

[0030] Figure 6 A method for preparing metadata to assist the alignment process of unaligned MVD images according to the present principles is illustrated.At a first step, for each color view, an object map and a depth view allocation map are generated. Figure 6 Shown for Figure 5The object map 61 and the depth view allocation map 62 are prepared for the color view 0. For each pixel of the color view, the object map identifies the object to which the color pixel belongs. For each pixel of the color view, the depth view allocation map identifies from which depth view the assigned depth value is warped. Figure 6 In , the object map 61 identifies four objects 61a to 61d corresponding to three characters and the background, and the depth view allocation map 62 identifies two depth views 62a and 62b that help generate a new depth map aligned with the color view. Figure 6 In the example of , most of the depth values ​​are distorted from, for example, a depth view 62a captured by the nearest depth sensor. Because the depth sensor is not aligned with the color camera, some areas are occluded and distorted from a second depth view 62b captured by a different depth sensor.

[0031] The depth view allocation map can be calculated using the following algorithm:

[0032]

[0033] In by Figure 6 At the second step of the illustrated method, patches are cut to prepare a patch atlas. Each color view can then be cut into patches that are homogeneous in terms of objects and also in terms of the source depth view, i.e., each patch has a single object ID and a single depth view index. These homogeneous patches are packed into a color atlas frame associated with a binary occupancy atlas frame to indicate which pixels of the patch are valid, i.e., belong to an object.

[0034] As an option, when defining a color patch, the bounding box that contains all depth samples of the current patch needed to warp depth into the attached depth view can be determined directly by computing the minimum and maximum values ​​of Ax[v][u] and Ay[v][u] of the current patch.

[0035] At the third step of the method, metadata for assisting the alignment process of the unaligned MVD images are prepared and encoded with the atlas. The syntax of these metadata can be based on the MIV syntax as described below. First, it is signaled in the sequence parameter set of each atlas whether patch depth warp assistance data is available. This depth warp assistance data is only applicable to atlases without geometric components (i.e. aligned depth patches). The basic assumption is that at the encoding stage, patches from color-only views and patches from depth-only views are packed in separate atlases.

[0036]

[0037] asme_depth_warping_assistance_flag equal to 1 indicates that the pdu_depth_warping_present_flag[tileID][p] syntax element is present in the pdu_miv_extension() syntax structure. asme_depth_warping_assistance_flag equal to 0 indicates that the pdu_depth_warping_present_flag[tileID][p] syntax element is not present in the pdu_miv_extension() syntax structure. When not present, the value of asme_depth_warping_assistance_flag is inferred to be equal to 0.

[0038] When asme_patch_constant_depth_flag is equal to 1 or vps_geometry_video_present_flag[aspsAtlasID] is equal to 1 or pin_geometry_present_flag[aspsAtlasID] is equal to 1, asme_depth_warping_assistance_flag equal to 0 is a requirement for bitstream conformance.

[0039]

[0040] pdu_depth_warping_present_flag[tileID][p] equal to 1 indicates that depth warping assistance data is present in the pdu_miv_extension(tileID, p) syntax structure. pdu_depth_warping_present_flag[tileID][p] equal to 0 indicates that depth warping assistance data is not present in the pdu_miv_extension(tileID, p) syntax structure.

[0041] pdu_depth_view_idx[tileID][p] specifies the index of the view with index k that has a geometry component that contributes to the geometry of the patch with index p in the tile with tile ID equal to tileID. The value of pdu_depth_view_idx[tileID][p] shall be in the range of 0 to mvp_num_views_minus1, inclusive.

[0042] pdu_depth_bounding_box_present_flag[tileID][p] equal to 1 indicates that there are bounding box parameters for the depth samples contributing to the geometry of the patch indexed p in the tile with tileID equal to tileID. pdu_depth_bounding_box_present_flag[tileID][p] equal to 0 indicates that there are no bounding box parameters.

[0043] pdu_depth_bb_pos_x[tileID][p] specifies the x coordinate of the upper left corner of the depth bounding box in the view with index equal to pdu_depth_view_idx[tileID][p].

[0044] pdu_depth_bb_pos_y[tileID][p] specifies the y coordinate of the upper left corner of the depth bounding box in the view with index equal to pdu_depth_view_idx[tileID][p].

[0045] pdu_depth_bb_size_x[tileID][p] specifies the width value of the depth bounding box in the view with an index equal to pdu_depth_view_idx[tileID][p].

[0046] pdu_depth_bb_size_y[tileID][p] specifies the height value of the depth bounding box in the view with index equal to pdu_depth_view_idx[tileID][p].

[0047] According to the present principles, a method is provided for decoding a patch atlas representing a 3D scene and comprising unaligned color and depth patches. The depth component of the color patch corresponding to an object of interest is computed using the visibility step of the view rendering algorithm, but is fed with only a subset of the decoded depth values ​​in the bounding box of the depth view signaled by metadata:

[0048]

[0049] Therefore, the decoding method consists of three steps:

[0050] -Decode depth, color and occupancy patch atlas frames;

[0051] - unpack the source depth view and reconstruct the color source view of the target object; and

[0052] - Generate color-depth aligned MVDs for target objects by warping the decoded depth samples, guided by the metadata attached to the color patches.

[0053] Figure 3 An example architecture of a processing engine 30 that can be configured to implement the two methods described herein is shown. Figure 3 The devices of the architecture are linked to other devices via the bus 31 of other devices and / or via the I / O interface 36.

[0054] The device 30 comprises the following elements linked together by a data and address bus 31:

[0055] - a microprocessor 32 (or CPU), which is for example a DSP (or digital signal processor);

[0056] - ROM (or read-only memory) 33;

[0057] - RAM (or random access memory) 34;

[0058] - storage interface 35;

[0059] - an I / O interface 36 for receiving data to be transmitted from an application; and

[0060] -power supply( Figure 3 not shown), such as batteries.

[0061] According to an example, the power supply is external to the device. In each of the memories mentioned, the word "register" used in the specification can correspond to an area of ​​small capacity (a few bits) or a very large area (such as an entire program or a large amount of received or decoded data). ROM 33 includes at least programs and parameters. ROM 33 can store algorithms and instructions to perform the technology according to the present principles. When turned on, CPU 32 uploads the program to RAM and executes the corresponding instructions.

[0062] RAM 34 includes in registers a program executed by CPU 32 and uploaded after device 30 is turned on, input data in registers, intermediate data in different states of a method in registers, and other variables in registers for executing the method.

[0063] The implementation described herein can be implemented in, for example, a method or process, a device, a computer program product, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method or device), the implementation of the discussed features can also be implemented in other forms (e.g., programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device (such as, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device). The processor also includes a communication device, such as, for example, a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate the communication of information between end users.

[0064] The device 30 is linked, for example, via a bus 31, to a set of sensors 37 and a set of rendering devices 38. The sensors 37 may be, for example, cameras, microphones, temperature sensors, inertial measurement units, GPS, humidity sensors, IR or UV light sensors, or wind sensors. The rendering devices 38 may be, for example, displays, speakers, vibrators, heaters, fans, etc.

[0065] According to an example, the device 30 is configured to implement these two methods according to the present principles and belongs to the set comprising:

[0066] -mobile device;

[0067] -communication equipment;

[0068] -Gaming equipment;

[0069] - Tablet (or tablet computer);

[0070] - Laptop;

[0071] -Still picture camera;

[0072] -Camcorder.

[0073] Figure 4An example of an embodiment of the syntax of a data stream for encoding an unaligned MVD image or a sequence of unaligned MVD images according to the present principles is shown. The structure includes a container that organizes the stream into independent grammatical elements. The structure may include a header portion 41, which is a set of data common to each grammatical element of the stream. For example, the header portion includes some of the metadata about the grammatical elements that describe the properties and functions of each of the grammatical elements. The structure also includes a payload that includes an element 42 of the syntax and an element 43 of the syntax. The syntax element 42 includes data representing the unaligned MVD image, namely a color view and a depth map (also called a depth view). The image may have been compressed according to a compression method. The element 43 of the syntax is part of the payload of the data stream and includes data for encoding auxiliary metadata as described in accordance with the present principles. The items of the auxiliary metadata refer to unaligned color views of the MVD image and include a subset of the depth views of the MVD image.

[0074] The implementation described herein can be implemented in, for example, a method or process, a device, a computer program product, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method or device), the implementation of the discussed features can also be implemented in other forms (e.g., programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device (such as, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device). The processor also includes a communication device, such as, for example, a smart phone, a tablet, a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate the communication of information between end users.

[0075] The realization of various processes and features described herein can be realized in a variety of different equipment or applications, particularly in equipment or applications associated with, for example, data encoding, data decoding, view generation, texture processing and other processing of images and related texture information and / or depth information. Examples of such equipment include encoders, decoders, post-processors for processing outputs from decoders, pre-processors for providing inputs to encoders, video encoders, video decoders, video codecs, web servers, set-top boxes, laptop computers, personal computers, cellular phones, PDAs and other communication devices. It should be clear that the equipment can be mobile and even installed in a mobile vehicle.

[0076] Furthermore, the method may be implemented by instructions executed by a processor, and such instructions (and / or data values ​​resulting from the implementation) may be stored on a processor-readable medium, such as, for example, an integrated circuit, a software carrier, or other storage device, such as, for example, a hard disk, a compact disk ("CD"), an optical disk (such as, for example, a DVD, commonly referred to as a digital versatile disk or digital video disk), a random access memory ("RAM"), or a read-only memory ("ROM"). The instructions may form an application program tangibly implemented on a processor-readable medium. The instructions may, for example, be in hardware, firmware, software, or a combination. The instructions may exist in, for example, an operating system, a separate application, or a combination of both. Thus, a processor may be characterized as, for example, a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. Furthermore, in addition to or in lieu of instructions, the processor-readable medium may store data values ​​resulting from the implementation.

[0077] As will be apparent to one skilled in the art, implementations may generate a variety of formatted signals to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the grammar of the embodiment, or to carry as data the actual grammar values ​​written by the embodiment. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is well known, signals may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0078] A variety of implementations have been described. However, it will be appreciated that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. Furthermore, it will be appreciated by those of ordinary skill that other structures and processes may replace those disclosed, and that the resulting implementations will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed implementations. Therefore, the present application contemplates these and other implementations.

Claims

1. A method for encoding a 3D scene in a data stream, the method comprising: - obtaining a multi-view plus depth image, the multi-view plus depth image comprising a color picture with color picture position information and a depth picture with depth picture position information, wherein at least one depth picture is not co-located with at least one color picture; - for at least one color picture, generating a depth view allocation map; - Generate a color patch atlas and a depth patch atlas, a patch is a part of a unique color picture or a part of a unique depth picture according to the depth view allocation map; as well as - Encoding in a data stream a color patch atlas, a depth patch atlas and metadata, the metadata including, for each color patch, information indicating which depth patch to use to unproject the color patch.

2. The method of claim 1, wherein for at least one color picture, an object map is generated, and wherein color patches and depth patches belonging to unique objects are generated based on the object map.

3. A method according to claim 1 or 2, wherein the metadata comprises information indicating whether a given depth patch has to be selected to unproject at least one color patch.

4. A device for encoding a 3D scene in a data stream, the device comprising a processor configured to: - obtaining a multi-view plus depth image, the multi-view plus depth image comprising a color picture with color picture position information and a depth picture with depth picture position information, wherein at least one depth picture is not co-located with at least one color picture; - for at least one color picture, generating a depth view allocation map; - Generate a color patch atlas and a depth patch atlas, a patch is a part of a unique color picture or a part of a unique depth picture according to the depth view allocation map; as well as - Encoding in a data stream a color patch atlas, a depth patch atlas and metadata, the metadata including, for each color patch, information indicating which depth patch to use to unproject the color patch.

5. The apparatus of claim 4, wherein for at least one color picture, an object map is generated, and wherein color patches and depth patches belonging to unique objects are generated based on the object map.

6. A device according to claim 4 or 5, wherein the metadata comprises information indicating whether a given depth patch has to be selected to unproject at least one color patch.

7. A method for rendering a 3D scene from a data stream, the method comprising: -decoding a color patch atlas, a depth patch atlas and metadata from the data stream, the metadata including, for each color patch, information indicating which depth patch to use to unproject the color patch; as well as - Renders a 3D scene by unprojecting color patches according to their corresponding depth patches.

8. A device for rendering a 3D scene from a data stream, the device comprising a processor configured to: - decoding a color patch atlas, a depth patch atlas and metadata from the data stream, the metadata including, for each color patch, information indicating which depth patch to use to unproject the color patch; and - Renders a 3D scene by unprojecting color patches according to their corresponding depth patches.

Images