Low-complexity multi-layer images with depth

Abstract

The present invention provides a method for generating a multi-layer image. The method includes receiving three-dimensional (3D) source data of a scene, generating a plurality of layers at different depths in a virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin. Image data is generated for each layer based on the 3D source data, the image data including texture data and transparency data, and a displacement map is generated for the layer based on the 3D source data and the depth value of the layer. The displacement map defines a depth of each point on a corresponding layer relative to a layer depth, and the generation of the displacement map is constrained such that a depth surface defined by any displacement map does not intersect with any other depth surface in the virtual 3D scene. The depth surface is defined as a mapping of the displacement map to a corresponding layer.

Description

[Technical field]

[0001] The present invention relates to multi-layer images, and in particular to generating and rendering multi-layer images. [Background technology]

[0002] Multi-Plane Imaging (MPI) is a potential alternative to Multi-View and Depth (MVD) formats for immersive video. Comparing the typical rendering errors of both formats, MVD suffers from noise on object boundaries, while MPI tends to blur the viewport.

[0003] A multi-plane image (MPI) is defined by multiple surfaces. Instead of a set of planes, it may be a set of spheres (onion rings) or any other type of surface. A sphere is useful for omnidirectional content, e.g., three degrees of freedom plus (3DoF+). Therefore, the term "multi-plane image" is generalized to "multi-layer image" in this application.

[0004] MVD type errors are caused by the assumption that a pixel has a single depth value. This is not true for object edges or semi-transparent objects. Depth coding noise amplifies this problem. To compensate for this, the views need to be spaced closer together, which increases the bitrate and pixel rate and therefore the processing cost of creating the bitstream (e.g., more hardware).

[0005] MPI type errors are caused by object surfaces that do not coincide with the image plane. To compensate for this, layers need to be spaced closer together, which increases the bitrate and pixel rate and therefore the processing cost to create the bitstream (e.g., more hardware).

[0006] A hybrid approach would be to add depth / displacement maps to each layer of MPI, but this would require a switch to a more computationally expensive rendering algorithm. The simplicity of rendering MPI compared to MVD is one of MPI's main advantages, as it allows it to be targeted to low-complexity devices. Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, there is a need for an image format that reduces both types of errors while also reducing the computational power required to render them. [Means for solving the problem]

[0008] The invention is defined by the claims.

[0009] According to an example embodiment of the present invention, there is provided a method for generating a multi-layer image, the method comprising: receiving three-dimensional (3D) source data for a scene; generating a number of layers at different depths within the virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin; generating image data for each layer based on the 3D source data, the image data including texture data and transparency data; generating a displacement map for the layer based on the 3D source data and a depth value of the layer, the displacement map defining a depth of each point on a corresponding layer relative to a layer depth; The generation of the displacement maps is constrained such that a depth surface as defined by any displacement map does not intersect with any other depth surface in the virtual 3D scene, and a depth surface is defined as the mapping of a displacement map onto a corresponding layer.

[0010] A multi-layer image (without a displacement map) suffers from blurring due to object surfaces that do not match the depth values ​​of any of the layers. More layers can be added, but at a computational and / or time cost. Depth can be added to a layer by having a displacement map for the layer that defines the depth (relative to the layer depth value) of each point on the layer.

[0011] However, adding additional depth to each layer can cause the layers to intersect, which significantly increases the computational resources required to render a multi-layer image and is likely to introduce artifacts into the final image. When rendering a single multi-layer image or pre-rendering multi-layer video, the time and resources required for decoding and rendering are usually not an issue. However, for broadcasting 3D multi-layer video live, it is preferable to synchronize the 3D video with its 2D counterpart.

[0012] The present invention is based in part on the realization that some rendering algorithms are faster than others. The inventors have determined that these fast rendering algorithms typically assume that layers do not intersect (when projected onto the viewport). This assumption is implicit since these algorithms are typically used to render multi-layer images without a displacement map. For example, the so-called painter's algorithm can be used to render multi-layer images / videos with displacement maps as long as the layers do not intersect.

[0013] The invention is therefore based on imposing restrictions on the generation of displacement maps that ensure that layers do not intersect. This allows any user to render multi-layer images / videos using faster and standardized rendering methods, thus enabling the broadcasting of live 3D multi-layer videos at stable, observer-friendly frame rates.

[0014] Mapping a displacement map to a corresponding layer may involve a translation from the origin to the corresponding layer, for example if the layer is planar. In addition, the displacement map may need to be stretched, for example if it is normalized.

[0015] The method may further include generating maximum and minimum depth values ​​for the depth surface using a corresponding displacement map, the generation of the displacement map being further constrained such that a difference between the maximum depth value of the first depth surface and the minimum depth value of the second adjacent depth surface is greater than or equal to a predetermined margin.

[0016] Coded video is useful because it minimizes the amount of data that must be transmitted. However, subsequent decoding of the data does not guarantee that the decoded data is exactly the same as the initial (pre-coded) data. Therefore, by having a margin between each layer, it is guaranteed that the decoded layers will not cross even after some encoding / decoding errors.

[0017] The margin may be, for example, at least 1% of the distance between each layer (without the displacement map). The exact distance may depend on the encoding / decoding algorithms used and the corresponding error margins of these algorithms.

[0018] The method may further include determining a plurality of first angles between a line of sight from the origin and a normal vector perpendicular to the depth surface, and generation of the displacement map is further constrained such that the first angles are less than or equal to a first predetermined angle.

[0019] When the normal vector of a surface of a layer with additional depth (i.e., a depth surface) makes a large angle with the line of sight (a straight line from the origin), the layer itself stretches when it is rendered, which reduces the quality of the rendered multi-layer image. This is usually due to the assumption that a pixel (a point on a layer) has a single depth value. Also, the large angle surface contains a large amount of data in a small space when viewed from the origin.

[0020] If the depth surface has a large "depth gradient" (i.e. the depth changes significantly in a small space), the depth difference between pixels is large. This can be measured by the angle (first angle) between the line of sight and the normal vector to the surface of the displacement map mapped onto the layer. Therefore, the stretching (when rendering) can be limited by limiting this angle (first angle).

[0021] The first predetermined angle may be limited to a maximum of 70 degrees. Preferably, the first predetermined angle is a maximum of 60 degrees.

[0022] The method may further include determining a plurality of second angles between a line of sight from a field of view space and a normal vector perpendicular to a depth surface having a corresponding displacement map, the field of view space defining a space in which the multi-layer image can be viewed, and generation of the displacement map is further constrained such that the second angles are less than or equal to a second predetermined angle.

[0023] Similar to the reason for limiting the first angle, limiting the "second angle" reduces the stretching effect when rendering a multi-layer image. However, the second angle takes into account the viewing space, which defines the space in which a viewer can see the multi-layer image.

[0024] In other words, all angles are considered from all points (not just the origin) from which the multilayer image can be seen. By restricting the second angle from any point in the visual space, the multilayer image is not stretched (at points with large depth gradients) from any point from which it can be seen (while satisfying the no-intersection requirement).

[0025] The generation of the displacement map may be further constrained such that the curvature at any point on the depth surface defined by the displacement map is less than or equal to a predetermined curvature value.

[0026] The curvature of a surface (i.e., the second derivative of a surface) defines the amount by which the surface deviates from being flat. For 2D curves, an example is that of a circle, which has a curvature equal to the inverse of its radius. Smaller circles curve more sharply and therefore have a higher curvature.

[0027] Therefore, for multi-layer images, it is advantageous to limit the curvature of the displacement map to avoid abrupt changes in depth. Furthermore, surfaces with relatively small curvature are smoother and therefore usually easier to render.

[0028] The generation of the displacement map is generating a coarse displacement map based on the 3D source data, where generation of the coarse displacement map is not limited; encoding the coarse displacement map; decoding the encoded coarse displacement map; generating a reconstruction filter based on the decoded coarse displacement map and a set of constraints, the displacement map being based on the coarse displacement map and the reconstruction filter.

[0029] Encoding and decoding data can introduce slight variations in the initial (pre-encoded) and final (post-decoded) data, and therefore constraints applied to the generation of the displacement map pre-encoding (i.e., non-intersecting layers, margins between layers, angle constraints and / or curvature constraints) may not be applied to the displacement map after it is decoded.

[0030] It is therefore advantageous to learn how encoding and decoding modify a coarse displacement map (which has no constraints), and then generate a reconstruction filter that applies the constraints when applied to the decoded coarse displacement map.

[0031] The method may further include separating the layer into a plurality of sub-layers based on image data for the layer, wherein a displacement map is generated for the sub-layers based on image data corresponding to the sub-layers.

[0032] For example, some regions of a layer may be completely transparent, and thus generation of a displacement map on these regions may not be necessary. Thus, a layer can be separated into sublayers based on image data (i.e., texture data and transparency data), and a displacement map can be generated for the sublayer instead of the entire layer. Furthermore, different constraints can be applied to the displacement map of each sublayer. This gives the encoder more room to optimize the representation of the content, but does not increase the complexity of decoding or rendering.

[0033] The present invention also provides a method for rendering a multi-layer image, the method comprising: obtaining image data suitable for generating a plurality of layers, a plurality of displacement maps and a multi-layer image; obtaining a constraint index, the constraint index including information about the constraint by which the displacement map was generated, the constraint index informing that a depth surface as defined by any displacement map does not intersect with any other depth surface in the virtual 3D scene, the depth surface being defined as a mapping of the displacement map onto a corresponding layer; determining a rendering method based on the constraint metric; Rendering a multi-layer image based on the layers, the displacement map and the image data using the determined rendering method.

[0034] The step of obtaining a displacement map may be based on obtaining a coarse displacement map corresponding to the displacement map generated without the constraint, obtaining a reconstruction filter, and applying the reconstruction filter to the coarse displacement map.

[0035] The present invention also provides a computer program product comprising computer program code means which, when executed on a computing device having a processing system, causes the processing system to perform all of the steps of a method for generating and / or rendering a multi-layer image.

[0036] The present invention also provides a system for generating a multi-layer image, the system comprising: Receive three-dimensional (3D) source data for the scene, generating a plurality of layers at different depths within the virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin; generating image data, including texture data and transparency data, for each layer based on the 3D source data; The processor is configured to generate, for a layer, a displacement map that defines depths of different points on a corresponding layer relative to a layer depth based on the 3D source data and a depth value of the layer.

[0037] The generation of the displacement maps is constrained such that the depth surface defined by any displacement map does not intersect with any other depth surface in the virtual 3D scene, and a depth surface is defined as the mapping of the displacement map onto the corresponding layer.

[0038] The processor is further configured to generate maximum and minimum depth values ​​of the depth surface using the corresponding displacement map, where equal, the generation of the displacement map is further constrained such that a difference between the maximum depth value of the first depth surface and the minimum depth value of the second adjacent depth surface is greater than or equal to a predetermined margin.

[0039] The processor can be further configured to determine a plurality of first angles between a line of sight from the origin and a normal vector perpendicular to the depth surface, and generation of the displacement map is further constrained such that the first angles are less than or equal to a first predetermined angle.

[0040] The processor can be further configured to determine a plurality of second angles between a line of sight from a field of view space and a normal vector perpendicular to a depth surface having a corresponding displacement map, the field of view space defining a space from which the multi-layer image can be observed, and generation of the displacement map is further constrained such that the second angles are less than or equal to a second predetermined angle.

[0041] The generation of the displacement map may be further constrained such that the curvature at any point on the depth surface defined by the displacement map is less than or equal to a predetermined curvature value.

[0042] The processor Generate a coarse displacement map based on the 3D source data (the generation of the coarse displacement map is not constrained); Encode the coarse displacement map, Decoding the encoded coarse displacement map; The method may be further configured to generate a displacement map by generating a reconstruction filter based on the decoded coarse displacement map and the set of constraints, said displacement map including a coarse displacement map and a reconstruction filter.

[0043] The processor may be further configured to separate the layer into a plurality of sub-layers based on the image data for the layer, wherein a displacement map for the sub-layers is generated based on the image data corresponding to the sub-layers.

[0044] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. [Brief description of the drawings]

[0045] For a better understanding of the present invention and to show more clearly how it may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which: [Figure 1a] FIG. 2 is a diagram illustrating layers in a multi-layer image. [Figure 1b] FIG. 1 illustrates layers in a multi-layer image with additional depth. [Diagram 2] FIG. 13 is a diagram showing an explanation of depth surface constraints. [Diagram 3] A first example of a method for generating a multi-layer image with constraints. [Figure 4] FIG. 13 illustrates a second example of a method for generating, transmitting, and rendering a multi-layer image with constraints. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] The present invention will now be described with reference to the drawings.

[0047] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the devices, systems and methods, are for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the devices, systems and methods of the present invention will become better understood from the following description, the appended claims, and the accompanying drawings. It should be understood that the drawings are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to denote the same or similar parts.

[0048] The present invention provides a method for generating a multi-layer image. The method receives three-dimensional (3D) source data of a scene and generates a number of layers at different depths in a virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin. Image data is generated for each layer based on the 3D source data, the image data including texture data and transparency data, and a displacement map is generated for the layer based on the 3D source data and the depth value of the layer. The displacement map defines a depth of each point on a corresponding layer relative to a layer depth, and the generation of the displacement map is constrained such that a depth surface defined by any displacement map does not intersect with any other depth surface in the virtual 3D scene. The depth surface is defined as a mapping of the displacement map to a corresponding layer.

[0049] Fig. 1a shows a diagram of layers 102 in a multi-layer image. Each layer 102 represents a different depth value in the virtual scene. A texture value and a transparency value are given to each pixel in each layer 102. In this case, the first layer 102a corresponds to the front of the surface of the object 104 closest to the first layer 102a. Thus, only the pixels corresponding to the front of the object 104 have texture, while the remaining pixels have a transparency value of 100%.

[0050] Similarly, the second layer 102b corresponds to the remainder of the front surface of the object 104 that is closer to the second layer 102b than to the first layer 102a. d The fourth layer 102c corresponds to the background 106. However, the layers 102 do not exactly match the object 104, which may cause blurring due to parallax between the layers 102.

[0051] In this example, the layers 102 are evenly spaced and are represented by planes, however the distance between the layers 102 may vary (e.g., relative to the distance from the origin) and the layers 102 may, for example, be spherical instead of planar.

[0052] Fig. 1b shows a diagram of layers 102 in a multi-layer image with additional depth. The additional depth can be defined by a displacement map of each layer 102. In this case, the layers 102 are warped towards the object surface, which significantly reduces the blurring from the parallax between the layers 102. However, multi-layer images with additional depth typically require significant computational resources to render (much more than the resources required for a multi-layer image without additional depth). Therefore, it is proposed to impose a constraint on the displacement map and convey this constraint in the bitstream.

[0053] 2 shows an illustration of constraints on a depth surface 202. The constraints are selected to allow the client to use a simpler rendering algorithm (e.g., the Inverse Painter's Algorithm) and further reduce the time and / or computational resources required to render a multi-layer image. Knowing the constraints, the encoder and decoder can exploit them in multiple ways.

[0054] The constraints are defined in terms of a "depth surface" 202. A depth surface 202 is essentially a layer 102 in virtual space with additional depth (i.e., with a displacement map applied to the layer 102). The constraints are as follows: - The displacement map is generated such that the depth surface 202 does not intersect in the virtual space. - A displacement map is generated such that the depth surface 202 is moved away in the virtual space by a specified margin 212. - Constrain the angle between any normal vector 206 from the depth surface 202 and the line of sight 204 from the virtual scene origin 208. - Limit the angle between any normal vector 206 from the depth surface 202 and any line of sight 204 through the defined viewing space 210. - Limit the curvature of the depth surface 202.

[0055] The spacing between layers 102 is generally proportional to their distance (r [m]) from the scene origin 208. That is, they are proportional to the normalized disparity (1 / r [m -1 ]) are equally spaced with respect to each other. The layers 102 do not overlap, and the two layers 102 are spaced apart by a reference distance r n <r n+1 , the boundary r between the two depth surfaces 202 is n, far is the geometric mean TIFF2024519659000002.tif914 or Arithmetic Mean It can be set to TIFF2024519659000003.tif922.

[0056] Each depth surface 202 can be defined by a tight box (or onion ring) in virtual space to include non-intersection constraints in the definition of the displacement map. The displacement map is calculated using delta depth values as TIFF2024519659000004.tif848 or normalized disparity The resulting image can be encoded as TIFF2024519659000005.tif750. Both choices are acceptable and not significantly different, unless it is acceptable to place the final layer 102 very far away or at infinite depth (i.e., zero normalized disparity), in which case it may be necessary to use normalized disparity.

[0057] It is also possible for the depth ranges of consecutive depth surfaces 202 to overlap, but still constrain them to not intersect, e.g., if the minimum distance of the depth surfaces 202 is less than the layer depth value of the adjacent layers 102 (i.e., TIFF2024519659000006.tif721 and TIFF2024519659000007.tif828). In this case, the constraint is not clear from the displacement map definition and needs to be constrained separately. The non-intersection constraint may also need to be signaled separately when transmitted, for example by adding an indicator (i.e., a flag) to the bitstream.

[0058] The displacement map may be encoded as a luma image, and the luma values ​​may be linearly mapped onto the delta depth or delta normalized disparity range of layer 102.

[0059] If the depth surfaces 202 do not intersect and the viewing zone does not intersect with any of the layers 102, then the depth order of the layers 102 is known and therefore the depth surfaces 202 can be rendered front to back (the inverse painter algorithm). When the depth surfaces 202 intersect, much more analysis (e.g., splitting, sorting, etc.) is required to blend the depth surfaces together.

[0060] If the margin 212 between the depth surfaces 202 is known, then r n, far <r n+1,near In other words, the difference between the maximum depth value of a depth surface 202 and the minimum depth value of an adjacent depth surface 202 is greater than or equal to a predetermined margin 212.

[0061] The margins may be specified as ratios where "zero" defines a multilayer image with no additional depth and "one" defines a multilayer image with non-overlapping, but potentially touching, depth surfaces 202. The margins 212 may be the same for all depth surfaces 202, or may be, for example, a ratio of maximum and minimum depth values ​​(for some or all depth surfaces 202). This may be specified for each individual layer 102 by explicitly providing TIFF2024519659000008.tif827.

[0062] Encoding and decoding errors may cause depth surfaces 202 to intersect after the displacement map has been encoded, transmitted, and decoded that were not initially intersected, therefore defining margins 212 between depth surfaces 202 ensures that they do not intersect after encoding / decoding.

[0063] When the normal vector 206 from the depth surface 202 makes a large angle with the line of sight 204 from the origin 208, the depth surface 202 may stretch during rendering, which may degrade quality. Typically, the rendering algorithm must compensate for this stretching (e.g., when rendering multiview+depth images), which requires additional time and / or computational resources. Thus, by limiting the angle, the overall computational resources and processing time required can be kept relatively low and consistent.

[0064] If an observer of a multi-layer image is expected to remain at the origin 208, then it is only necessary to constrain the line of sight 204 from the origin 208. However, if the observer is expected to move around, then it is necessary to limit the line of sight 204 from any point within the observation space 210.

[0065] The curvature of the depth surfaces 202 also impacts the time / computational resources required to render a multi-layer image. Sharp changes in the depth surfaces 202 (i.e., high curvature) are difficult to render, while smooth surfaces (i.e., small curvature) tend to be relatively easy to render. If the curvature is limited to a maximum curvature for one or more of the depth surfaces 202, the required time / computational resources can be kept even lower and consistent.

[0066] 3 shows a method for generating a multi-layer image 310 with constraints 302. Three-dimensional (3D) source data 304 of a scene is acquired, for example, from multiple cameras. Multiple layers are generated at different depths in the virtual 3D scene, with each layer corresponding to a particular layer depth value relative to an origin. Image data 308 is generated for each layer based on the 3D source data 304, the image data including texture data and transparency data.

[0067] A displacement map 306 is generated for each of the layers based on the 3D source data 304 and the depth values ​​of the layers, where the displacement map 306 defines the depth of each point on the corresponding layer relative to the layer depth. The generation of the displacement map 306 is also based on the constraints 302. The constraints 302 must include at least the non-intersection constraint mentioned above.

[0068] The displacement map 306 and the image data 308 can be rendered into a multi-layer image 310. Alternatively, the displacement map 306 and the image data 308 can be encoded and sent to a client for rendering by the client.

[0069] Optionally, the client (i.e., the decoder) has a filter that reconstructs the encoded displacement map 306 parameterized by the signaled constraints 302. For example, the decoder can use a blur filter or a morphological filter to remove abrupt transitions. The strength of the filter depends on the signaled constraints and may possibly vary locally.

[0070] 4 shows a method for generating, transmitting, and rendering a multi-layer image with constraints 302. The constraints 302 must include at least a "no intersecting layers" constraint. The constraints 302 may be obtained from user input or from a predefined list of constraints. The 3D source data 304 may be obtained from one or more cameras and / or additional depth sensors. The constraints 302 and the 3D source data 304 may be transmitted to the server 401, which transmits the data they need to the client 403 to render the multi-layer image.

[0071] The 3D source data 304 and constraints 302 are used by the server 401 to generate a coarse displacement map 402. The coarse displacement map 402 is then encoded (404) and decoded (406), and a reconstruction filter 408 is applied. The encoding 404 typically involves data compression or some type of data reduction to reduce the amount of data transmitted. Thus, when the encoded data is decoded in step 406, the resulting decoded data is not equal to the original data before encoding 408 (e.g., is of lower quality).

[0072] The reconstruction filter 408 filters the decoded displacement map according to the constraints 302 to ensure that the constraints are applied to the final displacement map. The filter can manipulate the displacement map values, for example to avoid crossing depth layers and to reduce their depth gradients and curvatures. Thus, the combination of the encoding and decoding steps and the reconstruction filter allows the transmission of a displacement map within the constraints 302, allowing transmission at a lower bit rate, since only a coarse displacement map is input to the encoder 404, which can apply data reduction techniques.

[0073] The 3D source data 304 is also used to generate image data 308 (i.e., transparency and texture / color values) for the layer 102. Thus, the coarse displacement map 402 and the image data 408 may be encoded 404 and transmitted, for example, from the server 401 to the client 403. The texture and transparency data may be optimized based on the reconstructed displacement map.

[0074] The client 403 can then decode (406) the transmitted data and apply a reconstruction filter 408 to the coarse displacement map 402. A multi-layer image can then be rendered (410) from the filtered displacement map and the decoded image data.

[0075] For example, the server 401 (i.e., the encoder) can utilize the reconstruction filter 408 by sending a coarse displacement map 402 with coarse and abrupt transitions. It is more efficient for the video codec to code 404 only the DC values ​​of the coding block using a discrete cosine transform (DCT). For example, when a block has uniform values, only one coefficient is needed to code the block. When this is applied, the coarse displacement map 402 appears "stair-stepped". This is possible because the reconstruction filter 408 filters the data to reduce depth gradients and curvatures. This can be achieved with an appropriate low-pass filter.

[0076] This is more efficient to encode (404), and both the encoder and the decoder can apply the specified reconstruction filter 408 to reconstruct the displacement map within the notified limits. The encoder in the server 401 can then calculate the appropriate values ​​of transparency and texture / color for each layer 102 based on the filtered displacement map. In this way, coding gain is achieved.

[0077] Additionally, the client 403 may have fallback rendering methods for rendering 410 outside the intended viewing space 210 where the constraints 302 are no longer guaranteed to be appropriate. This may involve scaling to reduce complexity, constraining the number of layers used for rendering, and / or approximating blending operations.

[0078] The layers 102 in a multi-layer image are typically sparse, and only a few (rectangular) patches are typically stored for each layer. Each patch can be considered as a separate sub-layer with separate parameters. This gives the encoder more room to optimize the representation of the content without increasing the complexity of the decoding or rendering.

[0079] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0080] A single processor or other unit may fulfill the functions of several items recited in the claims.

[0081] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0082] The computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0083] When the term "adapted for" appears in the claims or the specification, it is meant to be synonymous with the term "configured to."

[0084] Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. 1. A method for generating a multi-layer image, comprising: receiving three-dimensional (3D) source data of a scene; generating a number of layers at different depths in the virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin; generating image data for each layer based on the 3D source data, the image data having texture data and transparency data; generating a displacement map for the layer based on the 3D source data and the depth values ​​of the layer, the displacement map defining a depth of each point on a corresponding layer relative to the layer depth; A method in which the generation of the displacement map is constrained so that a depth surface defined by the displacement map does not intersect with other depth surfaces in the virtual 3D scene, and a depth surface is defined as a mapping of the displacement map onto a corresponding layer.

2. 2. The method of claim 1, further comprising the step of generating maximum and minimum depth values ​​of a depth surface having a corresponding displacement map, the generation of the displacement map being further constrained such that a difference between the maximum depth value of a first depth surface and the minimum depth value of an adjacent second depth surface is greater than or equal to a predetermined margin.

3. 3. The method of claim 1 or 2, further comprising determining a plurality of first angles between a line of sight from an origin and a normal perpendicular to the depth surface, and wherein generating the displacement map is further constrained such that the first angles are less than or equal to a first predetermined angle.

4. 3. The method of claim 1 or 2, further comprising the step of determining a plurality of second angles between a line of sight from an observation space and a normal perpendicular to a depth surface having a corresponding displacement map, the observation space defining a space from which the multi-layer image can be observed, and generation of the displacement map being further constrained such that the second angles are less than or equal to a second predetermined angle.

5. The method of claim 1 or 2, wherein generation of the displacement map is further constrained such that the curvature at any point on the depth surface defined by the displacement map is less than or equal to a predetermined curvature value.

6. The step of generating a displacement map comprises: generating a coarse displacement map based on the 3D source data, wherein generation of the coarse displacement map is unconstrained; and encoding the coarse displacement map; decoding the encoded coarse displacement map; generating a reconstruction filter based on the decoded coarse displacement map and a set of constraints; The method of claim 1 or 2, wherein the displacement map is based on the coarse displacement map and the reconstruction filter.

7. The method of claim 1 or 2, further comprising the step of separating a layer into a plurality of sub-layers based on the image data of the layer, and a displacement map is generated for a sub-layer based on the image data corresponding to the sub-layer.

8. 1. A method for rendering a multi-layer image, comprising: Obtaining a plurality of layers, a plurality of displacement maps and image data suitable for generating a multi-layer image; - obtaining an indication of a constraint, the indication of the constraint including information about the constraint with respect to which the displacement map was generated, the indication of the constraint informing that a depth surface defined by the displacement map does not intersect with any other depth surface in the virtual 3D scene, a depth surface being defined as a mapping of the displacement map onto a corresponding layer; determining a rendering method based on the constraint indicator; and rendering the multi-layer image based on the layers, the displacement map and the image data using the determined rendering method.

9. Obtaining the displacement map obtaining a coarse displacement map corresponding to the displacement map generated without the constraint; Obtaining a reconstruction filter; and applying the reconstruction filter to the coarse displacement map; The method according to claim 8, which is based on

10. A computer program product, when executed by a computing device having a processor system, causing said processor system to perform the method according to claim 1 or 8.

11. 1. A system for generating a multi-layer image, comprising: receiving three-dimensional (3D) source data of a scene; generating a number of layers at different depths in the virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin; generating image data for each layer based on the 3D source data, the image data having texture data and transparency data; generating a displacement map for the layer based on the 3D source data and the depth values ​​of the layer, the displacement map defining a depth of each point on a corresponding layer relative to the layer depth; a processor configured to execute The generation of the displacement map is constrained so that a depth surface defined by the displacement map does not intersect with other depth surfaces in the virtual 3D scene, and a depth surface is defined as a mapping of the displacement map onto a corresponding layer.

12. 12. The system of claim 11, wherein the processor is further configured to perform the step of generating maximum and minimum depth values ​​for a depth surface having a corresponding displacement map, the generation of the displacement map being further constrained such that a difference between the maximum depth value of a first depth surface and the minimum depth value of an adjacent second depth surface is greater than or equal to a predetermined margin.

13. 13. The system of claim 11 or 12, wherein the processor is further configured to perform the step of determining a plurality of first angles between a line of sight from an origin and a normal perpendicular to a depth surface, and generation of the displacement map is further constrained such that the first angles are less than or equal to a first predetermined angle.

14. The system of claim 11 or 12, wherein the processor is further configured to perform a step of determining a plurality of second angles between a line of sight from an observation space and a normal perpendicular to a depth surface having a corresponding displacement map, the observation space defining a space from which the multi-layer image can be observed, and generation of the displacement map is further constrained such that the second angles are less than or equal to a second predetermined angle.

15. The system of claim 11 or 12, wherein generation of the displacement map is further constrained such that the curvature at any point on the depth surface defined by the displacement map is less than or equal to a predetermined curvature value.

Images