Low-complexity multi-layer images with depth

Abstract

The present invention provides a method for generating a multi-layer image. The method includes receiving three-dimensional (3D) source data of a scene, generating a plurality of layers at different depths in a virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin. Image data is generated for each layer based on the 3D source data, the image data including texture data and transparency data, and a displacement map is generated for the layer based on the 3D source data and the depth value of the layer. The displacement map defines a depth of each point on a corresponding layer relative to a layer depth, and the generation of the displacement map is constrained such that a depth surface defined by any displacement map does not intersect with any other depth surface in the virtual 3D scene. The depth surface is defined as a mapping of the displacement map to a corresponding layer.

Description

[Technical Field]

[0001] The present invention relates to multi-layer images, and more particularly to generating and rendering multi-layer images. [Background technology]

[0002] Multi-plane imaging (MPI) is a potential alternative to multi-view and depth (MVD) formats for immersive video. Comparing the typical rendering errors of both formats, MVD suffers from noise on object boundaries, while MPI tends to blur the viewport.

[0003] A multi-plane image (MPI) is defined by multiple surfaces. Instead of a set of planes, it can be a set of spheres (onion rings) or any other type of surface. Spherical surfaces are useful for omnidirectional content, e.g., three degrees of freedom plus (3DoF+). Therefore, the term "multi-plane image" is generalized to "multi-layer image" in this application.

[0004] MVD-type errors are caused by the assumption that pixels have a single depth value. This is not true for object edges or semi-transparent objects. Depth coding noise amplifies this problem. To compensate, views need to be spaced closer together, which increases the bitrate and pixel rate and therefore the processing cost of creating the bitstream (e.g., more hardware).

[0005] MPI-type errors are caused by object surfaces that do not coincide with the image plane. To compensate for this, layers need to be spaced more closely, which increases the bitrate and pixel rate and therefore the processing cost (e.g., more hardware) to create the bitstream.

[0006] A hybrid approach would be to add a depth / displacement map to each layer of MPI, but this would require switching to a more computationally expensive rendering algorithm. The simplicity of rendering MPI compared to MVD is one of the main advantages of MPI, as it allows it to be targeted to low-complexity devices. Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, there is a need for a type of image format that reduces both types of errors while also reducing the computational power required to render them. [Means for solving the problem]

[0008] The invention is defined by the claims.

[0009] According to an example embodiment of the present invention, there is provided a method for generating a multi-layer image, the method comprising: receiving three-dimensional (3D) source data of a scene; generating a plurality of layers at different depths within the virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin; generating image data for each layer based on the 3D source data, the image data including texture data and transparency data; generating a displacement map for the layer based on the 3D source data and depth values ​​of the layer, the displacement map defining a depth of each point on the corresponding layer relative to the layer depth; The generation of the displacement maps is constrained so that the depth surface as defined by any displacement map does not intersect with any other depth surface in the virtual 3D scene, and a depth surface is defined as the mapping of the displacement map onto the corresponding layer.

[0010] Multi-layer images (without a displacement map) suffer from blurring due to object surfaces that do not match the depth values ​​of any of the layers. More layers can be added, but at a computational and / or time cost. Depth can be added to a layer by having a displacement map for the layer that defines the depth (relative to the layer depth value) of each point on the layer.

[0011] However, adding additional depth to each layer can cause layers to intersect, which significantly increases the computational resources required to render a multi-layer image and is likely to introduce artifacts into the final image. When rendering a single multi-layer image or pre-rendering multi-layer video, the time and resources required for decoding and rendering are usually not an issue. However, for broadcasting 3D multi-layer video live, it is preferable to synchronize the 3D video with its 2D counterpart.

[0012] The present invention is based in part on the recognition that some rendering algorithms are faster than others. The inventors have determined that these fast rendering algorithms typically assume that layers do not intersect (when projected onto the viewport). This assumption is implicit because these algorithms are typically used to render multi-layer images without displacement maps. For example, the so-called painter's algorithm can be used to render multi-layer images / videos with displacement maps, as long as the layers do not intersect.

[0013] Therefore, the present invention is based on imposing restrictions on the generation of displacement maps that ensure that layers do not intersect. This allows any user to render multi-layer images / videos using faster and standardized rendering methods, thus enabling the broadcast of live 3D multi-layer videos at stable, observer-friendly frame rates.

[0014] Mapping a displacement map to a corresponding layer may involve a translation from the origin to the corresponding layer, for example, if the layer is planar. In addition, the displacement map may need to be stretched, for example, if it is normalized.

[0015] The method may further include generating maximum and minimum depth values ​​for the depth surfaces using corresponding displacement maps, wherein the generation of the displacement maps is further constrained such that a difference between the maximum depth value of the first depth surface and the minimum depth value of the second adjacent depth surface is greater than or equal to a predetermined margin.

[0016] Coded video is useful because it minimizes the amount of data that must be transmitted. However, subsequent decoding of the data does not guarantee that the decoded data will be exactly the same as the initial (pre-encoded) data. Therefore, by having a margin between each layer, the decoded layers are guaranteed not to cross even after some encoding / decoding errors.

[0017] The margin may be, for example, at least 1% of the distance between each layer (without the displacement map). The exact distance may depend on the encoding / decoding algorithms used and the corresponding error margins of these algorithms.

[0018] The method may further include determining a plurality of first angles between a line of sight from the origin and a normal vector perpendicular to the depth surface, wherein generation of the displacement map is further constrained such that the first angles are less than or equal to a first predetermined angle.

[0019] When the normal vector of a layer's surface (i.e., depth surface) with additional depth makes a large angle with the line of sight (a straight line from the origin), the layer itself stretches when it is rendered, which reduces the quality of the rendered multi-layer image. This is usually due to the assumption that a pixel (a point on a layer) has a single depth value. Also, a surface with a large angle contains a large amount of data in a small space when viewed from the origin.

[0020] If a depth surface has a large "depth gradient" (i.e., the depth changes significantly over a small space), the depth difference between pixels is large. This can be measured by the angle (first angle) between the line of sight and the normal vector to the surface of the displacement map mapped onto the layer. Therefore, stretching (at rendering time) can be limited by limiting this angle (first angle).

[0021] The first predetermined angle may be limited to a maximum of 70 degrees. Preferably, the first predetermined angle is a maximum of 60 degrees.

[0022] The method may further include determining a plurality of second angles between a line of sight from a viewing space and a normal vector perpendicular to a depth surface having a corresponding displacement map, the viewing space defining a space in which the multi-layer image can be viewed, and generation of the displacement map further constrained such that the second angles are less than or equal to a second predetermined angle.

[0023] Similar to the reason for limiting the first angle, limiting the "second angle" reduces the stretching effect when rendering a multi-layer image. However, the second angle takes into account the viewing space, which defines the space in which a viewer can see the multi-layer image.

[0024] In other words, all angles are considered from all points (not just the origin) from which the multi-layer image can be viewed. By restricting the second angle from any point in visual space, the multi-layer image is not stretched (at points with large depth gradients) from any point from which it can be viewed (while satisfying the no-intersection requirement).

[0025] The generation of the displacement map may be further constrained so that the curvature at any point on the depth surface defined by the displacement map is less than or equal to a predetermined curvature value.

[0026] The curvature of a surface (i.e., the second derivative of a surface) defines the amount by which the surface deviates from being flat. For 2D curves, an example is that of a circle, which has a curvature equal to the reciprocal of its radius. Smaller circles curve more sharply and therefore have a higher curvature.

[0027] Therefore, for multi-layer images, it is advantageous to limit the curvature of the displacement map to avoid abrupt changes in depth. Furthermore, surfaces with relatively small curvature are smoother and therefore usually easier to render.

[0028] The generation of the displacement map is generating a coarse displacement map based on the 3D source data, wherein the generation of the coarse displacement map is not limited; encoding the coarse displacement map; decoding the encoded coarse displacement map; generating a reconstruction filter based on the decoded coarse displacement map and a set of constraints, wherein the displacement map is based on the coarse displacement map and the reconstruction filter;

[0029] Encoding and decoding data can introduce slight variations in the initial (pre-coded) and final (post-decoded) data, and therefore constraints applied to the generation of the displacement map pre-coding (i.e., non-intersecting layers, margins between layers, angle constraints, and / or curvature constraints) may not apply to the displacement map after it is decoded.

[0030] It is therefore advantageous to learn how encoding and decoding modify a coarse displacement map (which has no constraints), and then generate a reconstruction filter that enforces the constraints when applied to the decoded coarse displacement map.

[0031] The method may further include separating the layer into a plurality of sub-layers based on image data for the layer, and generating displacement maps for the sub-layers based on image data corresponding to the sub-layers.

[0032] For example, some regions of a layer may be completely transparent, and therefore generation of a displacement map on these regions may not be necessary. Therefore, a layer can be separated into sublayers based on image data (i.e., texture data and transparency data), and a displacement map can be generated for the sublayer instead of the entire layer. Furthermore, different constraints can be applied to the displacement map of each sublayer. This gives the encoder more room to optimize the representation of the content, but does not increase the complexity of decoding or rendering.

[0033] The present invention also provides a method for rendering a multi-layer image, the method comprising: obtaining image data suitable for generating a plurality of layers, a plurality of displacement maps and a multi-layer image; obtaining a constraint index, the constraint index containing information about the constraint by which the displacement map was generated, the constraint index informing that a depth surface as defined by any displacement map does not intersect with any other depth surface in the virtual 3D scene, a depth surface being defined as a mapping of the displacement map to a corresponding layer; determining a rendering method based on the constraint indicators; Rendering a multi-layer image based on the layers, the displacement map and the image data using the determined rendering method.

[0034] The step of obtaining the displacement map may be based on obtaining a coarse displacement map corresponding to the displacement map generated without constraints, obtaining a reconstruction filter, and applying the reconstruction filter to the coarse displacement map.

[0035] The present invention also provides a computer program product comprising computer program code means which, when executed on a computing device having a processing system, causes the processing system to perform all of the steps of a method for generating and / or rendering a multi-layer image.

[0036] The present invention also provides a system for generating a multi-layer image, the system comprising: Receives three-dimensional (3D) source data for the scene, generating a plurality of layers at different depths within the virtual 3D scene, each layer corresponding to a particular layer depth value relative to the origin; generating image data including texture data and transparency data for each layer based on the 3D source data; A processor configured to generate, for a layer, a displacement map based on the 3D source data and a depth value of the layer, the displacement map defining depths of different points on the corresponding layer relative to the layer depth.

[0037] The generation of the displacement maps is constrained such that the depth surface defined by any displacement map does not intersect with any other depth surface in the virtual 3D scene, and a depth surface is defined as the mapping of the displacement map onto the corresponding layer.

[0038] The processor is further configured to generate maximum and minimum depth values ​​for the depth surfaces using the corresponding displacement maps, whereby the generation of the equal displacement maps is further constrained such that a difference between the maximum depth value of the first depth surface and the minimum depth value of the second adjacent depth surface is greater than or equal to a predetermined margin.

[0039] The processor may be further configured to determine a plurality of first angles between the line of sight from the origin and a normal vector perpendicular to the depth surface, and generation of the displacement map is further constrained such that the first angles are less than or equal to a first predetermined angle.

[0040] The processor may be further configured to determine a plurality of second angles between a line of sight from a viewing space and a normal vector perpendicular to a depth surface having a corresponding displacement map, the viewing space defining a space from which the multi-layer image can be observed, and generation of the displacement map further constrained such that the second angles are less than or equal to a second predetermined angle.

[0041] The generation of the displacement map may be further constrained so that the curvature at any point on the depth surface defined by the displacement map is less than or equal to a predetermined curvature value.

[0042] The processor generating a coarse displacement map based on the 3D source data (the generation of the coarse displacement map is not constrained); Encode the coarse displacement map, Decoding the encoded coarse displacement map; The method may be further configured to generate a displacement map by generating a reconstruction filter based on the decoded coarse displacement map and the set of constraints, wherein the displacement map comprises the coarse displacement map and the reconstruction filter.

[0043] The processor may be further configured to separate the layer into a plurality of sub-layers based on the image data for the layer, and wherein a displacement map is generated for the sub-layers based on the image data corresponding to the sub-layers.

[0044] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. [Brief explanation of the drawings]

[0045] For a better understanding of the present invention and to show more clearly how the same may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which: [Figure 1a] FIG. 1 is a diagram illustrating layers in a multi-layer image. [Figure 1b] FIG. 1 illustrates layers in a multi-layer image with additional depth. [Figure 2] FIG. 10 is a diagram illustrating a depth surface restriction. [Figure 3] A first example of a method for generating multi-layer images with constraints. [Figure 4] FIG. 10 illustrates a second example of how to generate, transmit, and render a multi-layer image with constraints. DETAILED DESCRIPTION OF THE INVENTION

[0046] The present invention will now be described with reference to the drawings.

[0047] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the devices, systems, and methods, are intended for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the devices, systems, and methods of the present invention will become better understood from the following description, appended claims, and accompanying drawings. It should be understood that the drawings are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to indicate the same or similar parts.

[0048] The present invention provides a method for generating a multi-layer image. The method receives three-dimensional (3D) source data of a scene and generates multiple layers at different depths within a virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin. Image data is generated for each layer based on the 3D source data, the image data including texture data and transparency data, and a displacement map is generated for the layer based on the 3D source data and the layer depth value. The displacement map defines the depth of each point on a corresponding layer relative to the layer depth, and the generation of the displacement map is constrained so that a depth surface defined by any displacement map does not intersect with any other depth surface within the virtual 3D scene. A depth surface is defined as a mapping of the displacement map to a corresponding layer.

[0049] 1a shows a diagram of layers 102 in a multi-layer image. Each layer 102 represents a different depth value in the virtual scene. A texture value and a transparency value are assigned to each pixel in each layer 102. In this case, the first layer 102a corresponds to the front of the surface of the object 104 closest to the first layer 102a. Therefore, only the pixels corresponding to the front of the object 104 have texture, while the remaining pixels have a transparency value of 100%.

[0050] Similarly, the second layer 102b corresponds to the remainder of the front surface of the object 104, which is closer to the second layer 102b than to the first layer 102a. The fourth layer 102c corresponds to the background 106. However, the layers 102 do not exactly match the object 104, which can cause blurring due to parallax between the layers 102.

[0051] In this example, the layers 102 are evenly spaced and represented by planes, however the distance between the layers 102 (e.g., relative to the distance from the origin) may vary, and the layers 102 may, for example, be spherical instead of planar.

[0052] Figure 1b shows a diagram of layers 102 in a multi-layer image with additional depth. The additional depth can be defined by a displacement map for each layer 102. In this case, the layers 102 are warped toward the object surface, which significantly reduces blurring from disparity between the layers 102. However, multi-layer images with additional depth typically require significant computational resources to render (much greater than those required for a multi-layer image without additional depth). Therefore, it is proposed to impose constraints on the displacement maps and convey these constraints in the bitstream.

[0053] 2 shows an illustration of constraints on a depth surface 202. The constraints are selected to allow the client to use a simpler rendering algorithm (e.g., the Inverse Painter's Algorithm) and further reduce the time and / or computational resources required to render a multi-layer image. Knowing the constraints, the encoder and decoder can exploit them in multiple ways.

[0054] The constraints are defined in terms of a "depth surface" 202. A depth surface 202 is essentially a layer 102 in virtual space that has additional depth (i.e., with a displacement map applied to the layer 102). The constraints are as follows: The displacement map is generated so that the depth surfaces 202 do not intersect in the virtual space. A displacement map is generated such that the depth surface 202 is spaced apart in the virtual space by a specified margin 212 . Constrain the angle between any normal vector 206 from the depth surface 202 and the line of sight 204 from the virtual scene origin 208. - Limit the angle between any normal vector 206 from the depth surface 202 and any line of sight 204 through the defined viewing space 210. - Limit the curvature of the depth surface 202.

[0055] The spacing between layers 102 is generally proportional to the distance (r [m]) from the scene origin 208. That is, they are proportional to the normalized disparity (1 / r [m -1 ]) are equally spaced apart. The layers 102 do not overlap, and two layers 102 are spaced apart by a reference distance r n <r n+1 , the boundary r between the two depth surfaces 202 n, far is the geometric mean TIFF0007782576000001.tif914 or arithmetic mean It can be set to TIFF0007782576000002.tif922.

[0056] Each depth surface 202 can be defined by a tight box (or onion ring) in virtual space to include non-intersection constraints in the definition of the displacement map. The displacement map is calculated using delta depth values as TIFF0007782576000003.tif848 or normalized disparity TIFF0007782576000004.tif750. Both choices are acceptable and not significantly different, unless it is acceptable to place the final layer 102 very far away or at infinite depth (i.e., zero normalized disparity), in which case it may be necessary to use normalized disparity.

[0057] It is also possible for the depth ranges of consecutive depth surfaces 202 to overlap, but still constrain them to not intersect, e.g., if the minimum distance between the depth surfaces 202 is the layer depth value of the adjacent layer 102 (i.e., TIFF0007782576000005.tif721 and TIFF0007782576000006.tif828). In this case, the constraint is not clear from the displacement map definition and needs to be constrained separately. The non-intersection constraint may also need to be signaled separately when transmitted, for example by adding an indicator (i.e., a flag) to the bitstream.

[0058] The displacement map can be encoded as a luma image, and the luma values ​​can be linearly mapped onto the delta depth or delta normalized disparity range of layer 102.

[0059] If the depth surfaces 202 do not intersect and the viewing zone does not intersect with any of the layers 102, the depth order of the layers 102 is known, and therefore the depth surfaces 202 can be rendered front to back (the inverse painter algorithm). When the depth surfaces 202 intersect, much more analysis (e.g., splitting, sorting, etc.) is required to blend the depth surfaces together.

[0060] If the margin 212 between the depth surfaces 202 is known, then r n, far <r n+1,near In other words, the difference between the maximum depth value of a depth surface 202 and the minimum depth value of an adjacent depth surface 202 is greater than or equal to a predetermined margin 212.

[0061] The margins may be specified as a ratio where "zero" defines a multi-layer image with no additional depth and "one" defines a multi-layer image with non-overlapping but potentially touching depth surfaces 202. The margins 212 may be the same for all depth surfaces 202, or may be specified as, for example, maximum and minimum depth values ​​(for some or all depth surfaces 202). This may be specified for an individual layer 102 by explicitly providing TIFF0007782576000007.tif827.

[0062] Encoding and decoding errors may cause depth surfaces 202 that were not initially intersecting to intersect after the displacement map has been encoded, transmitted, and decoded. Therefore, defining margins 212 between depth surfaces 202 ensures that they do not intersect after encoding / decoding.

[0063] When the normal vector 206 from the depth surface 202 makes a large angle with the line of sight 204 from the origin 208, the depth surface 202 may stretch during rendering, which may degrade quality. Typically, the rendering algorithm must compensate for this stretching (e.g., when rendering multiview+depth images), which requires additional time and / or computational resources. Therefore, by limiting the angle, the overall computational resources and processing time required can be kept relatively low and consistent.

[0064] If the viewer of the multi-layer image is expected to remain at the origin 208, then it is only necessary to constrain the line of sight 204 from the origin 208. However, if the viewer is expected to move around, then it is necessary to restrict the line of sight 204 from any point within the viewing space 210.

[0065] The curvature of the depth surfaces 202 also affects the time / computational resources required to render a multi-layer image. Sharp changes in the depth surfaces 202 (i.e., high curvature) are difficult to render, while smooth surfaces (i.e., low curvature) tend to be relatively easy to render. If the curvature is limited to a maximum curvature for one or more of the depth surfaces 202, the required time / computational resources can be kept even lower and more consistent.

[0066] 3 shows a method for generating a multi-layer image 310 with constraints 302. Three-dimensional (3D) source data 304 of a scene is acquired, for example, from multiple cameras. Multiple layers are generated at different depths within the virtual 3D scene, with each layer corresponding to a particular layer depth value relative to the origin. Image data 308 is generated for each layer based on the 3D source data 304, where the image data includes texture data and transparency data.

[0067] A displacement map 306 is generated for each of the layers based on the 3D source data 304 and the depth values ​​of the layer, where the displacement map 306 defines the depth of each point on the corresponding layer relative to the layer depth. The generation of the displacement map 306 is also based on the constraints 302. The constraints 302 must include at least the non-intersection constraint described above.

[0068] The displacement map 306 and image data 308 can be rendered into a multi-layer image 310. Alternatively, the displacement map 306 and image data 308 can be encoded and sent to a client for rendering by the client.

[0069] Optionally, the client (i.e., decoder) has a filter that reconstructs the encoded displacement map 306 parameterized by the signaled constraints 302. For example, the decoder can use a blur filter or a morphological filter to remove abrupt transitions. The strength of the filter depends on the signaled constraints and may possibly vary locally.

[0070] 4 illustrates a method for generating, transmitting, and rendering a multi-layer image with constraints 302. The constraints 302 must include at least a "no intersecting layers" constraint. The constraints 302 may be obtained from user input or from a predetermined list of constraints. The 3D source data 304 may be obtained from one or more cameras and / or additional depth sensors. The constraints 302 and the 3D source data 304 may be transmitted to a server 401, which transmits the data necessary for the client 403 to render the multi-layer image.

[0071] The 3D source data 304 and constraints 302 are used by the server 401 to generate a coarse displacement map 402. The coarse displacement map 402 is then encoded (404) and decoded (406), and a reconstruction filter 408 is applied. The encoding 404 typically involves data compression or some type of data reduction to reduce the amount of data transmitted. Thus, when the encoded data is decoded in step 406, the resulting decoded data is not equal to the original data before encoding 408 (e.g., is of lower quality).

[0072] The reconstruction filter 408 filters the decoded displacement map according to the constraints 302 to ensure that the constraints are applied to the final displacement map. The filter can manipulate the displacement map values, for example, to avoid crossing depth layers and reduce their depth gradients and curvatures. Thus, the combination of the encoding, decoding, and reconstruction steps allows for transmission of the displacement map within the constraints 302, allowing for transmission at a lower bit rate since only a coarse displacement map is input to the encoder 404, which can apply data reduction techniques.

[0073] The 3D source data 304 is also used to generate image data 308 (i.e., transparency and texture / color values) for layer 102. Thus, the coarse displacement map 402 and image data 408 may be encoded 404 and transmitted, for example, from server 401 to client 403. The texture and transparency data may be optimized based on the reconstructed displacement map.

[0074] The client 403 can then decode (406) the transmitted data and apply a reconstruction filter 408 to the coarse displacement map 402. A multi-layer image can then be rendered (410) from the filtered displacement map and the decoded image data.

[0075] For example, the server 401 (i.e., the encoder) can utilize the reconstruction filter 408 by transmitting a coarse displacement map 402 with coarse, abrupt transitions. It is more efficient for a video codec to encode only the DC values ​​of a coding block using a discrete cosine transform (DCT) (404). For example, when a block has uniform values, only one coefficient is needed to encode the block. When this is applied, the coarse displacement map 402 appears "stair-stepped." This is possible because the reconstruction filter 408 filters the data to reduce depth gradients and curvature. This can be achieved using an appropriate low-pass filter.

[0076] This is more efficient to encode (404), and both the encoder and decoder can apply the specified reconstruction filter 408 to reconstruct the displacement map within the notified constraints. The encoder in the server 401 can then calculate appropriate values ​​for transparency and texture / color for each layer 102 based on the filtered displacement map. In this way, coding gain is achieved.

[0077] Additionally, the client 403 may have fallback rendering methods for rendering 410 outside the intended viewing space 210 where the constraints 302 are no longer guaranteed to be appropriate. This may involve scaling to reduce complexity, constraining the number of layers used in rendering, and / or approximating blending operations.

[0078] The layers 102 in a multi-layer image are typically sparse, with only a few (rectangular) patches typically stored for each layer. Each patch can be considered a separate sub-layer with separate parameters. This gives the encoder more room to optimize the representation of the content without increasing the complexity of decoding or rendering.

[0079] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprise" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0080] A single processor or other unit may fulfill the functions of several items recited in the claims.

[0081] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0082] The computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0083] When the term "adapted to" is used in the claims or the specification, it is meant to be equivalent to the term "configured to."

[0084] Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. 1. A method for generating a multi-layer image, comprising: receiving three-dimensional (3D) source data of a scene; generating a plurality of layers at different depths in the virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin; generating image data for each layer based on the 3D source data, the image data including texture data and transparency data; generating a displacement map for the layer based on the 3D source data and the depth values ​​of the layer, the displacement map defining a depth of each point on a corresponding layer relative to the layer depth; A method in which the generation of the displacement map is constrained so that the depth surface defined by the displacement map does not intersect with other depth surfaces in the virtual 3D scene, and a depth surface is defined as a mapping of the displacement map onto a corresponding layer.

2. 2. The method of claim 1, further comprising generating maximum and minimum depth values ​​for depth surfaces with corresponding displacement maps, wherein the generation of the displacement maps is further constrained such that the difference between the maximum depth value of a first depth surface and the minimum depth value of an adjacent second depth surface is greater than or equal to a predetermined margin.

3. 3. The method of claim 1, further comprising determining a plurality of first angles between a line of sight from the origin and a normal perpendicular to the depth surface, wherein generating the displacement map is further constrained such that the first angles are less than or equal to a first predetermined angle.

4. 3. The method of claim 1 or 2, further comprising determining a plurality of second angles between a line of sight from an observation space and a normal perpendicular to a depth surface having a corresponding displacement map, the observation space defining a space from which the multi-layer image can be observed, and generation of the displacement map further constrained so that the second angles are less than or equal to a second predetermined angle.

5. The method of claim 1 or 2, wherein the generation of the displacement map is further constrained such that the curvature at any point on the depth surface defined by the displacement map is less than or equal to a predetermined curvature value.

6. generating a displacement map, generating a coarse displacement map based on the 3D source data, wherein generation of the coarse displacement map is unconstrained; encoding the coarse displacement map; decoding the encoded coarse displacement map; generating a reconstruction filter based on the decoded coarse displacement map and a set of constraints; The method of claim 1 or 2, wherein the displacement map is based on the coarse displacement map and the reconstruction filter.

7. The method of claim 1 or 2, further comprising the step of separating a layer into a plurality of sub-layers based on the image data of the layer, and wherein a displacement map is generated for a sub-layer based on the image data corresponding to the sub-layer.

8. 1. A method for rendering a multi-layer image, comprising: obtaining a plurality of layers, a plurality of displacement maps and image data suitable for generating a multi-layer image; - obtaining an indication of a constraint, the indication of the constraint including information about the constraint by which the displacement map was generated, the indication of the constraint informing that a depth surface defined by the displacement map does not intersect with any other depth surface in the virtual 3D scene, a depth surface being defined as a mapping of the displacement map onto a corresponding layer; determining a rendering method based on the constraint indicator; and rendering the multi-layer image based on the layers, the displacement map and the image data using the determined rendering method.

9. Obtaining the displacement map obtaining a coarse displacement map corresponding to the displacement map generated without constraints; Obtaining a reconstruction filter; and applying the reconstruction filter to the coarse displacement map; The method of claim 8, based on

10. A computer program product, when executed by a computing device comprising a processor system, causing said processor system to perform the method according to claim 1 or 8.

11. 1. A system for generating a multi-layer image, comprising: receiving three-dimensional (3D) source data of a scene; generating a plurality of layers at different depths in the virtual 3D scene, each layer corresponding to a particular layer depth value relative to an origin; generating image data for each layer based on the 3D source data, the image data including texture data and transparency data; generating a displacement map for the layer based on the 3D source data and the depth values ​​of the layer, the displacement map defining a depth of each point on a corresponding layer relative to the layer depth; a processor configured to execute The generation of the displacement map is constrained so that the depth surface defined by the displacement map does not intersect with other depth surfaces in the virtual 3D scene, and a depth surface is defined as a mapping of the displacement map onto a corresponding layer.

12. 12. The system of claim 11, wherein the processor is further configured to perform the step of generating maximum and minimum depth values ​​for depth surfaces with corresponding displacement maps, and wherein the generation of the displacement maps is further constrained such that a difference between the maximum depth value of a first depth surface and the minimum depth value of an adjacent second depth surface is greater than or equal to a predetermined margin.

13. 13. The system of claim 11 or 12, wherein the processor is further configured to perform the step of determining a plurality of first angles between a line of sight from an origin and a normal perpendicular to a depth surface, and wherein generation of the displacement map is further constrained such that the first angles are less than or equal to a first predetermined angle.

14. 13. The system of claim 11 or 12, wherein the processor is further configured to perform a step of determining a plurality of second angles between a line of sight from an observation space and a normal perpendicular to a depth surface having a corresponding displacement map, the observation space defining a space from which the multi-layer image can be observed, and generation of the displacement map is further constrained so that the second angles are less than or equal to a second predetermined angle.

15. 13. The system of claim 11 or 12, wherein generation of the displacement map is further constrained such that the curvature at any point on the depth surface defined by the displacement map is less than or equal to a predetermined curvature value.

Images