Signaling of visual content

Abstract

Concepts for signaling visual content are described. According to one aspect, a picture representing a projection of a volumetric representation onto a projection plane and a geometric representation of the projection plane are provided in a bitstream. According to another aspect, a region of a synthesized picture signaled in a bitstream is associated with a plurality of pictures of a plurality of projections of an object. According to another aspect, a region of a synthesized picture signaled in a bitstream is associated with a respective information type of a picture.

Description

Technical Field

[0001] Embodiments of this disclosure relate to bitstreams, such as video bitstreams, for signaling visual content. Other embodiments relate to means for providing bitstreams, such as servers for providing visual content. Other embodiments relate to means for processing bitstreams, such as client devices for processing and / or presenting visual content. Other embodiments relate to methods for providing bitstreams and methods for processing bitstreams. Some embodiments of the invention relate to segmented rendering signaling. Background Technology

[0002] Recent advancements in VR glasses, and especially mixed reality (XR) glasses (where the user's real-world environment overlaps with volumetric video objects or virtual objects), have led to the widespread distribution and consumption of a new media type known as volumetric video objects, such as... Figure 1 As shown, a bitstream representing a volumetric video object is sent to a client for consumption. This data can be represented in several ways, such as point clouds and 3D meshes. A point cloud consists of a large number of points in 3D space that have color and other attributes, and these points together represent an object. Alternatively, this volumetric video object can also be represented as a textured 3D mesh.

[0003] When considering distribution and consumption, the two representations of such virtual objects share two common drawbacks: firstly, a considerable amount of data is required to store and transmit such virtual objects, and secondly, rendering volumetric video objects for presentation on user-side client devices involves considerable complexity.

[0004] One way to mitigate these drawbacks in these scenarios is to remove rendering from the client device and move it over the network, for example, by transferring rendering to a device such as... Figure 2 The example shown is an entity such as an edge cloud server. This edge cloud server has a network link to the client device, which introduces only very small communication latency. Therefore, offloading the computationally complex task of rendering views of such virtual objects to such an edge cloud server reduces the load on the terminal device. On the edge cloud server, these rendered object views are then encoded into video, and the encoded video bitstream is subsequently sent to the terminal device for rendering. Summary of the Invention

[0005] The object of this invention is to provide a concept for transmitting visual content that allows for an improved trade-off between low data rates, low workload on the client side, and high information content in the visual content.

[0006] A first aspect of the invention relies on the idea of ​​projecting a volumetric representation of an object onto a projection plane located in 3D space, and transmitting a geometric representation of the projection plane in the bitstream in addition to transmitting an image representing the projection. Representing the object by perspective projection onto a plane in 3D space, combined with a geometric representation of the projection plane signaled to it, allows the apparatus for processing the bitstream to rely on geometric information when integrating the transmitted image of the object into a 3D scene. Therefore, when rendering an image of a 3D scene with the object, movement of the user's viewpoint or position from which the 3D scene is rendered can be taken into account when presenting the transmitted image of the object. This reduces or avoids aberrations, allowing the user to experience more realistic rendered images. Furthermore, the availability of the geometric representation to the client device allows the client device to more accurately consider the interaction between the object represented by the transmitted image and other objects within the 3D scene, such as occlusion. Specifically, using planar projection provides a good trade-off between the small amount of data required to signal the geometric representation and the beneficial amount of information that allows for the processing of effects such as aberrations and occlusion. Therefore, the integration of a signaled image of the object into the 3D scene can benefit from geometric or spatial information without transmitting a volumetric representation of the object. This reduces the amount of data sent and avoids the need for rendering volume representation on the client side.

[0007] An embodiment of the first aspect provides a bitstream for signaling visual content. The bitstream includes a representation of an image. For example, the representation may include a texture map and / or other images. The image represents a volumetric representation of an object projected onto a projection plane. For example, the volumetric representation may include a point cloud representation or a mesh representation of the object. The projection plane is located in 3D space (e.g., the 3D space in which the object resides). The projection of the object's volumetric representation onto the projection plane is viewed from a viewpoint in 3D space. For example, 3D space may be considered to contain the object, the projection plane, and the viewpoint. In other words, the viewpoint may be considered in the example to define the origin of 3D space. In some examples, the projection onto the projection plane may be a plane projection along the plane's normal through the viewpoint. The bitstream also includes a geometric representation of the projection plane in 3D space.

[0008] According to an embodiment, the geometric representation indicates the position and / or orientation of the projection plane in 3D space. Having this information allows the client device to consider the deviation between the current user position and the view position, and to perform object projection from the view position, for example, by wrapping an image. Furthermore, this information allows the client device to determine whether the image is occluded or occluded by another image relative to other objects in the 3D scene into which the image can be integrated.

[0009] Other embodiments of the first aspect provide an apparatus and method for providing a bitstream (e.g., a video bitstream). The method includes obtaining (e.g., receiving or exporting) an image representing a volumetric representation (e.g., a point cloud representation or mesh representation) of an object projected onto a projection plane located in the 3D space from a viewpoint at a view location in 3D space. The method also includes providing a representation of the image and a geometric representation of the projection plane in 3D space within the bitstream. The apparatus is configured to perform the steps of the method.

[0010] Other embodiments of the first aspect provide an apparatus and method for processing a bitstream (e.g., a video bitstream). The method includes acquiring a representation of an image from the bitstream, the image representing a volumetric representation of an object (e.g., a point cloud representation or a mesh representation) projected onto a projection plane located in the 3D space from a viewpoint at a view location in 3D space. The method also includes acquiring a geometric representation of the projection plane in 3D space from the bitstream. The apparatus is configured to perform the steps of the method.

[0011] A second aspect of the invention relies on the idea of ​​signaling a composite image comprising multiple images representing projections of multiple corresponding objects onto multiple corresponding projection regions within a bitstream. The idea of ​​signaling the projection of one or more objects within a composite image structure offers the advantage of efficiently packing and processing projections of a flexible number of objects. For example, the composite image can be processed as images of specific time frames, such that, for example, in scenarios where objects and / or projection planes can move, a series of projections are provided in the bitstream. The composite image allows the projections of multiple objects to be associated with a common time frame, thus allowing joint processing, for example, when deciding whether to use or ignore projections of a specific time frame. More generally, signaling the projection of multiple objects within a composite image reduces signaling overhead by providing the possibility of signaling the information required for decoding the images, where this information refers to all images at the composite image level. Therefore, the signaling representation of multiple objects in a composite image allows the bitstream to be implemented as a video bitstream, enabling the efficiency of the video codec to be used for volumetric rendering projections, for example, sent from a server to a client.

[0012] According to an embodiment of the second aspect, a bitstream, such as a video bitstream, is provided for signaling visual content. The bitstream includes representations of composite images, which comprise multiple images, such as one or more images. For example, the composite images may provide a structure for signaling one or more images. Each image is represented by one or more regions of the composite image. The images represent projections of multiple corresponding volume representations (e.g., point cloud representations or mesh representations) of multiple corresponding objects onto multiple corresponding projection regions. For example, each image may represent a projection of a corresponding volume representation onto a corresponding projection region, where the corresponding volume representation represents a corresponding object. The projection onto the corresponding projection region may be a projection from a viewpoint (e.g., a common viewpoint). In the example, the projection region may be a planar or curved region. The bitstream also includes a region projection indication that associates one region of the composite image with one of the images.

[0013] According to other embodiments of the second aspect, an apparatus and method are provided for providing a bitstream (e.g., a video bitstream). The method includes obtaining (e.g., receiving or exporting) a composite image comprising a plurality of images. Each image is represented by one or more regions of the composite image. The images represent projections of multiple corresponding volume representations (e.g., point cloud representations or mesh representations) of multiple corresponding objects onto multiple corresponding projection regions. The method also includes providing a representation of the composite image and a region projection indication in the bitstream, the region projection indication associating one of the regions of the composite image with one of the images. The apparatus is configured to perform the steps of the method.

[0014] According to other embodiments of the second aspect, an apparatus and method for processing a bitstream (e.g., a video bitstream) are provided. The method includes obtaining from the bitstream a representation including a composite image, the composite image comprising a plurality of images. Each image is represented by one or more regions of the composite image. The images represent projections of multiple corresponding volume representations (e.g., point cloud representations or mesh representations) of multiple corresponding objects onto multiple corresponding projection regions. The method also includes obtaining from the bitstream a region projection indication that associates one region of the composite image with one of the images. The apparatus is configured to perform the steps of the method.

[0015] A third aspect of the invention relies on the idea of ​​signaling multiple information types of an image within a composite image, such that one of multiple regions indicates multiple samples of one of the information types of the image. Signaling multiple information types in a composite image provides an efficient way to send a flexible number of information types of the image. For example, the image may include one or more samples of each of the multiple information types, such as texture maps (e.g., chroma and / or luminance channels), depth maps, transparency masks, reflection maps, and semi-transparency maps. Using a composite image structure can reduce signaling overhead compared to signaling individual images, because information referring to the image as an entity can be signaled at the level of the composite image. Furthermore, a flexible number of information types can be signaled without changing multiple composite images, thus keeping the bitstream structure simple and / or allowing the integration of signaling a flexible number of information types using video streaming codecs (e.g., AVC, HEVC, or VVC).

[0016] According to an embodiment of the third aspect, a bitstream, such as a video bitstream, is provided for signaling visual content. The bitstream includes a representation of a composite image comprising multiple regions. The composite image indicates an image comprising corresponding multiple sample values ​​of various information types for a sample of the image. The bitstream also includes a region type indication that associates one of the regions of the composite image with one of the information types.

[0017] According to other embodiments of the third aspect, an apparatus and method are provided for providing a bitstream (e.g., a video bitstream). The method includes obtaining (e.g., receiving or exporting) a composite image, the composite image comprising a plurality of images. The composite image indicates images comprising corresponding plurality of sample values ​​of various information types for samples of the images. The method also includes providing a representation of the composite image and a region type indication in the bitstream, the region type indication associating one region of the composite image with one of the information types. The apparatus is configured to perform the steps of the method.

[0018] According to other embodiments of the third aspect, an apparatus and method for processing a bitstream (e.g., a video bitstream) are provided. The method includes obtaining from the bitstream a representation including a composite image, the composite image comprising multiple regions. The composite image indicates an image comprising corresponding multiple sample values ​​of multiple information types for samples of the image. The method also includes a region type indication that associates one of the regions of the composite image with one of the information types. The apparatus is configured to perform the steps of the method.

[0019] Other embodiments of the invention combine two or all of the first, second, and third aspects. Further embodiments are described in the dependent claims. Attached Figure Description

[0020] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings, in which:

[0021] Figure 1 Examples of application scenarios for the embodiments are shown.

[0022] Figure 2 An exemplary implementation framework of the embodiment is shown.

[0023] Figure 3 An apparatus for providing a bit stream according to an embodiment is shown.

[0024] Figure 4 An apparatus for processing a bit stream according to an embodiment is shown.

[0025] Figure 5 The projection of an object according to an embodiment is shown.

[0026] Figure 6 A bitstream according to an embodiment of the first aspect is shown.

[0027] Figure 7 An example of a geometric representation indicating three vertices is shown.

[0028] Figure 8 Another example of the geometric representation of the projection plane is shown.

[0029] Figure 9 A bitstream according to an embodiment of the second aspect is shown.

[0030] Figure 10 A bitstream according to an embodiment of the third aspect is shown.

[0031] Figure 11 A bitstream according to an embodiment combining the second and third aspects is shown.

[0032] Figures 12a to 12e Examples of various information types according to embodiments are shown.

[0033] Figure 13 An example is shown where the image covers the entire user viewport. Detailed Implementation

[0034] In the following description, embodiments are discussed in detail; however, it should be understood that the embodiments provide many applicable concepts that can be embodied in various encodings of visual content. The specific embodiments discussed are merely illustrative of specific ways of implementing and using the inventive concepts and are not intended to limit the scope of the embodiments. In the following description, numerous details are set forth to provide a more thorough explanation of embodiments of this disclosure. However, it will be apparent to those skilled in the art that other embodiments can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the examples described herein. Furthermore, unless otherwise specifically indicated, features of the different embodiments described herein can be combined with each other.

[0035] In the following description of the embodiments, the same or similar elements or components having the same function are given the same reference numerals or identified by the same name, and repeated descriptions of elements given the same reference numerals or identified by the same name are generally omitted. Therefore, the descriptions provided for elements having the same or similar reference numerals or identified by the same name are interchangeable or can be applied to each other in different embodiments.

[0036] Figure 1 An example application scenario of the present invention is illustrated, wherein a volumetric video object 20 is placed in a user view 94 of a user 80. The user view 94 is a view of a real-world environment 90, which may include a background and real-world scene objects 92. For this purpose, the user can wear XR glasses 120, which can overlay the user's view with a rendered representation 38 of the volumetric video object 20. The rendered representation 38 can be provided to the XR glasses by a client device connected to an edge cloud server 110. The user view 94 may also be referred to as the user viewport 94. The edge cloud server 110 may have already stored a volumetric representation 22 of the video object 20. The edge cloud server 110 provides a bitstream 101 indicating the video object 20, and the client device can obtain the rendered representation 38 based on the bitstream 101. As described in the introduction, the bitstream 101 typically includes the volumetric representation 22, such that the client device needs to render the volumetric representation to derive the rendered representation 38. In other words, the bitstream 101 can send a volumetric video object.

[0037] Figure 2 It shows in Figure 1 The exemplary implementation framework of the present invention is used in the context of the illustrated scenario. According to... Figure 2In this implementation, the volumetric video object 20 is rendered by the edge cloud server 110. To this end, the edge cloud server 110 renders the object 20 from the perspective of the virtual camera 26 to obtain a rendered view 224 of the object 20. Instead of sending the entire volumetric representation, the edge cloud server 110 can send the rendered view 224 within the encoded video bitstream 201. For the rendering of the volumetric object 20, the edge cloud server 110 can consider client-side information 202, such as information about the position of user 80 in the real-world scene 90, the position of object 20 in the real-world scene, the distance between user 80 and object 20, and the user's focus. Client-side information 202 can be provided by a client device 212, which processes the bitstream 201 and provides a rendered representation 38 to the XR glasses 120. For example, the rendered representation 38 can be provided as a texture plane with the rendered view 224, which can be placed within the user view 94 based on the position of object 20 in the real-world scene 90. Therefore, in Figure 2 In the implementation shown, a rendered view of object 20, i.e., a picture representation, is sent in bitstream 201. Therefore, a video codec can be used to encode the rendered view 224 into bitstream 201, which can thus be formatted as a video bitstream.

[0038] Processing such encoded video bitstreams on terminal devices typically consumes fewer resources because these devices are likely equipped with energy-efficient hardware for video decoding. Furthermore, the high coding efficiency of the latest generation of video codecs can significantly reduce the required transmission bitrate compared to the bitrate of a point cloud represented by a grid. It should also be noted that when sending volumetric video data in its raw format (e.g., a grid or point cloud), a large amount of data may ultimately not be presented to the user, for example, due to occlusion when viewing the projection from a single viewpoint. Therefore, sending only the user's view of the object can omit sending unused information to the client, as is done in viewport-dependent streaming of 360-degree video.

[0039] The rendered view can be accompanied by other auxiliary information (e.g., alpha or transparency masks, semi-transparent masks, depth maps, or other material information) to assist rendering, helping to achieve a more realistic presentation on the client side. This includes removing the background of rendered objects or creating additional lighting or reflection effects to correctly integrate them into the scene as seen by the user. The latest XR headsets can even be equipped with sensors to capture eye movements and focus in the scene to reconstruct the viewer's depth of field; that is, depending on the viewer's focus, objects may appear sharp or blurry. Such a system would require the encoded video bitstream to be a combination of multiple forms of information. Such a system can even combine information from multiple objects into a single bitstream. For example, when system design and server resources allow, this combination could be the result of co-rendering objects on the same cloud server instance.

[0040] However, this combination of required information can be broken down into separate parts on the client side. It is also expected that the video bitstream sent from the server to the client in these systems is self-contained and can be reused when consumed outside the original application, at different times, or on different non-XR devices. Having a self-contained bitstream format also allows for more efficient design of XR systems using information provided within the bitstream, rather than replicating this information on a separate control channel, which is considered beneficial. Therefore, current state-of-the-art systems lack video bitstream signaling to address the problems of cloud-based rendering systems for volumetric video. Specifically, there is a need to identify what types of regions exist within the bitstream and how to process them, for example:

[0041] 1) Encode which regions in the image are different and where they are located in the image plane, both spatially and temporally.

[0042] 2) Are (some) samples of the region intended to be rendered directly (to render the texture view) or are they intended to render subsequent processing of the texture view (alpha, depth)?

[0043] 3) How to recreate objects in 3D space when consuming in an XR scene.

[0044] Several closely related techniques exist at this point. For example, for projected 360-degree video (even if the coverage area is less than 360 degrees), bitstream signaling exists in AVC, HEVC, and VVC in the form of projection SEI messages (cube graph, equal rectangle SEI message) and a packing format that maps decoded samples to region-level packing of the projected image. However, these methods lack indication of the correct projection characteristics and cannot properly describe other properties associated with volumetric content (or rendered view), such as alpha, depth, or material information.

[0045] Figure 3 An apparatus 10 according to an embodiment is shown. The apparatus 10 is configured to provide a bit stream 14. Figure 4 An apparatus 12 according to another embodiment is shown. The apparatus 12 is configured to process bit stream 14. Figure 3 and Figure 4 Devices 10, 12, and bitstream 14 should be understood as a framework within which embodiments can be implemented. Note that, as Figure 4 The bitstream 14 processed by device 12 shown can correspond to, for example, Figure 3 The bitstream 14 shown is provided by device 10; however, this is not always the case. For example, other devices may be adapted to the bitstream provided by device 10, or a portion of the bitstream provided by device 10 may be selected to provide the bitstream to be processed by device 12. In other words, Figure 4 The bitstream 14 can be based on Figure 3 The bitstream 14, or can correspond to Figure 3 Bitstream 14.

[0046] According to an embodiment, bitstream 14 is a video bitstream. For example, bitstream 14 may conform to a video codec such as AVC, HEVC, or VVC.

[0047] For example, device 10 can be a server (e.g., an edge cloud server, such as...). Figure 1 or Figure 2 The device 10 may be an edge cloud server 110, or it may be part of a server, or it may be connected to a server. For example, the device 10 may run a server application. The device 10 may include data of visual content or data that can access visual content, and may provide a representation of the visual content in bitstream 14. To this end, the device 10 may include an encoder for encoding visual content into bitstream 14.

[0048] For example, device 12 may be a client device or part of a client device, or may be connected to a client device. Device 12 receives bitstream 14. In this example, device 12 may include a decoder for decoding visual content from the bitstream. Device 12 may provide visual content to other devices for rendering and / or for presenting visual content to a user. In this respect, device 12 may be part of a user device.

[0049] Figure 5 A projection of an object according to an embodiment is shown, which, for example, can be transmitted from device 10 to device 12 in bitstream 14. Figure 5 As shown, from the perspective of view position 26, the volume representation 22 of object 20 can be projected onto projection area 24 to obtain the projection 28 of object 20. For this purpose, projection area 24 can be located in the 3D space 30 where object 20 and view position 26 are located, for example, as per [reference to...]. Figure 3 and Figure 4 As described. In some embodiments, such as according to a first aspect of the invention, the projection area 24 is a plane. In other embodiments, such as according to a second and third aspect, the projection area 24 is not necessarily a plane, but may be a curved surface.

[0050] Projection can be performed from the viewpoint of view position 26. The viewpoint 26 for projecting the volumetric representation 22 can be selected based on the user's position in the 3D scene (e.g., a real-world scene). To this end, device 12 can provide information 16 about the user's viewpoint, such as a user of a user application to which device 12 can provide visual content obtained from bitstream 14. Information 16 may include, for example, one or more of the user's position and the user's viewing direction.

[0051] For example, the device 12 for processing the bitstream (e.g., a client device running a user application) can provide the device 10 for providing the bitstream (e.g., a server for providing visual content (e.g., a representation or view of one or more virtual objects, or a virtual world scene) with the user's position and / or viewing orientation in the user's 3D space (where the user is virtually or truly located). In other words, the user's 3D space can describe the real world or a virtual world presented to the user.

[0052] For example, a user application on a client device can be used to present a blend of real-world backgrounds and virtual objects to the user. Virtual objects can be freely placed by the user within their real-world scene, but they can also move around within the real-world scene driven by content animation (e.g., a walking character). In other words, the user application can place object 20 in the user 80's real-world view 94, for example, based on user interaction or based on a scene created by the application. In the example, one or both of object 20 and user 80 can move within a second world (real or virtual) 90. For example, the user application or client can provide server 12 with information 16 about the object's position in the user's 3D space, for example, relative to the user's position and / or viewing direction, or absolutely relative to the coordinate system of the user's 3D space. Alternatively or additionally, virtual object 20 can be placed in the user world 90 based on server-side knowledge of the real-world scene (e.g., placing a character on the real-world ground). For example, server-side knowledge of the real-world scene can be based on the user's position and / or viewing direction provided to server 10 by the user application or client 12. For example, the server can access a 3D model of the (virtual or real) world 90 where user 80 resides. Projection 224 can be obtained using view position / orientation 26 and the position of objects (which can be determined based on the 3D model of the world).

[0053] The user's position and / or projection viewing direction in 3D space can be described by coordinate system 32 (e.g., Cartesian coordinates and yaw / pitch). For example, the user position and / or user viewing direction can represent the user's position / view in the virtual world. In other examples, the user position and / or viewpoint can represent the user's position / view in the real world. The means for providing the bitstream (e.g., a server device) can apply the user position and / or user viewing direction, or an adapted version thereof (e.g., adapted based on the user's predicted movement in 3D space), as the view position and / or projection viewing direction, respectively, to project objects onto projection plane 24 (or projection area 24). In other words, the view position for projecting objects onto the projection plane can, for example, represent the user's position in the 3D space of the server application or server side, or the assumed / predicted user position, or the position from which the view on the object is to be determined. The projection (or viewpoint) viewing direction can correspondingly represent the user's viewing direction on the server side. The view position can be referred to, for example, as the camera position or camera viewing position of a virtual camera capturing an image of the object's projection. Similarly, the projection (or viewpoint) viewing direction can be referred to as the camera viewing direction, which can be described by the projection / camera viewpoint center vector or normal vector. For example, the user viewing direction from which the viewpoint viewing direction can be derived can be the normal vector to the user's view, for example, referred to as the user normal vector.

[0054] The coordinate systems of the user's 3D space and the object's 3D space (e.g., the server-side 3D space) can be the same or different (in the latter case, transformation information can be sent between the server and the client in one or both directions). For example, the coordinate system of the user's 3D space can be centered on the user's position in the user's 3D space and can move with the user. Optionally, the coordinate system can also move (rotate) in the user's viewing direction.

[0055] Figure 6 A bitstream 601 according to an embodiment of the first aspect is shown. Bitstream 601 may optionally be an example of bitstream 14. Bitstream 601 includes a representation 40 of an image representing a volume representation 22 of object 20 projected onto a projection plane 24 as a projection 28. The projection plane is located in 3D space. The projection 28 onto plane 24 is performed from a viewpoint 26. Therefore, according to an embodiment of the first aspect, the projection 28 signaled in bitstream 601 is a projection onto a plane. Bitstream 601 also includes a geometric representation 50 of the projection plane 24 in 3D space 30, such as... Figure 5 As shown. For example, geometric representation 50 indicates the relative position of view position 26 with respect to projection plane 24.

[0056] The advantage of using a plane as the projection area 24 is that the projection 28 of object 20 can depend on the user's position, and therefore on the view position 26, but can be independent of the viewing direction 82. Therefore, a change in the user's viewing direction does not necessarily require a new rendering of object 20 on the server side.

[0057] According to an embodiment, the geometric representation indicates the position of the projection plane 24 in 3D space 30. For example, this position can be indicated by an absolute position in 3D space 30, wherein the coordinate system 32 of 3D space can be the same as the coordinate system of the 3D space used for processing the bitstream (e.g., the coordinate system of the 3D scene 90 in which the user of the user application is moving). Alternatively, the position of the projection plane 24 can be indicated by a relative position relative to the view position 26. Alternatively or additionally, the geometric representation 50 can indicate the orientation of the projection plane 24 in 3D space 30. For example, this orientation can be indicated by an absolute orientation relative to the coordinate system of 3D space, or by a relative orientation, for example, relative to the viewing direction or relative to a plane perpendicular to the viewing direction.

[0058] According to an embodiment, the geometric representation 50 can indicate the position and / or orientation of the projection plane relative to the view location. For example, the projection plane can be indicated according to relative coordinates with respect to the view location 26, which reference coordinate system 32.

[0059] For example, geometric representation 50 includes coefficients of a plane equation describing the position and orientation of the projection plane 24. For example, for the general form of the plane equation ax + by + cz + d = 0, geometric representation 50 may include at least four coefficients a, b, c, d (see, for example, embodiment b described with respect to Table 2 below). In this case, the origin of the coordinate system may be chosen, for example, as the view position 26, such that the plane equation defines the plane 24 relative to the view position.

[0060] According to an embodiment, the geometric representation 28 indicates the position and / or orientation of the projection plane 24 relative to the absolute coordinates of the 3D space 30, and the view position 26 is represented by the absolute coordinates of the 3D space 30.

[0061] For example, geometric representation 50 includes coefficients of a plane equation describing the position and orientation of the projection plane 24, and these coefficients further describe the coordinates of the origin of the coordinate system 32 to which the plane equation references. In later examples, for example for a plane equation of the form a(x-x0)+b(y-y0)+c(z-z0)=0, where x0, y0, and z0 may describe the origin of the coordinate system, the geometric representation may include, for example, at least six coefficients (see, for example, embodiment c described with respect to the following 2 tables).

[0062] Figure 7A geometric representation of the projection plane 24 according to an embodiment is shown. According to this embodiment, the geometric representation 50 indicates at least three points in the 3D space 30, such as three points located in the projection plane 24. For example, these three points may define the vertices of a rectangular projection plane, such as... Figure 7 The fourth vertex, P3, can be derived by mirroring P1, thus eliminating the need for signaling.

[0063] In other words, in one embodiment, the vertices of the plane (e.g., three points P0 to P2, where P...) are provided in the perspective projection SEI message. n =(x n ,y n ,z n In order to define such Figure 7 Instead of providing a plane equation with horizontal and vertical dimensions, a polygon is shown. The fourth point P3 can be derived by mirroring P1 on axis P0P2, or explicitly notified by a signal. This representation of the plane is easier to handle in client-side rendering because the polygon can be directly reconstructed from the points used as vertices. This approach also has no restrictions on plane placement or location.

[0064] Figure 8 An example of a geometric representation 50 of a projection plane 28 according to another embodiment is shown. The geometric representation 50 may indicate the position and / or orientation of the projection plane 24 relative to the viewing direction 82. For example, the projection plane 24 may be indicated by a vector pointing from the view position 26 to the plane 24 (e.g., the center or edge of the plane 24). The direction of the vector 52 may be defined relative to the viewing direction 82, and the length of the vector 52 may define the distance between the view position 26 and the projection plane 24. Figure 8 In the description, for simplicity, it is assumed that the user's viewing direction and the server's viewing direction are the same. However, as mentioned above, coordinate transformations and / or offsets (e.g., time offsets) may be considered.

[0065] according to Figure 8 For example, projection plane 28 is perpendicular to vector 52 between view position 26 and projection plane 24, and vector 52 is in Figure 8 The projection 28 is indicated by the projection onto the projection plane 24.

[0066] Therefore, geometric representation 50 can indicate the distance between view position 26 and projection plane 24, such as defining the length of vector 52.

[0067] Vector 52 can, for example, point to a specific point on the projection plane 24, such as the center, vertex, or corner. Figure 8In the example, where the projection plane is oriented perpendicular to vector 52, a small number of parameters may be sufficient to signal the geometric representation 50.

[0068] like Figure 8 As shown, the geometric representation 50 can indicate the orientation 54 of vector 52 relative to the user's viewing direction 82, also known as the user normal vector, for example, used to position projection 28 within the user's field of view 84. Furthermore, the geometric representation can indicate the distance between view position 26 and projection plane 24, based on which projection 28 can be positioned within the user's field of view. For example, embodiment a) described with respect to Table 2 is an example of indicating the distance and orientation 54 in the geometric representation 50.

[0069] In other words, such as Figure 8 As shown, on the edge cloud server side, a rendered camera view 28 can be obtained by projecting a volumetric video object 20 onto a plane within the field of view 25 of a virtual camera 26. The rendered view 28 can be sent to the client side and integrated into the user's field of view 84 in the 3D scene on the client side.

[0070] According to an embodiment, vector 52 is not perpendicular to projection plane 24, but projection plane 24 may be tilted relative to a normal plane perpendicular to the line (e.g., vector 52) between the view position 26 and the point describing the position of the projection plane (e.g., the center or edge of the projection plane). In other words, the projection plane may be tilted relative to... Figure 8 The plane shown is tilted.

[0071] Therefore, in this embodiment, geometric representation 50 indicates the orientation (e.g., tilt) of projection plane 24 relative to the straight line between the view position and the projection plane. This straight line may coincide with vector 52. In this example, the straight line may be defined as the guide line of plane 24 passing through the normal of view position 26. Signaling the orientation of plane 24 allows projection 28 to accurately adapt to the movement of the user position relative to view position 26 and allows for accurate accounting of occlusion or other object interactions.

[0072] In other words, in the so-called planar mode, the volume object 20 is rendered as a texture of the plane 24 in 3D user space. For example, when the user changes his viewport 82 by, for example, changing the yaw, the server-side rendering is unaffected because only the user orientation, not the user position, changes relative to the object.

[0073] As part of this invention, sample mapping can be based on perspective projection (or zero projection or planar projection), i.e., the projected image describes a plane in 3D space, rather than a sphere in an equivalent rectangular projection of a cube, as in the case of using cube map projection.

[0074] In all the foregoing embodiments, the geometric representation 50 may optionally indicate the size of the projection plane 24, for example, the first and second dimensions of the projection plane, such as the horizontal and vertical dimensions, for example, the parameters plane_hor_size and plane_ver_size as described below. Specifically, regarding Figure 8 In the described embodiment, the boundary of the projection plane 24 may be defined by the size of the projection plane 24 measured from the point pointed to by the vector 52.

[0075] Furthermore, according to an embodiment, bitstream 601 may include information about view position 26 and / or viewing direction, which, for example, describes the direction of the view from view position 26 used for projection 28. Therefore, when rendering a rendered image to present it to a user, device 12 can take into account the difference between the view position / viewing direction used for projection object 20 and the current user position / user viewing direction.

[0076] According to an embodiment, bitstream 601 may include supplementary information describing the bitstream, such as a Supplemental Enhancement Information (SEI) message, which includes a projection syntax structure for signaling the geometric representation, such as the perspective_projection syntax structure as described in Table 1 or Table 3. For example, if the bitstream is a video bitstream, such as according to AVC, HEVC, or VVC, the projection syntax structure can be signaled in the Supplemental Enhancement Information (SEI) message. Examples of syntax for signaling the geometric representation 50, applicable to VVC, are given below; see Tables 1 to 7.

[0077] In other words, perspective projection can be indicated via the Perspective Projection (SEI) message. As a first option, the SEI message indicates the planar position where a decoded image has been produced, i.e., the planar position where perspective projection has been performed on the object.

[0078] Table 1

[0079]

[0080] The `plane_params` syntax structure contains all the parameters required for client-side plane identification, localization, and orientation; in other words, the geometric properties of the plane. Table 2 provides examples of `plane_params`.

[0081] Table 2

[0082]

[0083] plane_id indicates the identifier of the plane indicated by the plane_params() syntax structure.

[0084] coord_scale determines the scale of the coordinate system used for a plane relative to the real world.

[0085] `num_plane_coeffs_minus1` incremented by 1 indicates the number of coefficients used to define the plane. `plane_coeff[i]` represents the parameters of the corresponding plane. Different implementations exist:

[0086] a) Three parameters, such as the distance between the camera / user and the object, and the orientation (yaw / pitch) relative to the user's viewing direction, or

[0087] b) The four coefficients a, b, ... of the general form of the plane equation ax + by + cz + d = 0

[0088] c, d, or

[0089] c) The six coefficients and coordinates a, b, c, x, y, z of the plane equation a(x-x0) + b(y-y0) + c(z-z0) = 0 in point form, where x0, y0, and z0 are the origin of the coordinate system, such as the user's or camera's position, or

[0090] `plane_hor_size` and `plane_ver_size` define the horizontal and vertical sizes of the plane around the points on the represented plane (e.g., derived from coefficients or additional signaling of the plane's center point or from the user's / camera's view center vector intersecting the indicated plane). Alternatively, this can represent the field of view (FoV) of the virtual camera used to depict the object.

[0091] Depending on which plane_coeff embodiment is used, the positioning of the indicated plane may be more or less constrained. For example, when only the distance and orientation of the plane are indicated, the plane is constrained to be oriented perpendicular to the user, such as... Figure 8 As shown. Higher degrees of freedom in the positioning plane can be advantageous when communication latency between the server and client causes a significant difference between the user's position on the client side and the rendering camera position on the server side (e.g., due to rapid user or object movement). When the client is aware of the communication latency, it can compensate for the latency with surrounding texture views and other corresponding information (alpha mask, depth mask, etc.) to mitigate visual effects. Furthermore, when rendering a stereoscopic view, higher degrees of freedom in the positioning plane can facilitate positioning the plane perpendicular to the viewing direction of the eyes.

[0092] Continue to follow the basis Figures 6 to 8The first aspect of the embodiment is generally described below. According to the embodiment, bitstream 601 may include corresponding representations 40 of a plurality of objects 20, which are one or more objects, each object being represented by a corresponding volume representation. More precisely, bitstream 601 may include representations of a plurality of images, each image representing the volume representation of one of the objects projected onto a projection 28 on one of a plurality of projection planes 24, the projection 28 being as shown in relation to... Figures 6 to 8 According to these embodiments, bitstream 601 includes a plurality of geometric representations 50, such as the geometric representation of each of a plurality of projection planes 24.

[0093] In other words, besides about Figure 6 In addition to the image mentioned, there may be one or more other images involving one or more other objects, as well as projections of these images onto the corresponding plane.

[0094] According to an embodiment, the bitstream is a multi-layered video bitstream, and multiple images are signaled in their respective layers of the bitstream. For example, the geometric representation of the corresponding projection plane can be signaled in the projection syntax structure. According to an alternative embodiment, as described with respect to the second aspect, multiple images are signaled by synthesizing an image. Note that the second aspect can be fully integrated with the first aspect using planar projection.

[0095] The perspective_projection syntax structure shown in Table 3 can represent an example of a projection syntax structure used to signal multiple geometric representations of a corresponding number of images.

[0096] In other words, in a typical scenario, the application under consideration consists of multiple virtual objects added to the scene. In this case, several planes can be used to project different objects, which are placed in different locations. Therefore, another option that can be applied is to have signaling for each region, indicating which plane it corresponds to for each region, especially when applying the region-level signaling described in the second aspect.

[0097] Table 3

[0098]

[0099]

[0100] Since the user's position can change over time, the projection plane 24 can change accordingly. Therefore, the projection plane 24 can be associated with a specific time instance. In the case that the bitstream 601 is a video bitstream, one or more projection planes 24 can thus be indicated based on each time frame (for the VVC example, an access unit of the video bitstream). Alternatively, each time frame or access unit can be associated with one or more geometric representations, making it unnecessary to signal the geometric representation for each time frame. In the case that the bitstream 601 includes representations 40 of multiple objects, the multiple geometric representations can be associated with one or more time instances respectively. For example, this association can be implicit, such as signaling the geometric representation 50 in an information syntax structure (e.g., an SEI message) associated with a picture or picture group of a time instance (e.g., associated with an access unit). As described above, one of the projection planes 24 is used to perform each of the projections 28; however, in the example, the projection plane 24 can be used for projections 28 or more objects. According to an embodiment, the bitstream does not have two geometric representations of equal size and / or position associated with the same time instance and indicating the corresponding projection plane. Therefore, if the projection plane 24 used for the projection 28 of two objects is the same, they can be associated with the same projection plane without having to notify the projection plane twice with signals.

[0101] In other words, in one embodiment, the requirement for bitstream consistency is that two planes not indicated in the Perspective_projectionSEI message for a time instance (i.e., access unit) have the same horizontal and vertical dimensions.

[0102] Alternatively or additionally, a bitstream may not have two geometric representations associated with the same time instance and the same identifier.

[0103] In other words, in another embodiment, the requirement for bitstream consistency is that two planes not indicated in the Perspective_projectionSEI message for a time instance (i.e., access unit) have the same plane_id value.

[0104] Continuing with the general description of the embodiments, the representation 40 of the image representing projection 28 can indicate the values ​​of samples of the image. According to the embodiments, for the samples of the image, the image includes corresponding values ​​of one or more of color information, texture information, transparency information, depth information, reflection information, and translucency information. In other words, the representation of the image can include an overlay of one or more maps such as a texture map, transparency map, depth map, reflection map, and / or translucency map. Note that these maps do not necessarily have the same resolution or number of samples. Beneficial signaling of representation 40, particularly if the representation indicates multiple information types, such as describing not only texture but also one or more of depth, transparency, reflectivity, and translucency regarding a third aspect, the third aspect can be fully integrated with the first aspect or additionally integrated with the second aspect. Signaling one or more of depth, transparency, reflectivity, and translucency can allow device 12 to integrate projection 28 more realistically into view 94 of user 80.

[0105] Bitstream 601 can be provided by means 10 for providing bitstream (e.g., according to...). Figure 3 The device 10 provides the image. Therefore, the device 10 can obtain an image representing the projection 28, and can provide an image representation 40 and a geometric representation 50 in the bitstream 601. For example, for this purpose, the device 10 can encode the image to obtain the image representation 40.

[0106] Device 10 can receive an image, or it can perform a projection of volume representation 22 to obtain an image. Naturally, the same applies to multiple images when encoding multiple images of multiple objects.

[0107] Device 10 can receive bitstream 16 (e.g., a feedback bitstream, which includes, for example, information about...) Figure 3 and Figure 4 The user position is obtained from the described information. The device 10 can use the user position to determine the view position, for example, by using the user position as the view position or by performing a coordinate transformation and / or by taking into account the user's predicted movement.

[0108] According to some embodiments, device 10 can obtain the position of object 20 in 3D space 30 from bitstream 16 (feedback bitstream) and determine projection 28 accordingly, for example, taking into account the relative arrangement between user position and object position. Alternatively or additionally, device 10 can also receive user viewing direction from bitstream 16 and obtain projection 28 by taking into account user viewing direction, for example, by determining viewing direction based on user viewing direction, such as using user viewing direction as viewing direction.

[0109] According to an embodiment, device 10 can obtain the user focus from bitstream 16 and use the user focus to determine an image, for example by filtering the image based on the relationship between the object position in 3D space and the user focus (e.g., blurring the image).

[0110] Furthermore, bitstream 601 can be processed by device 12, for example, as per [reference to...]. Figure 4 As described. Therefore, device 12 can acquire representation 40 and geometric representation 50 from the bitstream. For example, since geometric representation 50 or the image is described as including features or information, device 12 can be configured to acquire features or information from bitstream 601.

[0111] Device 12 can render a rendered image of a 3D scene in 3D space from the perspective of the current user position using geometric representation 50. As previously described, the user position can, for example, correspond to a view position in 3D space. For instance, view position 26 can be based on the user position sent by device 12 for processing the bitstream to the server (e.g., device 10 for providing the bitstream), or an updated position relative to a previous user position, such as corresponding to a view position in 3D space (e.g., view position 26), which may have already been used as a view position or used to determine the view position. For example, in an XR scene, the rendered image can be used as an overlay of the real world in user 80's view 94. To this end, device 12 can include images of one or more objects 20 in the rendered image.

[0112] According to an embodiment, device 12 can use geometric representations to determine the position of an object within a 3D scene and / or determine the contribution of an image to a rendered image. For example, a portion of the non-occluded part of the geometric representation 50 of an image contributes to the rendered image.

[0113] According to an embodiment, device 12 can compare view position 26 with the current user position. For example, bitstream 601 may include information about the view position. Device 10 can accordingly take into account the differences between these two positions when rendering the image, for example, by wrapping the image based on the positional differences.

[0114] Naturally, these embodiments of device 12 can be related to the following Figures 9 to 13 The described feature combination. Specifically, the apparatus 12 can combine the projections 28 of multiple objects within a rendered image. For example, the apparatus 12 can obtain images of multiple projections from a composite image, and / or can consider multiple information types from one or more images.

[0115] Figure 9A bitstream 901 for signaling visual content is illustrated according to an embodiment of the second aspect. Bitstream 901 may optionally be an example of bitstream 14, and / or may be a bitstream according to bitstream 601 and / or 1001. In an embodiment, bitstream 901 is a video bitstream. Bitstream 901 includes a representation 40 of a composite image 62, which includes (e.g., represents) multiple images, such as one or more images. Each image is represented by one or more regions 64 of the composite image. The one or more regions 64 are in... Figure 9 The image is illustrated by two regions (i.e., region 64a and region 64b) of an exemplary number. The image represents multiple corresponding volume representations 22 of multiple corresponding objects 20 projected onto multiple corresponding projection regions 24 as projections 28. For example, in... Figure 9 In the image, region 64a represents projection 28a, and region 64b represents projection 28b. Bitstream 901 also includes a region projection indicator 360 that associates one of the regions 64 of the composite image 62 with one of the images. For example, region indicator 360 associates each of one or more images represented by the composite image 62 with a corresponding region 64 within the composite image.

[0116] For example, the images in composite image 62 can have a uniform or individual size. The images can be positioned regularly or irregularly within composite image 62. The images can be positioned adjacent to each other or can be separated by samples (e.g., spaced samples). The portions of composite image 62 associated with region 64 can be non-overlapping.

[0117] Note, as regarding the first aspect, especially Figures 6 to 8 The descriptions of the projections 28 and how they are obtained, as well as the description of representation 40, provided may optionally be applied to embodiments of the second aspect. However, it should be noted that, according to the second aspect, the projection 28 is not necessarily a planar projection, i.e., the projection area 24 is not necessarily a plane, but rather, for example, a spherical area. Depending on the type of projection, projection can be performed from the perspective of the view location, as in the case of a planar projection described with respect to the first aspect. For example, a common view location can be used to obtain the projection. In other words, each of the projections 28 (e.g., 28a or 28b) can be a projection onto the projection plane 24 associated with that projection.

[0118] For example, the first region 64a may include sample values ​​of the first image, and the second region 64b may include sample values ​​of the second image. For one or more of the images representing projections 28a, 28b of the corresponding objects, the composite image 62 may include sample values ​​of one or more information types (see the third aspect). For example, region 64 of the composite image 62 may refer to a portion of the samples of the composite image 62. In other words, the composite image may be represented by a graph that stores the sample values ​​of the composite image's samples. Region 64 of the composite image may refer to the region of the graph that includes the corresponding samples of the composite image.

[0119] Therefore, an image may include a plurality of corresponding samples, each sample being associated, for example, with a location in the sample array of the composite image 62. A sample from one of the images may be associated with a location within region 64, which is associated with the image. For example, region 64 associated with an image includes the sample value for each sample; each region may include the sample value for each sample of the image associated with that region.

[0120] According to an embodiment, bitstream 901 includes a corresponding geometric representation 50 of projection region 24, and the corresponding geometric representation 50 is used to obtain projection 28, such as projections 28a, 28b. For example, in the case that the projection region is planar, the geometric representation 50 can be indicated as described with respect to the first aspect.

[0121] Therefore, bitstream 901 can indicate one or more geometric representations 50. For example, each of the geometric representations can be associated with an identifier such as the plane_id parameter in the perspective projection syntax structure described above.

[0122] Therefore, each of the projections 28 can have an associated geometric representation describing the projection region 24 of the corresponding projection. The association between the projection 28 and the geometric representation can be achieved by indicating the geometric representation of each projection (e.g., by an identifier). The geometric representation associated with the projection plane 24 of a particular projection 28 can be referred to as a predetermined geometric representation.

[0123] According to an embodiment, bitstream 901 includes supplementary information, such as an SEI message. The supplementary information may include a region syntax structure that signals the region projection indication 60. For example, in an example of a VVC-compliant bitstream, the region syntax structure may be a regionwise_packing (RWP) syntax structure, or the regionwise_packing_extension syntax structure (RWPE SEI message) or split_rendering_information syntax structure described below.

[0124] According to an embodiment, region projection indication 60 indicates one or more or all of the geometric representations 50 in the projection region 24. For example, region projection indication 60 may include one or more or all of the geometric representations 50. Geometric representations 50 not signaled in the region syntax structure may be signaled in an additional projection syntax structure (e.g., the projection syntax structure described with respect to the first aspect), which may be part of supplementary information. That is, for example, for a geometric representation 50 not signaled in region projection indication 60, region projection indication 60 may point to a geometric representation signaled in the projection syntax structure. Referring to an embodiment, for example, in Table 4, where rwp_rlp_plane_id[i] (where i specifies region 64) points to a geometric representation when the geometric representation (plane_parameters()) is not indicated in the region syntax structure. According to an alternative embodiment, all geometric representations are signaled in the projection syntax structure, and region projection indication 60 points to the corresponding geometric representation.

[0125] According to an embodiment of the interaction between the region projection indicator 60 and the projection syntax structure, the region projection indicator 60 indicates whether (or alternatively, whether not) a geometric representation of one of the regions 64 is signaled in the projection syntax structure, for example, the syntax element rwp_plane_params_present_flag[i] in Table 4. Furthermore, the region projection indicator 60 may optionally indicate, for example, via rwp_rlp_plane_id[i] in Table 4, which geometric representation of the region 64 is associated with in the geometric representation signaled in the projection syntax structure.

[0126] According to an embodiment, the region projection indicator 60 explicitly indicates the geometric representation 50 of each region in region 64, for example, by signaling the geometric representation 50 or by pointing to a geometric representation 50 in a projection syntax structure. Alternatively, the region projection indicator 60 may not necessarily explicitly indicate the geometric representation 50 of each region, but may indicate only the geometric representations of one or more regions. In this case, the region projection indicator 60 may indicate whether it includes explicit signaling, see, for example, plane_info_present_flag[i] in Table 4. For example, in the absence of explicit signaling, the geometric representation signaled for the previous region will be used for the corresponding region.

[0127] Figure 10A bitstream 1001 for signaling visual content is shown according to an embodiment of the second aspect. Bitstream 1001 may optionally be an example of bitstream 14, and / or may conform to bitstream 601 and / or 1001. In an embodiment, bitstream 1001 is a video bitstream. Bitstream 1001 includes a representation 40 of a composite image 62, which includes a plurality of regions 64. For example, the concept of a composite image may correspond to... Figure 9 The described concept. In Figure 10 In the example, multiple regions 64 are represented by two regions (i.e., regions 64' and 64") of an exemplary number. Figure 10 The composite image 62 indicates that the image includes corresponding multiple sample values ​​of various information types for the samples of the image. Furthermore, the bitstream 1001 includes a region type indicator 70 that associates one of the regions 64 of the composite image 62 with one of the information types. Note that the number of sample values ​​may be different or equal for each information type. Therefore, the regions may have different or equal sizes.

[0128] For example, one area in region 64 associated with one of the information types includes sample values ​​of the image of the associated information type. The association between region 64 and the information type can be signaled in region type indication 60.

[0129] Information types can include information about Figure 9 The types of information described. For example, multiple information types include one or more of color information, texture information, transparency information (e.g., alpha channel), depth information, reflection information, and translucency information.

[0130] According to embodiments, multiple information types include one, two, or all of a first type, a second type, and a third type of reflection information. The first type of reflection information indicates brightness that depends on the type of lighting (e.g., diffuse lighting or specular lighting) and / or on the intensity of the lighting. The second type of reflection information indicates the degree of reflection, for example, the amount of light reflected by the corresponding sample. The third type of reflection information indicates the type of reflection, such as diffuse or specular reflection. Therefore, the device 12 can render a rendered view based on the type or intensity of lighting actually applicable to the scene for the user 80. In the example, one, two, or all of the first to third types of reflection information include information about the dependence of the corresponding reflection information on, for example, the angle of incidence of the reflected light.

[0131] According to an embodiment, bitstream 1001 includes supplementary information, such as SEI messages, as described with respect to the first and second aspects. The supplementary information includes region syntax structures, such as those related to... Figure 9 The structure of the described type, the region syntax structure is notified by the signal indicating the region type 70.

[0132] The syntax region structure of the signal notification region type indicator 70 may include, for example, an indicator of key color background information, which is used to indicate one or more key colors (e.g., one or more color values ​​or ranges of color values) to be interpreted as transparency indicators, such that samples whose sample values ​​correspond to key colors can be interpreted as transparent.

[0133] According to an embodiment, the region syntax structure includes region-specific information of one or more information types associated with a region, wherein the region-specific information is used to interpret sample values ​​of the corresponding information type. That is, for example, for the information type associated with a region, the region-specific information can indicate how to interpret that information type. Therefore, the same information type can be interpreted differently for different regions, allowing for more flexible indication of region types. Alternatively, the region-specific information can be applied to all regions of that information type. For example, the corresponding region-specific information (e.g., transparency information) of at least one information type indicates how sample values ​​of the corresponding region will be interpreted according to the corresponding information type-specific information. For example, bitstream 1001 includes information type-specific information, for example, as part of information referring to a set of images or access units. Therefore, it is not necessary to signal the region-specific information separately for each region. Therefore, in an embodiment, bitstream 1001 includes at least one of region-specific information or information type-specific information for at least one information type (e.g., transparency information).

[0134] Figure 11 A bitstream 1101 for signaling visual content is shown according to an embodiment of the second aspect. Bitstream 1001 may optionally be an example of bitstream 14. Bitstream 1101 conforms to... Figure 10 and Figure 11 Bitstreams 601 and 1001. That is, bitstream 1101 is an example of bitstream 1001, and according to this example, the composite image 62 includes multiple images represented by corresponding regions 64 of the image, as shown in the example. Figure 10 As described. Therefore. Figure 11 Examples of how the second and third aspects can be combined are shown. For example... Figure 11 As shown, for the first image representing projection 28a, bitstream 1101 includes regions 64a' and 64a'", which contain sample values ​​of two information types of projection 28a. Similarly, for the second image representing projection 28b, bitstream 1101 includes regions 64b' and 64b'", which contain sample values ​​of two information types of projection 28b (e.g., but not necessarily the same two information types as those of projection 28a).

[0135] In addition, bitstream 1101 includes supplementary information 65, which includes a region projection indicator 60 and a region type indicator 70. Note that information about... Figure 9 and 10 All the features described.

[0136] In other words, for example, a first region (e.g., region 64a') may include sample values ​​of a first information type from the first image, and optionally, a second region (e.g., region 64a") may include sample values ​​of a second information type from the first image. Similarly, for the second image, a third region (e.g., region 64b') may include sample values ​​of a third information type (e.g., the first information type), and optionally, a fourth region (e.g., region 64b") may include sample values ​​of a fourth information type (e.g., the first information type, the second information type, or another information type). For example, regions of an image may have a uniform or individual size, i.e., the number of samples. For example, the first information type (e.g., texture mapping) may be signaled at a higher resolution than the second information type (e.g., reflection information or depth map).

[0137] In other words, based on the described projection (e.g., the projection of the first aspect), and based on any other projection (e.g., a non-planar projection), the RWP SEI message (e.g., the parameter region_type[i]) in Table 4 can be extended to the following RWPE SEI message for indication of the invention in this invention (e.g., an embodiment of the third aspect). Furthermore, perspective projection-related variants (embedded signaling in the RWPE or associating a region with a plane via plane_id) are represented by plane_info_present_flag[i] and the following if structures in Table 4 (e.g., an embodiment of the second aspect).

[0138] For example, depending on the implementation, the RWPE SEI message may indicate the region type of region i (e.g., the parameter region_type[i]) or it may indicate the region projection indication 60 of region i (e.g., associating region i with a projection region, such as a projection plane), or it may indicate both the region type and the projection region of region i (e.g., plane_info_present_flag[i] and the following if structures in Table 4).

[0139] In the example, the RWPE SEI may optionally also indicate one or more region-specific information for a region type (e.g., for region type 1, if (region_type == 1)).

[0140] Table 4

[0141]

[0142]

[0143]

[0144] For example, region_type[i] indicates the type of the packaged region, as shown in the example in Table 5. Figures 12a to 12e An example of a sample value graph for the corresponding information type is shown.

[0145] Table 5

[0146] 0 Rendering texture views, for example Figure 12a 1 Alpha or transparent mask, for example Figure 12b 2 Depth map information, such as Figure 12c 3 Reflected information, such as Figure 12d 4 Transparency information, such as Figure 12e 5-15 Reserved

[0147] Note that in the embodiments in Table 4, the second and third aspects can be implemented independently of each other, for example by skipping the region_type or plane_information parameter.

[0148] For example, when plane parameters are indicated in a perspective projection SEI, such as through the identifier rwp_rlp_plane_id[i] or other means, each region of the RWPE is associated with a specific plane, and the plane parameters are directly embedded in the RWPE syntax (e.g., region projection indication 60). In another embodiment, the requirement for bitstream consistency is that no two regions with equal region_type[i] values ​​have the same rwp_rlp_plane_id[i] value within the Regionwise_packing_extension SEI message. Therefore, embodiments in which the bitstream includes projection syntax structures and region syntax structures in supplementary information (e.g., as per [reference to...]) are used. Figure 9 In the described image 62, there are no two regions associated with the same geometric representation 50 and the same information type as the projection syntax structure.

[0149] In another embodiment (e.g., as regarding Figure 9 In the embodiments described in claims B3.27 and B1.8a, when there is no explicit signaling for the plane parameters or plane identifier of the region (e.g., in Table 4, plane_info_present_flag[i] can be used to indicate whether explicit signaling exists), the region inherits the plane parameters or plane identifier of the previous region within the RWPE syntax.

[0150] The syntax sent in the placeholder parameter ("[...] / / region-specific content") can be equivalent to the information in the Alpha Channel Information (SEI) message, for example, because it is used to interpret the encoded sample values ​​(how information is applied, such as by multiplying them with texture sample values, or by cropping and scaling the values). However, the indicated information applies only to the region represented as the Alpha mask and other regions associated with the same plane.

[0151] Therefore, the region syntax structure of bitstream 901 or bitstream 1001 may include region-specific information. This region-specific information may relate to one or more information types, as shown in Table 4. In this case, the region-specific information indicates how to interpret sample values ​​of the corresponding information type. Alternatively, the region-specific information may relate to one or more of regions 64. In this case, the region-specific information indicates how to interpret sample values ​​of the image associated with the corresponding region. In other alternatives, region type information may be specific to both the region and the information type. In this case, the region-specific information indicates how to interpret sample values ​​of the image associated with region 64, which includes sample values ​​of the corresponding information type.

[0152] In another embodiment, when the RWPE SEI message (e.g., a region syntax structure) contains a region of type alpha mask and the alpha channel information SEI message is associated with an AU, the information from the alpha channel information SEI message will be applied to the values ​​of the alpha mask regions representing all representations within the encoded image, see, for example, regarding... Figure 10 The information described is of a specific type.

[0153] In another embodiment (e.g., the embodiment described by claim B1.21; C1.10), when there is no separate region for providing a dedicated alpha mask, but the texture view is accompanied by a keyed background to allow removal of the background via chroma keying on the client side, additional information (such as a dedicated background color value) can be indicated in the RWPE message to support client-side processing of the texture region, see as regarding Figure 10 The description indicates the background information of the key color.

[0154] The aforementioned reflection information describes how the surface of a given depicted object generally reflects light. In one embodiment, this information is specific to the lighting conditions on the rendering (or capturing) side, i.e., which areas of the object's surface are brightened due to diffuse or specular reflection from the lighting applied during rendering / capturing, or which areas are bright without reflection. When the object is adjusted to the lighting conditions on the playback side, the client can utilize this information, for example, by reducing reflections on some parts of the object or emphasizing others.

[0155] In another embodiment, the reflection information depends on the geometry of the object. For example, sharp corners of the object are highlighted / emphasized (i.e., indicated as highly reflective) in a sample of the reflection information area, while planar areas of the object's surface may not be highlighted / emphasized.

[0156] In another embodiment, the reflection information depends on the material properties of various object regions; for example, a region may be indicated as highly diffuse or specularly reflective.

[0157] In another embodiment, the reflection information is enhanced with angle information, such as light-specific angle of attack.

[0158] Naturally, all the above embodiments regarding the type of reflection information can be combined to export a complete view of the object's reflection properties on the client side and apply corrections in any aspects deemed necessary.

[0159] For example, the foregoing embodiments can be used to implement the following: Figure 10 The description includes the first to third reflection information.

[0160] Depth information from samples in the corresponding region, combined with general planar position (i.e., distance to the user) and information about the user's focus, can be used to render a textured view of a volumetric video object (or a portion thereof) with appropriate depth blur on the client side or directly on the server side before encoding, so that it appears out of focus when the user focuses on a point at a different distance and the volumetric video object leaves the user's depth field.

[0161] Such as about Figures 9 to 11 As described, the region projection indicator 60 and / or the region type indicator 70 can signal in the region syntax structure that the association between the picture and region 64 and / or the information type and region 64 is established.

[0162] According to an embodiment, the region syntax structure further associates region 64 with a corresponding part of the composite image; that is, the region syntax structure can indicate which part or which samples of the composite image belong to a specific region. For example, the region syntax structure indicates one or more of the following: the location of the region within the composite image, the size of the region, and the arrangement of the regions. For example, the location can be indicated by the position of a specific sample, such as the top-left sample of the region, and the size can be represented by height and width. Table 4 provides an example of the region-level packing extended SEI message.

[0163] According to an alternative embodiment, the latter information (i.e., information about the position of region 64 within the composite image 62) can be signaled in a separate syntax structure (e.g., a region information syntax structure). A VVC region-level packing SEI message can be an example of a syntax structure that can be used as a region information syntax structure. For this case, an example of a region syntax structure that signals region projection indication 60 and region type indication 70 is the segmentation rendering information syntax structure described in Table 6.

[0164] According to the embodiment, in bitstreams 901 and 1001, the region information syntax structure of the synthesized image 62 precedes (either directly or separated by other information) the region syntax structure of the synthesized image.

[0165] According to an embodiment, the region syntax structure and / or region information syntax structure can be signaled for one or more composite images 64, i.e., associated with one or more composite images of the bitstream. For example, composite image 62 can be part of an access unit, i.e., associated with a time instance (or a set of images for a time instance). The region syntax structure and / or region information syntax structure can be signaled for an access unit or a set of images (or within an access unit or a set of images).

[0166] According to an embodiment, the number of region indicators in the region information syntax structure (e.g., sri_num_packed_regions, which indicates the number of regions described by the region information syntax structure) is equal to the number of region indicators in the region syntax structure (e.g., rwp_num_packed_regions, which indicates the number of regions described by the region syntax structure).

[0167] In other words, by repeating the number of regions in the signaling and further indicating only the syntax elements of the new invention as follows, the alternative embodiment of the above invention serves as an additional SEI message (referred to as a Segmented Rendering Information (SRI) SEI message) designed from the original RWP SEI message. That is, the syntax signaled in the SRI SEI message is associated with the regions indicated in the RWP SEI message. In this embodiment, the requirement for bitstream consistency is that the Segmented Rendering SEI message follows the corresponding original RWP SEI message in bitstream order within the access unit. Furthermore, the requirement for bitstream consistency is that the value of sri_num_packed_regions in the SRI SEI message is equal to the value of rwp_num_packed_regions in the corresponding RWP SEI message.

[0168] Table 6

[0169]

[0170]

[0171] Other embodiments will include region location and planar information, thus eliminating the need for additional SEI messages (e.g., RWP). The syntax can be as shown in Table 7. Therefore, the segmentation rendering SEI message in Table 7 can be an alternative example of a region syntax structure that includes information about the location of the region in the composite image 62.

[0172] Table 7

[0173]

[0174]

[0175] The syntax for describing a region is represented by the syntax elements packed_region_width[i], packed_region_height[i], packed_region_top[i], and packed_region_left[i]. The options shown above indicate regions explicitly signaled to correspond to a specific plane or region type. It is required that the regions defined in this SEI message are non-overlapping.

[0176] exist Figures 3 to 1 In the foregoing description of 2, the size (or corresponding size) of the image representing the projection 28 of object 20 can depend on the size of projection 28, or can be adapted to the size of projection 28, and therefore can depend on the size of the object and / or the desired resolution. That is, the size of the image can be independent of the size and / or aspect ratio of the user viewport. Since objects typically do not fill the entire user viewport, but only a portion of it, adapting the image size to the object size avoids signaling unnecessary samples. Furthermore, the geometric representation 50 allows the projection 28 to be positioned within the user viewport 94. This method has the advantage that no new rendering of projection 28 is required when the user changes position.

[0177] Figure 13 An alternative embodiment of an image representing projection 28 is shown. According to this embodiment, one or more images cover the user viewport. For example, according to this embodiment, the size of the image may be independent of the size of object 20 or projection 28. With the image covering the entire viewport, it can serve as an overlay on the user viewport. Furthermore, the image can represent projections of multiple objects. Note that according to these embodiments, the projection may optionally be a planar projection. The features described with respect to the second and third aspects may also optionally be used with these embodiments of the image.

[0178] like Figure 13 As shown, the volumetric video object 20 can be projected onto a plane, and the captured plane depicts one or more objects in the user's viewport. An image representing the rendered view is sent from server 10 to client 12. Client device 12 can provide projection 28 as a texture plane for the rendered view depicting virtual objects in the user's viewport.

[0179] In other words, in this so-called overlay mode, the volume object 20 is rendered as a texture that overlays the plane of the user's viewport. For example, when the user changes his viewport by changing the yaw in this method, the server-side rendering is affected by the object and changes its position on the rendering viewport overlay (e.g., from the left side to the right side).

[0180] This section can be similar to a planar model, as it should be described as the location of the already calculated coverage, i.e., the location of the "camera" (e.g., the server's perceived location of the observer, position 26), and the viewing direction and FoV should be defined. Using this information, along with other auxiliary information, occlusion between virtual and real-world objects can be correctly estimated on the client side to hide the corresponding "occluded" portions of the rendered coverage. Furthermore, if the user is not in the same location (orientation, position) as the virtual camera on the server side, certain corrections (around) can be performed on the texture view (and the corresponding alpha mask, etc.) to reduce the so-called photon-to-motion latency.

[0181] As already pointed out, regarding Figures 5 to 13 All the features described can also be embodied by means of devices 10 and 12. For this purpose, device 10 can provide bitstreams 601, 901, and 1001 as previously described. For example, device 10 can be configured for step 101, wherein a picture representing projection 28 or a composite picture 62 is obtained, and a representation 40 of the picture or composite picture can be provided in the bitstream. Device 10 can also provide, as per the description of... Figures 6 to 8 The described geometric representation 50 is optional for bitstreams 901 and 1001. Device 10 may also optionally provide, as shown regarding, the bitstream. Figure 9 The described area projection indicates 60 and / or as per the description Figures 10 to 13 The described region type indication 70. Similarly, the device 12 can be configured for step 121, wherein representation 40 and one or more of geometric representation 50, region projection indication 60 and region type indication 70 are obtained from the bit stream.

[0182] Although some aspects have been described as features within the context of the device, it should be clear that such descriptions can also be considered as descriptions of corresponding features of the method.

[0183] Some or all of the method steps may be performed by (or using) hardware devices (such as microprocessors, programmable computers, or electronic circuits). In some embodiments, one or more of the most important method steps may be performed by such devices.

[0184] The encoded image signals of the present invention can be stored on a digital storage medium or transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium (e.g., the Internet). In other words, other embodiments provide video bitstream products including video bitstreams according to any of the embodiments described herein, for example, digital storage media having bitstreams stored thereon.

[0185] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or software, or at least partially in hardware or at least partially in software. Implementation can be performed using a digital storage medium (e.g., floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory) on which electronically readable control signals are stored, which cooperate with (or are capable of cooperating with) a programmable computer system to perform the corresponding methods. Therefore, the digital storage medium can be computer-readable.

[0186] Some embodiments of the invention include a data carrier having electronically readable control signals, capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0187] Typically, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer. This program code may, for example, be stored on a machine-readable medium.

[0188] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0189] In other words, embodiments of the method of the present invention are therefore computer programs having program code for performing one of the methods described herein when the computer program is run on a computer.

[0190] Therefore, other embodiments of the method of the present invention are data carriers, digital storage media, or computer-readable media on which a computer program is recorded, the computer program being used to perform one of the methods described herein. Data carriers, digital storage media, or recording media are generally tangible and / or non-transitory.

[0191] Therefore, another embodiment of the method of the present invention represents a data stream or signal sequence of a computer program used to perform one of the methods described herein. The data stream or signal sequence may, for example, be configured to be transmitted via a data communication connection (e.g., via the Internet).

[0192] Another embodiment includes a processing means, such as a computer or a programmable logic device, which is configured or adapted to perform one of the methods described herein.

[0193] Another embodiment includes a computer having a computer program installed thereon for performing one of the methods described herein.

[0194] Another embodiment of the invention includes an apparatus or system configured to transmit a computer program to a receiver (e.g., electronically or optically) for performing one of the methods described herein. The receiver may be, for example, a computer, mobile device, storage device, etc. The apparatus or system may, for example, include a file server for transmitting the computer program to the receiver.

[0195] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware device.

[0196] The apparatus described herein can be implemented using hardware devices, a computer, or a combination of hardware devices and a computer.

[0197] The methods described herein can be performed using hardware devices, computers, or a combination of hardware devices and computers.

[0198] As can be seen in the detailed embodiments described above, various features are combined together as examples for the purpose of simplifying this disclosure. This manner of disclosure should not be construed as reflecting an intention that the claimed examples require more features than are expressly recited in each claim. Rather, as reflected in the following claims, the subject matter may have fewer features than all the features of a single disclosed example. Therefore, the following claims are incorporated into the detailed embodiments, wherein each claim may stand alone as a separate example. While each claim may stand alone as a separate example, it should be noted that although dependent claims may relate to a specific combination with one or more other claims, other examples may also include combinations of dependent claims with the subject matter of each other dependent claim or combinations of each feature with other dependent or independent claims. Such combinations are presented herein unless it is stated that a particular combination is not intended to be used. Furthermore, even if a claim does not directly depend on an independent claim, it is intended to include features of any other independent claim.

[0199] The above embodiments are merely illustrative of the principles of this disclosure. It should be understood that modifications and variations of the arrangements and details described herein will be readily apparent to those skilled in the art. Therefore, it is intended to be limited only by the scope of the appended claims and not by the specific details given by way of the description and explanation of the embodiments herein.

Claims

1. An apparatus (10) for providing a bitstream (601), the apparatus comprising a processor configured to: Obtain an image representing the volume representation (22) of the object (20) as seen from a viewpoint (26) in 3D space, projected onto a projection plane (24) in the 3D space as a projection (28). The representation of the image and the geometric representation (50) of the projection plane (24) in the 3D space are provided in the bit stream (601). in, The geometric representation (50) includes the following indications: The projection plane (24) is inclined relative to the normal plane that is perpendicular to the line between the view position (26) and the point describing the position of the projection plane (24).

2. The apparatus (10) according to claim 1. in, The geometric representation (50) also indicates the position of the projection plane (24) in the 3D space.

3. The apparatus (10) according to claim 2. in, The geometric representation (50) also indicates the position of the projection plane (24) relative to the view position (26).

4. The apparatus (10) according to claim 2. in, The geometric representation (50) also indicates the position of the projection plane (24) relative to the viewing direction (82).

5. The apparatus (10) according to claim 2. in, The geometric representation (50) also indicates the position of the projection plane (24) relative to the absolute coordinates of the 3D space, and The view position (26) is represented by the absolute coordinates of the 3D space.

6. The apparatus (10) according to claim 1. in, The geometric representation (50) also indicates the distance between the view position (26) and the projection plane (24).

7. The apparatus (10) according to claim 1. in, The geometric representation (50) also indicates the size of the projection plane (24).

8. The apparatus (10) according to claim 1. in, The geometric representation (50) includes coefficients of the plane equation describing the position and orientation of the projection plane (24).

9. The apparatus (10) according to claim 1. in, The geometric representation (50) includes the coefficients of the plane equation describing the position and orientation of the projection plane (24) and the coordinates of the origin of the coordinate system referenced by the plane equation.

10. The apparatus (10) according to claim 1, wherein, The processor is configured to provide in the bitstream: The corresponding representations of multiple images, which represent multiple volume representations (22) of multiple corresponding objects (20) projected onto multiple corresponding projection planes (24) in the 3D space from the view position (26) in the 3D space. The projection plane (24) has multiple corresponding geometric representations (50), which respectively represent the projection. The plurality of images include the images, the plurality of objects (20) include the objects (20), the plurality of volume representations (22) include the volume representations (22), the plurality of corresponding projection planes (24) include the projection planes (24), and the plurality of corresponding geometric representations (50) include the geometric representations (50).

11. The apparatus (10) according to claim 10. in, The plurality of geometric representations (50) are associated with one or more of one or more time instances, and Wherein, the bit stream (601) has no two geometric representations (50) that are of equal size and / or position to the same time instance of the time instance and indicate the corresponding projection plane (24).

12. The apparatus (10) according to claim 11. in, The plurality of geometric representations (50) are associated with one or more of one or more time instances, and The bit stream (601) has no two geometric representations (50) associated with the same time instance and the same identifier.

13. The apparatus (10) according to claim 1. in, The image sample includes one or more corresponding values ​​of color information, texture information, transparency information, depth information, reflection information, and semi-transparency information.

14. The apparatus (10) according to claim 1, wherein, The bitstream (601) is a video bitstream.

15. The apparatus (10) according to claim 1, wherein, The processor is configured to provide in the bitstream: Supplementary information describing the bitstream (601) includes a projection syntax structure that signals the geometric representation (50).

16. The apparatus (10) according to claim 1, wherein, The processor is configured to provide in the bitstream: Information regarding the view position (26) and / or the viewing direction.

17. The apparatus (10) according to claim 1, configured as follows: The image is encoded to obtain an encoded image, and The encoded image is provided as a representation of the image.

18. The apparatus (10) according to claim 1, configured as follows: An image of the object (20) is obtained by projecting the volume representation (22) of the object (20) onto the projection plane (24) using the viewpoint (26) in the 3D space.

19. The apparatus (10) according to claim 1, configured as follows: Obtain multiple images, which represent multiple volumes of multiple corresponding objects (20) as projected onto multiple corresponding projection planes in the 3D space from the view position (26) in the 3D space. Based on the provided images and geometric representations (50), the plurality of images and the plurality of corresponding geometric representations of the projection plane (24) in the 3D space are provided in the bitstream (601). in, The plurality of images include the images, the plurality of objects (20) include the objects (20), the plurality of volume representations include the volume representation (22), the plurality of corresponding projection planes include the projection plane (24), and the plurality of corresponding geometric representations include the geometric representation (50).

20. The apparatus (10) according to claim 19, configured as follows: The plurality of images are obtained by projecting the corresponding volume representation (22) of the object (20) onto the corresponding projection plane (24) using the perspective of the view position (26) in the 3D space.

21. The apparatus (10) according to claim 19, configured as follows: Obtain the user location from the bitstream (16), and The view location is determined using the user's location (26).

22. The apparatus (10) according to claim 19 is configured as follows: Obtain the position of the object (20) in the 3D space from the bit stream (16), and Based on the position of the object (20), the volume is represented by a projection (28) onto the projection plane (24) from the view position (26).

23. The apparatus (10) according to claim 19 is configured as follows: Obtain the user's viewing direction from the bitstream (16), and The projection (28) of the object (20) onto the projection plane (24) is determined by taking into account the user's viewing direction.

24. The apparatus (10) according to claim 19 is configured as follows: Obtain user focus from the bitstream, and The image of the object (20) is determined using the user focus.

25. An apparatus (12) for processing a bitstream (601), the apparatus comprising a processor, the apparatus being configured to obtain from the bitstream (601): The image represents the volume representation (22) of the object (20) as seen from the viewpoint (26) in 3D space, projected onto the projection plane (24) in the 3D space as a projection (28). The geometric representation (50) of the projection plane (24) in the 3D space. in, The geometric representation (50) includes the following indications: The projection plane (24) is inclined relative to the normal plane that is perpendicular to the line between the view position (26) and the point describing the position of the projection plane (24).

26. The apparatus (12) according to claim 25 is configured as follows: The geometric representation (50) is used to render a rendering image of the 3D scene in the 3D space from the perspective of the current user's position.

27. The apparatus (12) according to claim 26 is configured as follows: The geometric representation (50) is used to determine the position of the object (20) within the 3D scene and / or to determine the contribution of the image to the rendered image.

28. The apparatus (12) according to claim 26 is configured as follows: The view position (26) is compared with the current user position to obtain the position difference. Based on the view position (26), the image represents a view of the object (20). The rendering of the image is based on the positional difference.

29. The apparatus (12) according to claim 25. in, The image represents a sample including one or more of the following: color information, texture information, transparency information, depth information, reflection information, and semi-transparency information.

30. The apparatus (12) according to claim 25 is configured as follows: Obtain supplementary information describing the bitstream (601) from the bitstream (601), wherein, The supplementary information includes the projection syntax structure of the geometric representation (50) that is signaled.

31. The apparatus (12) according to claim 25. in, The geometric representation (50) also indicates the position of the projection plane (24) in the 3D space.

32. The apparatus (12) according to claim 31. in, The geometric representation (50) also indicates the position of the projection plane (24) relative to the view position (26).

33. The apparatus (12) according to claim 31. in, The geometric representation (50) also indicates the position of the projection plane (24) relative to the viewing direction (82).

34. The apparatus (12) according to claim 31. in, The geometric representation (50) also indicates the position of the projection plane (24) relative to the absolute coordinates of the 3D space, and The view position (26) is represented by the absolute coordinates of the 3D space.

35. The apparatus (12) according to claim 25. in, The geometric representation (50) also indicates the distance between the view position (26) and the projection plane (24).

36. The apparatus (12) according to claim 25. in, The geometric representation (50) also indicates the size of the projection plane (24).

37. The apparatus (12) according to claim 25. in, The geometric representation (50) includes coefficients of the plane equation describing the position and orientation of the projection plane (24).

38. The apparatus (12) according to claim 25. in, The geometric representation (50) includes the coefficients of the plane equation describing the position and orientation of the projection plane (24) and the coordinates of the origin of the coordinate system referenced by the plane equation.

39. The apparatus (12) according to claim 25, wherein, The bit stream includes: The corresponding representations of multiple images, which represent the projections of multiple corresponding objects (20) onto multiple corresponding projection planes located in the 3D space from the view position (26) in the 3D space. The projection plane (24) has multiple corresponding geometric representations, each of which represents the projection. The plurality of images include the images, the plurality of objects (20) include the objects (20), the plurality of volume representations (22) include the volume representations (22), the plurality of corresponding projection planes include the projection planes (24), and the plurality of corresponding geometric representations include the geometric representations (50).

40. The apparatus (12) according to claim 39. in, The plurality of geometric representations (50) are associated with one or more of one or more time instances, and Wherein, the bit stream (601) has no two geometric representations (50) that are of equal size and / or position to the same time instance of the time instance and indicate the corresponding projection plane (24).

41. The apparatus (12) according to claim 40. in, The plurality of geometric representations (50) are associated with one or more of one or more time instances, and The bit stream (601) has no two geometric representations (50) associated with the same time instance and the same identifier.

42. The apparatus (12) according to claim 25, wherein, The bitstream (601) is a video bitstream.

43. The apparatus (12) according to claim 25, wherein, The device is configured to obtain the following information from the bitstream: Information regarding the view position (26) and / or the viewing direction.

44. A method for providing a bit stream (601), comprising: Obtain (101) an image representing the volume representation (22) of an object (20) as a projection (28) onto a projection plane (24) in the 3D space, from the viewpoint (26) in the 3D space. Provide (102) a representation (40) of the image and (24) a geometric representation (50) of the projection plane (24) in the 3D space in the bitstream. The geometric representation (50) includes the following indication: the inclination of the projection plane (24) relative to the normal plane perpendicular to the line between the view position (26) and the point describing the position of the projection plane (24).

45. A method for processing a bit stream (601), comprising obtaining (121) the following from the bit stream: The image representation (40) shows the volume representation (22) of the object (20) from the viewpoint (26) in 3D space to the projection (28) onto the projection plane (24) in the 3D space; and The geometric representation (50) of the projection plane (24) in the 3D space. in, The geometric representation (50) includes the following indications: The projection plane (24) is inclined relative to the normal plane that is perpendicular to the line between the view position (26) and the point describing the position of the projection plane (24).

Images