Method and device for achieving layered representation of volumetric content

Abstract

The invention relates to a method and device for rendering, in an image plane, a volumetric image decomposed into a plurality of 3D Gaussians. The method comprises the following steps for at least one sample (P) to be reconstructed in the image plane (IP): - identifying (E21, E22) a plurality of 3D Gaussians along a ray (R, R1, R2) traced from the sample, a 3D Gaussian being delimited by two bounds (Bk+1) (Bk); - identifying (E23) a plurality of layers (Bk-Bk+1, Lk), each layer being delimited by two of the adjacent bounds and comprising at least one identified 3D Gaussian, referred to as the 3D contributing Gaussian. The method further comprises: - projecting (E24) the 3D contributing Gaussians as respective 2D contributing Gaussians into the image plane; - accumulating (E25) between the 2D contributing Gaussians a layer comprising at least two 2D contributing Gaussians, - blending (E27) at least two of the layers of the plurality.

Description

Method and device for representing volumetric content by layer.

[0001] The invention relates to the general field of volumetric images. It relates more particularly to the representation and rendering of such content.

[0002] A volumetric image corresponds to three-dimensional content or a three-dimensional scene. Such an image can be part of a sequence of images, or a video. Several approaches exist for representing volumetric content, notably those based on an explicit or implicit representation of the 3D scene in question.

[0003] An explicit approach represents a 3D scene using geometric primitives (volumes, surface elements, voxels, etc.) and then uses image synthesis techniques to render an image of this scene associated with a viewpoint. An implicit approach uses an implicit representation of the scene. Examples include approaches that define a 3D map of distances to objects in the scene (e.g., Signed Distance Function), or a 3D map of the density and colorimetry properties of different 3D positions in the scene.

[0004] Recently, we have seen significant improvements in 3D scene construction and representation techniques, particularly through implicit representation based on neural networks and ray tracing. However, these approaches remain too complex in practice.

[0005] A technique for representing a 3D scene as a set of 3D Gaussian primitives, known as 3DGS, was proposed in the article "3D Gaussian Splatting for Real-Time Radiance Field Rendering" (Kerbl et al., ACM Transactions on Graphics, July 2023). Compared to the approaches mentioned previously, this method is significantly less complex, while offering better reconstruction quality for volumetric content. It enables real-time rendering on current devices. This is particularly related to the rendering technique associated with a set of 3D Gaussian primitives as defined in previous work on Gaussian splatting, notably in the work of Zwicker (Surface Splatting, Zwicker et al., in Proceedings of the 28th annual conference on Computer graphics and interactive techniques: August 12-17, 2001).

[0006] Although highly effective, the rendering technique introduced by 3DGS is not without its flaws. These include undesirable induced transparency, linked to the density factor of a Gaussian distribution, rendering distortion depending on the viewpoint, and inherently low reconstruction quality.

[0007] Therefore, there is a need for a solution that effectively corrects current volumetric rendering techniques.

[0008] The invention relates to a method for rendering, in an image plane, a volumetric image decomposed into a plurality of 3D Gaussians, comprising the following steps for at least one sample to be reconstructed in the image plane: - identification of a plurality of 3D Gaussians along a ray launched from said sample, a said 3D Gaussian being delimited by two bounds; - identification of a plurality of layers, a said layer being delimited by two of said adjacent bounds and comprising at least one contributing 3D Gaussian; - projection of said contributing 3D Gaussians into respective contributing 2D Gaussians in the image plane; - accumulation of at least two parameters of at least two of said contributing 2D Gaussians only if the at least two respective 3D Gaussians are in the same layer.

[0009] According to one embodiment, the process further comprises a blending step between at least two of said layers of the plurality.

[0010] A "volumetric image" is defined as a 3D scene that can be observed from different angles and viewpoints. The volumetric representation is the set of data associated with such an image: for example, a set of 3D samples (voxels), or a set of 3D Gaussians and associated density information.

[0011] The term "rendering" refers to the operation of representing all volume data on a flat surface, called the image plane, corresponding to a viewpoint of the scene.

[0012] The term "sample to be reconstructed" refers to a value in the image plane that we seek to reconstruct. The sample could be, for example, a color pixel traditionally represented by a triplet of values, such as (R, G, B) or (Y, U, V). Generally, a sample is represented by a vector of one or more components, or data points.

[0013] A "3D Gaussian" is defined as a reconstruction kernel in space. Volume data is thus interpreted as a field of 3D reconstruction kernels, with each kernel typically associated with a point, or voxel, in space. Each 3D reconstruction kernel can be considered a "fingerprint" function, distributing the contribution of each voxel across nearby voxels. A 3D Gaussian is usually defined by its position in space, its density, its covariance, and its color coefficients (or harmonics). Such data can be obtained, or calculated, from latent codes, particularly during the training, construction, or learning of 3D Gaussians. Any standard, state-of-the-art method can be used. A 3D Gaussian is bounded by two limits, or real values, on the radius axis.

[0014] The term "projection of 3D Gaussians into 2D Gaussians" refers to the projection, or splatting, operation that produces 2D, or flattened, Gaussians in the image plane. A 2D Gaussian is represented by parameters: its position, its covariance matrix, its color coefficients (e.g., (R, G, B)), and its density coefficient. Opacity is defined as a function of density. Alternatively, the concept of opacity can be replaced by that of transparency, as opacity and transparency are complementary. An object is totally opaque if the opacity value is at its maximum, i.e., at the value 1. Conversely, it is invisible if this value is at 0. To each pixel of the image plane, we can associate, during rendering, color coefficients (which are those of the 2D Gaussian), and a density coefficient defined as the product of the density coefficient of the 2D Gaussian and the value of the 2D Gaussian at that point.Thus, the density coefficient of a pixel varies and tends to become zero as one moves away from the center of the 2D Gaussian.

[0015] By "ray," we mean, according to current best practices, a virtual half-line originating at the camera (or the observer's eye) and passing through the image plane at the position of the sample (pixel) that we wish to reconstruct. Volumetric rendering thus simulates a physical optical process by projecting or casting rays along which the volumetric density of the scene can be calculated. This is also referred to as ray integration.

[0016] A "layer" is defined as an interval bounded by two endpoints on the radius. A layer includes at least one contributing Gaussian portion. A layer can contain one or more contributing Gaussian portions. A Gaussian portion can correspond to the complete Gaussian if its endpoints coincide with the layer's boundaries; otherwise, it is considered to be the portion congruent to the interval. In other words, a Gaussian is considered to contribute to one or more layers between its entry and exit points.

[0017] By "accumulation" we mean the summation of data associated with a sample of the image plane, originating from the different portions of Gaussian matrices contributing to a layer. The 2D Gaussians resulting from the projection of the 3D Gaussians can overlap after projection (splatting) onto the image plane. For example, averaging the parameters of several Gaussians allows us to calculate a resulting average color value, after possible weighting. Summing the densities yields a resulting density value. From this resulting density, a resulting opacity can be calculated for the layer.

[0018] "Blending" refers to the combination of the contributions of different layers associated with a sample of the image plane, according to their opacity factor. Blending the parameters of several layers allows for the consideration of transparency effects.

[0019] The 3D Gaussians composing the scene are thus associated with constituent layers of the scene. If portions of Gaussians overlap, the layer formed by their combination can correspond to a local surface element of the scene. The process performs an accumulation step to reconstruct this layer. Once the accumulation is complete for this layer, it can be blended with the other layers. The process thus reconstructs, layer by layer, the contribution of the layers that can correspond to local surfaces, and then performs a blending operation to take into account the composition of the contributions of these different layers according to their transparency factor.

[0020] Advantageously, instead of using a recursive reconstruction formula based solely on the blending principle, as proposed in the state of the art, the process interweaves accumulation and blending approaches. This allows, in particular, for correcting the viewpoint-dependent rendering distortion encountered in state-of-the-art methods such as 3DGS. As will be explained later with reference to Figures 2 and 3, 3DGS considers an order for the 3D Gaussians to be composed. However, this order can depend on the viewpoint, especially if there is an overlap of the 3D Gaussians. This leads to rendering distortion when switching from one viewpoint to another. Accumulation processing within a layer greatly mitigates this phenomenon.

[0021] Furthermore, in the case of a 3D overlap, 3DGS combines the contributions of each Gaussian using the blending formula, whereas in this scenario, accumulation is more appropriate. The accumulation process according to the invention thus enables the expected reconstruction in this context.

[0022] Generally, the steps in a rendering process should not be interpreted as being linked to a notion of temporal succession. In other words, the steps can be carried out in a different order than that indicated in the independent claim, or even in parallel.

[0023] According to different implementation methods of the rendering process:

[0024] - the parameters of said 2D Gaussians include a color and a density, and:- the accumulation step includes calculating an average of the color of said at least two contributing 2D Gaussians to obtain a resulting color, and a combination of the density of said at least two contributing 2D Gaussians to obtain a resulting density and a resulting opacity.- the blending step uses said average color and said resulting opacity from the accumulation step.

[0025] Advantageously, such a process allows us to obtain, for a sample (pixel), a resulting color and opacity by accumulating several portions of Gaussians within a layer. The resulting color is an average color of the contributing 2D Gaussians. The resulting opacity of a layer is a function of the resulting density calculated from the densities of the contributing 2D Gaussians. Subsequently, these resulting values ​​are used in the blending step as if they had been obtained from a single Gaussian.

[0026] - the density of a said contributing 2D Gaussian is calculated according to a distribution law of the respective 3D Gaussian along the radius; advantageously according to this method, the rendering is closer to the underlying model which would be the integration of the 3D Gaussians along the radius.

[0027] - The density of a contributing 2D Gaussian is calculated as a function of the interval defined by the boundaries of said layer; advantageously, according to this method, the densities of a Gaussian are distributed among the different layers to which they contribute, taking into account the thickness of these layers. This allows us to consider only the portion of a contributing Gaussian within the layer. For example, the initial density of the 2D Gaussian can be corrected by a ratio calculated between the thickness of the portion of the 3D Gaussian within the layer and the total thickness of the 3D Gaussian.

[0028] – the resulting color is further weighted by said resulting density; advantageously this allows the average value of the color to be calculated on this layer, thus proposing a continuous variation of the color spatially.

[0029] The resulting opacity depends on the product of the complements of the densities of at least two contributing 2D Gaussians; advantageously, in this mode, the resulting opacity corresponds to the equivalent opacity after the ray passes through the considered Gaussians. Thus, objects located behind this layer are seen with the same transparency factor in this mode as in the alpha-blending composition mode.

[0030] - The resulting opacity depends on the resulting density; advantageously, according to this method, the resulting opacity thus defined can correspond to the resulting density associated with the layer, calculated from the densities of the various Gaussians constituting it. This is notably the case when the opacity is simplified to the density. It can also correspond to any function of this resulting density.

[0031] - the resulting opacity is a function of the exponential of the resulting density; advantageously according to this variant, the opacity is expressed, when not simplified, as a function of the exponential of the density.

[0032] - The position of the bounds of a 3D Gaussian is defined by a parallelepiped encompassing said 3D Gaussian. Advantageously, according to this variant, the bounds are defined very simply by the intersection of the radius with the encompassing parallelepiped. A 3D Gaussian is not strictly speaking a 3D shape, but such a 3D shape can be associated with a level curve of the 3D Gaussian. By "parallelepiped encompassing said 3D Gaussian" we therefore mean, for the sake of simplicity, "parallelepiped encompassing a level curve of said 3D Gaussian".

[0033] Correspondingly, the invention also aims at a rendering device.

[0034] The characteristics and advantages of the rendering process apply in the same way to the rendering device according to the invention and vice versa.

[0035] The invention also relates to a computer program on a recording medium, this program being capable of being implemented in a computer or rendering device according to the invention. This program includes instructions adapted to the implementation of the corresponding method. This program may use any programming language and be in the form of source code, object code, or code intermediate between source and object code, such as in a partially compiled form, or in any other desirable form.

[0036] The invention also relates to a computer-readable information or recording medium containing the aforementioned computer program instructions. The information or recording medium can be any entity or device capable of storing programs. For example, the medium may include a storage means, such as a ROM (e.g., a CD-ROM or a microelectronic circuit ROM), or a magnetic recording means, such as a floppy disk or hard drive, a DNA sequence, or flash memory. Furthermore, the information or recording medium may be a transmissible medium, such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, radio link, wireless optical link, or other means.

[0037] The program according to the invention can in particular be downloaded onto an Internet-type network.

[0038] Alternatively, each information or recording medium can be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of a process according to the invention.

[0039] Other features and advantages of the present invention will become apparent from the description given below, with reference to the attached drawings which illustrate examples of embodiment without any limiting character.

[0040] Lare represents schematically a volumetric rendering device according to an embodiment of the invention.

[0041] The schematic representation of the volumetric rendering of a scene according to the state of the art.

[0042] The diagram schematically represents the volumetric rendering of a scene from two different points of view according to the state of the art.

[0043] Larepresents a graphic result of the volumetric rendering of a scene from two different points of view according to the state of the art in accordance with the.

[0044] Lare represents schematically the definition of the limits of the Gaussians, according to one embodiment of the invention.

[0045] laest un logicgram representing one embodiment of the volumetric rendering process according to the invention.

[0046] lare represents schematically the definition of localization intervals of 3D Gaussians along a radius according to two modes of realization.

[0047] The diagram schematically represents the hardware architecture of a volumetric rendering device (Render) according to an embodiment of the invention. The rendering device is connected via an optional human-machine interface (I / O) to a database module (BDS) containing the representation of the 3D scene (3DS) to be rendered. This module can take the form of a server, a database, etc. It includes the 3D scene representations.

[0048] The rendering device typically comprises memory (MEM) associated with a processor (PROC) in a processing unit, controlled by a computer program (PGR) stored in read-only memory (ROM or hard drive). At initialization, the computer program's code instructions are loaded into RAM before being executed by the processor. Memory can be of the ROM (Read Only Memory), RAM (Random Access Memory), or Flash type.

[0049] In accordance with the law, the system also includes:

[0050] – an optional GEN module for creating 3D Gaussians representing the volumetric image. The 3D scene can indeed be provided in another format, for example, as a set of voxels or as a set of latent codes. In this case, the GEN module creates the 3D Gaussians from this data. Any known state-of-the-art method can be used, for example, the one described in the aforementioned 3DGS document.

[0051] – an ANAL module responsible for performing ray tracing and scene analysis (integration) along the ray. In particular, this module is responsible for projecting 3D Gaussians into 2D Gaussians on the image plane, defining a list of 3D Gaussians along a ray, and the boundaries of each Gaussian, as well as the different layers, depending on the chosen boundary definition method.

[0052] – an ACC module responsible for accumulation, within a layer defined by two consecutive boundaries;

[0053] – a BLEND module responsible for blending between two layers;

[0054] – an optional DISP module responsible for displaying the resulting image I, for example on a computer screen.

[0055] All modules communicate conventionally with each other via a data bus (B).

[0056] The schematic representation of the volumetric rendering of a scene according to the state of the art.

[0057] The user's viewpoint (whose eye is located at the position of the CAM camera) is represented by the two dotted half-lines which delimit the 3D scene according to this viewpoint as well as the projection onto the IP image plane.

[0058] The 3D (3DS) scene shown is composed of 3D Gaussians. To perform the rendering, they are projected onto the camera's image plane. These projections are referred to as 'splat' (or 2D splat) hereafter. We are particularly interested in the three Gaussians GA, GB, and GC in the figure, which lie on the projected (or launched) radius R from the CAM camera. They are projected as three splats: GA', GB', and GC'. The ellipsoids corresponding to the projected Gaussians are artificially positioned above or below the image plane to distinguish them. However, within the image plane, these projected ellipsoids are at the same level.

[0059] Obtaining 3D Gaussians can be done, as proposed in 3DGS, by classic optimization techniques which allow, from an initial point cloud, to control the number and parameters of the Gaussians for optimal rendering of the scene.

[0060] The rendering principle proposed by 3DGS is to work within the image plane, utilizing a 3D Gaussian scanning rendering technique. A painter's algorithm can be used to scan the Gaussians. This algorithm draws the different Gaussians in order of depth, from the furthest to the nearest. The final color of the pixel is thus rendered step by step using an α-blending algorithm, in which each layer contributes a coefficient α, representing the opacity of the Gaussian. Gaussians can also be rendered from front to back. For example, the rendering of pixel P, shown in the figure, can be done using the order GA, GB, GC, or GA, GC, GB. Each time a layer is added, it contributes to the pixel's rendering with a factor α and attenuates the background by a factor of (1-α).The final color in P, when the Gaussians are treated in the order GA, GB and GC, can be expressed by C = α. c * c c + α b * (1- α c ) * c b + α a * (1- α c ) (1- α b ) * c a. It should be noted that in this order, for an identical opacity factor, Gaussian A is more attenuated than B, which is itself more attenuated than C, thus giving the strongest weighting to the foreground.

[0061] More generally, the reconstruction of the image plane into a single pixel x can be expressed by the following formula:

[0062]

[0063] Where Ci is the color value associated with Gaussian i (this value can be a scalar, a multicomponent representation as for color images, for example RGB or YUV, or a multidirectional representation of color, for example via spherical harmonic coefficients, as in 3DGS); according to one embodiment, the opacity of the i-th Gaussian for the sample x is defined as , And represents the attenuation (transmittance) to be applied to the i-th Gaussian due to the transparencies of the objects present "above" this Gaussian (in the chosen order).

[0064] In the aforementioned 3DGS algorithm, the transparency at a pixel x depends on two parameters: the density factor oi of the Gaussian, and the attenuation G i (x) relative to the position within the Gaussian. This attenuation can be defined by the following formula, in which µ i 2Dis the center of the Gaussian curve projected onto the image plane.

[0065]

[0066] According to the aforementioned state of the art, the normalization factor used in front of the exponential can take the value 1, or alternatively a value calculated based on the 2D covariance matrix Σ 2D . The covariance matrix Σ 2D is obtained by projecting the 3D covariance matrix associated with the 3D Gaussian, as detailed in the state of the art, where it is proposed to use a low-pass filter, modeled by a simple Gaussian, to avoid aliasing effects due to unsatisfactory sampling of samples in the image plane. The effect of this low-pass filter is simply expressed by adding a term to the diagonal of the matrix. In the expression for the attenuation factor, the covariance matrix is ​​therefore modified to Σ 2D'To account for the dilation effect of 2D Gaussians, the ability to perform anti-aliasing without multiple sub-pixel renderings is unique to the use of Gaussians (according to the principle outlined in Zwickler's original paper). Other approaches, particularly ray-tracing, require multiple ray tracings per pixel to achieve anti-aliasing.

[0067] As shown in figures 3 and 4, this approach suffers from a difference in rendering of the scene depending on the viewpoints.

[0068] The schematic representation of the volumetric rendering of a scene from two different viewpoints according to the state of the art (3DGS).

[0069] Lare represents a flat surface (hatched surface) with a texture gradient, which we want to render from two viewpoints, denoted LV (Left View) and RV (Right View). The algorithm presented in support of la orders the projected 3D Gaussians in depth. Thus, in the left view, the leftmost Gaussian will be the furthest, while the opposite is true in the right view. In other words, in the left view, the 2D Gaussians are rendered 'from left to right', while in the right view, they are rendered 'from right to left'. This variation leads to differences in rendering that can fluctuate considerably, since, as mentioned previously, in 3DGS, the rendered value depends on the depth order of the Gaussians (the closest one taking precedence over the furthest).This fluctuation can be particularly important when the Gaussians are very close in depth, since in this case the order can vary very rapidly from one viewpoint to another.

[0070] Larepresents a graphic result of the volumetric rendering of a scene from two different points of view according to the state of the art in accordance with the.

[0071] On the graph, 3 curves are represented: - the solid line curve (ground truth) illustrates the result obtained by composing the Gaussians according to an "ideal" rendering; - the thin dotted line curve (left to right) shows the result perceived for a left view; - the wider dotted line curve (right to left) shows the result perceived for a right view.

[0072] It is clear that the two curves are offset, which means that the rendering will be different depending on the left or right viewpoint according to the state of the art.

[0073] The diagram schematically represents the volumetric rendering of a scene according to an embodiment of the invention.

[0074] The context of the 3D scene is similar to that of the: the user's viewpoint and the 3D scene are represented similarly, the 3D Gaussians can be obtained by classical optimization techniques, and the three Gaussians GA, GB and GC which lie on the ray R launched from the CAM camera are projected into 3 splats GA', GB' and GC'.

[0075] The rendering principle differs from that of the previous method: each Gaussian is associated with an entry and an exit boundary for the considered radius. These boundaries are defined by their depth, with the entry boundary being shallower than the exit boundary. A Gaussian is considered a contributor to a layer between its entry and exit boundaries. The set of boundaries for all Gaussians defines a layered structure, each layer being the area delimited by two consecutive boundaries. Within each layer, the contribution of each Gaussian is defined according to its own characteristics and the characteristics of the layer, such as its thickness, i.e., the size of the interval between its two boundaries. Within each layer, the contributions of the Gaussians are accumulated, and a blending is performed between the layers.

[0076] According to one embodiment, within a layer, as will be detailed below, the resulting color is calculated as the average of the colors of the contributing Gaussians, weighted by their density contribution.

[0077] According to one embodiment, within a layer, as will be detailed below, the density contribution of the Gaussians is accumulated to obtain a resulting density.

[0078] In one embodiment, the opacity of the layer is calculated as a function of this resulting density. For example, this function is expressed as the 1's complement of the exponential of the inverse of the resulting density. For example, the resulting density is simply bounded in an interval [0,1].

[0079] In the figure, only the endpoints of Gaussians intersecting with the radius are shown. A layer is represented by its two endpoints. The Gaussian GB is the sole contributor to layers B1-B2 and B3-B4, while layer B2-B3 has two contributing Gaussians, GB and GC. Layer B4-B5 has no contributing Gaussian, and layer B5-B6 has GA as its only contributor. Therefore, four layers must be considered, labeled L1 to L4 in the figure.

[0080] Once the layers have been identified, the blending steps are carried out in the usual way between these layers.

[0081] Thus, the final color of the pixel is rendered step by step by an α-blending algorithm performed between layers (and not between Gaussian colors), and an accumulation within the layers. Each layer contributes according to its opacity coefficient α. The layers can be rendered in any order (from front to back, or from back to front, with accumulation and blending being sequential or not). For example, the rendering of pixel P on the image can be done by considering the order B1 to B6 or B6 to B1. Each time a layer is added, it contributes to the pixel's rendering via a factor α and attenuates the background by a factor of (1-α). The final color of P, when the Gaussian colors are processed in the order B1 to B6, can be expressed as C = α4 * c4 + α3 * (1-α4) * c 3 +… where c k represents the resulting color in the k and α layers kreprésente l'opacité résultante de la couche k.

[0082] According to one embodiment, the density contribution of the Gaussian i is distributed according to a distribution law which achieves an approximation of the Gaussian along the radius.

[0083] According to another embodiment, the density contribution of the Gaussian i is distributed proportionally to the thickness of the layers to which it contributes. Its contribution to a layer k delimited by the boundaries B k-Bk+1est : oùni, fisont les indices des bornes d’entrée et de sortie de la Gaussienne i, au sein de la liste de toutes les bornes, ordonnées par profondeur, etdiest la densité de la Gaussienne 2D.

[0084] For example, on the set, the Gaussian GB contributes to the interval B2-B3 by a factor of (B2-B3) / (B1-B4). We then denote the sum of the density contributions, or resultant density, over the set Gaussian columns contributing to the k-th layer:

[0085] According to this embodiment, the average color on layer B k -B k+1 East

[0086] The opacity coefficient α kassociated with layer k, and which will be used for blending, can be defined in different ways from the density contributions.

[0087] According to one embodiment, it is equal to the resulting density, bounded to the interval [0,1].

[0088] In another way, it is a function of the product of the complements of these densities.

[0089] According to a third method, it is defined as a function of the resulting density, according to:

[0090] This is a flowchart representing an example of a volumetric rendering process that can be implemented by the rendering device according to the example of the.

[0091] During the initialization step E20, the 3D scene is constructed as a set of 3D Gaussians. Any algorithm within the grasp of a person skilled in the art can be used to generate the Gaussians, for example, the algorithm proposed in the aforementioned 3DGS method, which uses gradient descent to minimize a quality criterion for reconstructing the rendered views. The algorithm then proceeds for each pixel of the image plane according to steps E21 to E26.

[0092] During step E21, a list of Gaussians intersecting the ray associated with the pixel of the image plane is established. Following the example above, 3 Gaussians GA, GB and GC are identified on the ray, and listed for example in this order (from the furthest to the nearest).

[0093] In step E22, input / output bounds are defined for each Gaussian, and then these bounds are sorted (e.g., B1, B2, B3, B4, B5, B6). The definition of the bounds associated with a Gaussian can be done in various ways, as described in the supporting documentation. The resulting values ​​of the bounds are, for example, real numbers indicating a depth position along the radius axis.

[0094] During an E23 step, a plurality of layers is identified from these boundaries. A layer Lk is identified by an interval Bk-Bk+1 between two successive, or adjacent, boundaries, that is, the two closest boundaries in the set of boundaries.

[0095] The 3D Gaussians are projected onto the image plane as many 2D Gaussians as possible during step E24. This projection step has been described previously and is known from prior art, notably from the aforementioned Zwicker article. After projection, a 2D Gaussian, or splat, is described by its position parameters, covariance matrix, color coefficients (e.g., (R, G, B)), and density coefficient. It should be noted that this step can be performed at any point in the algorithm, provided that the contribution of the 3D Gaussian via its projection is available before performing the calculation steps related to a layer to which it contributes (steps E25 and E26).

[0096] Next, the pixel value in each Lk layer is evaluated during steps E25 and E26, layer by layer.

[0097] In step E25, the density contribution of each Gaussian for this layer is evaluated, and the density contributions are accumulated to calculate the resulting density A k As explained in support of the, the resulting color C k in the layer is calculated as an average value of the colors of the contributing Gaussians weighted by their contribution to the resulting density. The resulting opacity coefficient α k The density of the layer is then evaluated as described above, either from the density contributions of each Gaussian, or from the resulting density A. k .

[0098] Then, during step E26, the layer thus obtained is composed via an alpha-blending operation using the resulting opacity of the layer and that of the previous layer.

[0099] At the end of the layer iteration (when step E27, testing the last layer, is successful), the rendering operations are finalized in step E28. Typically, a background color, denoted C, is applied. on the, can be accumulated by alpha blending behind the last bound.

[0100] lare represents schematically the definition of the 3D Gaussian bounds along a radius according to two modes of realization.

[0101] This localization step details in particular step E22 of the logic diagram. Each of the Gaussians which has previously been placed in the sorted list (E1, E2, E3) for a given radius (R1, R2) is associated with a localization interval whose bounds are the bounds associated with the Gaussian.

[0102] Present two modes of embodiment:

[0103] According to a first mode, represented on the right, the localization intervals between the terminals are defined by large, unoriented boxes, by the dotted rectangles.

[0104] Such a box, called a global bounding box, can be used to define a localization interval between two bounds, for example from the position parameters µ and the covariance matrix of the 3D Gaussian, Σ 3D , as the interval on the z-axis of the radius having center µz and width (k being a factor defining the confidence interval, typically a value of 2 to 3).

[0105] The position parameters µ and the covariance matrix of the 3D Gaussian can be expressed by:

[0106]

[0107]

[0108] According to a second method, shown on the left, the localization intervals are defined by oriented bounding boxes, referred to as "tight" bounding boxes, represented by the dashed quadrilaterals. For the sample associated with radius R1, there is no overlap of the intervals; the Gaussians each contribute to only one layer. For the sample associated with radius R2, the Gaussians E1 and E2 overlap, and therefore both contribute to a common layer.

[0109] Such a tight bounding box has the advantage of fitting the orientation of the 3D Gaussian and thus having a narrower interval. It can be calculated, for example, from the position parameters µ and the inverse matrix of the covariance matrix of the 3D Gaussian, Λ 3D , as the interval on the z-axis of the radius having its center + , where (x,y) are the pixel coordinates in the image plane. and for width (k being defined as before).

Claims

A method for rendering, in an image plane, a volumetric image decomposed into a plurality of 3D Gaussians, comprising the following steps for at least one sample (P) to be reconstructed in the image plane (IP): - identification (E21, E22) of a plurality of 3D Gaussians along a radius (R, R1, R2) launched from said sample, a said 3D Gaussian being delimited by two bounds (B k+1 ) (B k ) ;- identification (E23) of a plurality of layers (Bk-Bk+1, Lk), a said layer being delimited by two of said adjacent bounds and comprising at least one contributing 3D Gaussian.- projection (E24) of said contributing 3D Gaussians into respective contributing 2D Gaussians in the image plane ;- accumulation (E25) of at least two parameters of at least two of said contributing 2D Gaussians only if the at least two respective 3D Gaussians are in the same layer. Rendering method according to claim 1, further comprising a blending step (E27) between at least two of said layers of the plurality. A rendering method according to claim 1 or 2, wherein the parameters of said 2D Gaussians include a color (Ci) and a density (Oi), and: - the accumulation step comprises calculating an average of the color of said at least two contributing 2D Gaussians to obtain a resulting color, and a combination of the density of said at least two contributing 2D Gaussians to obtain a resulting density and a resulting opacity; - the blending step uses said resulting color and said resulting opacity from the accumulation step. Rendering method according to claim 3, wherein the density of a said contributing 2D Gaussian is calculated as a function of a distribution law of the respective 3D Gaussian along the radius. Rendering method according to claim 3, wherein the density of a said contributing 2D Gaussian is calculated as a function of the interval defined by the bounds of said layer. Rendering method according to claim 3, wherein the resulting color is further weighted by said resulting density. Rendering method according to claim 3, wherein the resulting opacity depends on a product of the complements of the densities of said at least two contributing 2D Gaussians. Rendering method according to claim 3, wherein the resulting opacity depends on said resulting density. Rendering method according to claim 8, wherein the resulting opacity is a function of the exponential of the resulting density. A method for rendering a volumetric image according to one of the preceding claims, characterized in that the position of the bounds of a 3D Gaussian is defined by a parallelepiped encompassing said 3D Gaussian. A rendering device (RENDER), in an image plane (IP), of a volumetric image (3DS) decomposed into a plurality of 3D Gaussians, characterized in that the rendering device is configured to implement, for at least one sample (P) to be reconstructed in the image plane (IP): - identification (ANAL) of a plurality of 3D Gaussians along a radius (R, R1, R2) launched from said sample, a said 3D Gaussian being delimited by two bounds (B k+1 ) (B k) ;- identification (ANAL) of a plurality of layers (Bk-Bk+1, Lk), a said layer being delimited by two of said adjacent bounds and comprising at least one contributing 3D Gaussian.- projection (ANAL) of said contributing 3D Gaussians into respective contributing 2D Gaussians in the image plane ;- accumulation (ACC) of at least two parameters of at least two of said contributing 2D Gaussians only if the at least two respective 3D Gaussians are in the same layer. Computer program comprising instructions for executing the steps of a rendering process according to claim 1 when said program is executed by a computer.

Images