Segmented rendering method based on three-dimensional Gaussian sputtering

Abstract

The invention provides a segmented rendering method based on three-dimensional Gaussian sputtering, and relates to the technical field of scene rendering, and the method comprises the steps: carrying out the partitioning processing of the point cloud data of a scene through employing an axis alignment bounding box, and obtaining a plurality of non-overlapped partitioned scenes; detecting intersection points of the light and the boundary of the axis alignment bounding box, and segmenting the light based on the intersection points to obtain a plurality of light segments; distributing a part of scenes through which the segmented light segments pass to corresponding 3DGS modules for rendering to obtain a block scene rendering result; combining the scene rendering results of the blocks to obtain an overall scene rendering result; the scene is divided into a plurality of non-overlapped block scenes, so that redundant calculation is avoided, and the calculation efficiency is improved; light rays are segmented and distributed to corresponding 3DGS modules, so that efficient management of the light rays among different block scenes is realized; and the block scene rendering results of the block scenes are merged through the segmentation rendering equation, so that the overall visual effect of the scene is ensured.

Description

Technical Field

[0001] The invention relates to the technical field of scene rendering, and in particular to a segmented rendering method based on three-dimensional Gaussian sputtering. Background Art

[0002] In scene rendering, the divide and conquer strategy is a common method for dealing with large-scale scenes. This strategy needs to solve the problem of visual discontinuity between block scenes, that is, obvious seams and artifacts may appear at the junction of block scenes. The existing solution is to use regional overlap between block scenes to achieve a certain degree of visual smooth transition between blocks, but the overlap between block scenes will lead to a large amount of redundant calculations and the effect is not ideal. Summary of the invention

[0003] In response to the above problems, the present invention provides a segmented rendering method based on three-dimensional Gaussian sputtering, which solves the technical problem in the prior art that overlap between block scenes will lead to a large amount of redundant calculations and unsatisfactory results. It can divide the scene into multiple non-overlapping block scenes, avoid redundant calculations, and improve computing efficiency.

[0004] The embodiment of the present invention provides a segmented rendering method based on three-dimensional Gaussian sputtering, comprising:

[0005] Use axis-aligned bounding boxes to block the point cloud data of the scene to obtain several non-overlapping block scenes, where one block scene corresponds to one 3DGS module;

[0006] detecting intersections of the ray with the boundary of the axis-aligned bounding box, and segmenting the ray based on the intersections to obtain a plurality of ray segments;

[0007] Distribute the part of the scene that the segmented light segment passes through to the corresponding 3DGS module for rendering to obtain the block scene rendering result;

[0008] Merge the rendering results of each block scene to get the overall rendering result of the scene.

[0009] In some embodiments, including:

[0010] Load the point cloud data of the scene and preprocess the point cloud data;

[0011] Assigning initial Gaussian distribution parameters to the preprocessed point cloud data, wherein the Gaussian distribution parameters include position, scale matrix, color value and density value;

[0012] The point cloud data assigned with Gaussian distribution parameters is divided into blocks using an axis-aligned bounding box to obtain several non-overlapping block scenes.

[0013] In some embodiments, including:

[0014] The preprocessing includes denoising, simplification or resampling.

[0015] In some embodiments, including:

[0016] If the intersection points of the ray r and the axis-aligned bounding box boundary are defined as t0 and t1 respectively, the formula is expressed as follows:

[0017] r(t)=o+t·d,t0≤t≤t1;

[0018] Where t is a scalar parameter, r(t) is the point on the ray at the scalar parameter t, o is the starting point of the ray, and d is the direction vector.

[0019] In some embodiments, including:

[0020] The 3DGS module calculates the color contribution of the light segment through the piecewise integration formula:

[0021] C(r)=∑ k T(t k )·σ(t k )·C(t k )·Δt;

[0022] Where C(r) is the color of a pixel in the block scene, T(t k ) is the transmittance, σ(t k ) is the density, C(t k ) is the color value, and Δt is the distance.

[0023] In some embodiments, including:

[0024] For a ray that spans multiple tiled scenes, the final color of a pixel in the tiled scene is obtained by accumulating the contribution of each ray segment of the ray:

[0025]

[0026] In the formula, C final (r) is the final color of a pixel in the block scene, T k is the transmittance of the kth segment, C k The color value of this segment.

[0027] In some embodiments, including:

[0028] Independent distributed training is performed on the 3DGS module corresponding to each block scene. The 3DGS module corresponding to each block scene processes the light segment that it is responsible for. Among them, the 3DGS modules corresponding to all light segments of the light across multiple block scenes share the transmittance and transparency parameters.

[0029] After the training is completed, the rendering results of all block scenes are summarized.

[0030] In some embodiments, including:

[0031] For all ray segments of a ray across a tiled scene, the 3DGS modules corresponding to the ray use the shared parameter T final To express the transmittance:

[0032] T final =∏ i exp(-σ i ·l i );

[0033] In the formula, σ i is the density of the i-th light segment, l i is the length of the i-th light segment;

[0034] For the 3DGS modules corresponding to all ray segments of the ray across the tiled scene, a shared parameter α is used to represent transparency:

[0035]

[0036] In the formula, σ i is the density of the i-th light segment, Δt i is the length of the i-th light segment.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The point cloud data of the scene is processed in blocks by using an axis-aligned bounding box to obtain several non-overlapping block scenes, wherein one block scene corresponds to one 3DGS module; the intersection of the light and the boundary of the axis-aligned bounding box is detected, and the light is segmented based on the intersection to obtain several light segments; the part of the scene passed by the segmented light segment is distributed to the corresponding 3DGS module for rendering to obtain the block scene rendering result; the rendering results of each block scene are merged to obtain the overall rendering result of the scene; the scene is divided into multiple non-overlapping block scenes to avoid redundant calculations and improve calculation efficiency; the light is segmented and distributed to the corresponding 3DGS modules to achieve efficient management of the light between different block scenes; the block scene rendering results of each block scene are merged through the segmented rendering equation to ensure the overall visual effect of the scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The embodiments of the present invention are further described below with reference to the accompanying drawings:

[0040] Figure 1 A schematic diagram of the implementation flow of a segmented rendering method based on three-dimensional Gaussian sputtering provided in an embodiment of the present invention;

[0041] Figure 2A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0043] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0044] If similar descriptions of "first\second\third" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0046] Based on the problems existing in the related art, an embodiment of the present invention provides a segmented rendering method based on three-dimensional Gaussian sputtering, and the execution subject of the rendering method can be an electronic device. The electronic device can be various types of terminals such as a laptop, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device), and can also be implemented as a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content De l ivery Network, CDN), and big data and artificial intelligence platforms.

[0047] In some embodiments, the functions implemented by the rendering method provided by the embodiments of the present invention may be implemented by a processor of an electronic device calling a program code, wherein the program code may be stored in a computer storage medium.

[0048] The embodiment of the present invention provides a segmented rendering method based on three-dimensional Gaussian sputtering. Figure 1 A schematic diagram of the implementation flow of the segmented rendering method based on three-dimensional Gaussian sputtering provided in an embodiment of the present invention is shown in FIG. Figure 1 As shown, including:

[0049] Step S1: Use an axis-aligned bounding box to block the point cloud data of the scene to obtain a number of non-overlapping block scenes, where one block scene corresponds to one 3DGS module;

[0050] In some embodiments, the step S1 includes:

[0051] Step S11: loading the point cloud data of the scene and preprocessing the point cloud data;

[0052] In some embodiments, including:

[0053] The preprocessing includes denoising, simplification or resampling.

[0054] In the embodiment of the present invention, the data quality is ensured by denoising, simplifying or resampling the point cloud data of the scene.

[0055] Step S12: assigning initial Gaussian distribution parameters to the preprocessed point cloud data, wherein the Gaussian distribution parameters include position, scale matrix, color value and density value;

[0056] Step S13: Use an axis-aligned bounding box to perform block processing on the point cloud data assigned with Gaussian distribution parameters to obtain a number of non-overlapping block scenes.

[0057] In the embodiment of the present invention, when constructing a space division structure, an axis-aligned bounding box can be used as the boundary of a node to facilitate rapid determination of a space region. Therefore, the point cloud data in the scene is processed in blocks by using an axis-aligned bounding box. When performing the block processing, the entire scene can be divided in the screen space to obtain a plurality of block scenes, and the block rules can be adaptively adjusted according to the size and complexity of the scene. Each block scene is a non-overlapping area, which avoids redundant calculations of the traditional overlapping area method and improves the calculation efficiency.

[0058] Step S2: detecting intersections of the light ray and the boundary of the axis-aligned bounding box, and segmenting the light ray based on the intersections to obtain a plurality of light ray segments;

[0059] In some embodiments, the step S2 comprises:

[0060] Step S21: If the intersection points of the ray r and the axis-aligned bounding box boundary are defined as t0 and t1 respectively, the formula is as follows:

[0061] r(t)=o+t·d,t0≤t≤t1;

[0062] Where t is a scalar parameter, r(t) is the point on the ray at the scalar parameter t, o is the starting point of the ray, and d is the direction vector.

[0063] In the embodiment of the present invention, the above formula is used to determine the intersection of a ray and an axis-aligned bounding box when performing ray tracing or collision detection in computer graphics. By calculating t0 and t1, it is known in which range the ray intersects with the axis-aligned bounding box. Thus, the ray is segmented according to the starting point, end point and intersection point of the ray to obtain ray segments.

[0064] Step S3: distribute the part of the scene through which the segmented light segment passes to the corresponding 3DGS module for rendering, and obtain the block scene rendering result;

[0065] In some embodiments, step S3 includes:

[0066] Step S31: The 3DGS module calculates the color contribution of the light segment using the piecewise integral formula:

[0067] C(r)=∑ k T(t k )·σ(t k )·C(t k )·Δt;

[0068] Where C(r) is the color of a pixel in the block scene, T(t k ) is the transmittance, σ(t k ) is the density, C(t k ) is the color value, and Δt is the distance.

[0069] In the embodiment of the present invention, the scene through which the light passes is divided into many small segments, and then each segment is integrated to calculate the color contribution of the light in the segment. k ) is used to adjust the intensity of light, because light gradually weakens when passing through matter, and the density σ(t k ) and color value C(t k ) is used to determine the color contribution of the material in each segment. Finally, the color contributions of all light segments are accumulated to obtain the color C(r) of a pixel in the block scene. Thus, the part of the scene passed by the segmented light segment is distributed to the corresponding 3DGS module for rendering, and the block scene rendering result is obtained.

[0070] Step S4: Merge the rendering results of each block scene to obtain the overall rendering result of the scene.

[0071] In some embodiments, the step S4 comprises:

[0072] Step S41: For a light ray that spans multiple segmented scenes, the final color of a pixel in the segmented scenes is obtained by accumulating the contribution of each light ray segment of the light ray:

[0073]

[0074] In the formula, C final (r) is the final color of a pixel in the block scene, T k is the transmittance of the kth segment, C k The color value of this segment.

[0075] In an embodiment of the present invention, for each block scene, the portion of the light passing through the block scene is rendered separately, and the color and transparency value of the light in each block need to be calculated to obtain the block scene rendering result. The rendering results of each block scene are merged to ensure that the light across the block scenes is visually consistent and avoid seams. The rendering results of different block scenes can be mixed or interpolated to ensure a smooth transition of color and transparency. In the merging stage, the visual smooth transition at the boundary is ensured by summing the rendering results of the light segments of each light to ensure the continuity and smoothness of the final image. For light that spans multiple block scenes, the final color of a pixel in the block scene can be obtained by accumulating the contribution of each light segment of the light, that is, multiplying the color value and transmittance of the light passing through each block scene, and then adding up these products to obtain the final color value. It can ensure that the calculation of color and transparency is continuous and consistent when the light passes through multiple block scenes, thereby visually avoiding seams and discontinuities. In this way, a smooth and realistic volume rendering image can be generated. In addition, the light at the junction of the block scenes is blended or interpolated to ensure smooth and seamless visual effects.

[0076] In summary, the point cloud data of the scene is processed in blocks using an axis-aligned bounding box to obtain several non-overlapping block scenes, wherein one block scene corresponds to one 3DGS module; the intersection of the light ray and the boundary of the axis-aligned bounding box is detected, and the light ray is segmented based on the intersection to obtain several light segments; the partial scene passed by the segmented light segment is distributed to the corresponding 3DGS module for rendering to obtain the block scene rendering result; the rendering results of each block scene are merged to obtain the overall rendering result of the scene; the scene is divided into multiple non-overlapping block scenes to avoid redundant calculations and improve calculation efficiency; the light ray is segmented and distributed to the corresponding 3DGS modules to achieve efficient management of the light between different block scenes; the block scene rendering results of each block scene are merged through the segmented rendering equation to ensure the overall visual effect of the scene.

[0077] In some embodiments, including:

[0078] Step S100: performing independent distributed training on the 3DGS module corresponding to each block scene, wherein the 3DGS module corresponding to each block scene processes the light segment that it is responsible for, wherein the transmittance and transparency parameters are shared between the 3DGS modules corresponding to all light segments of the light across multiple block scenes;

[0079] Step S200: After the training is completed, the rendering results of all the block scenes are summarized.

[0080] In some embodiments, including:

[0081] Step S101: For the 3DGS modules corresponding to all the light segments of the light across the block scene, use the shared parameter T fimal To express the transmittance:

[0082] T fimsl =∏ i exp(-σ i ·l i );

[0083] In the formula, σ i is the density of the i-th light segment, l i is the length of the i-th light segment;

[0084] Step S102: For the 3DGS modules corresponding to all the light segments of the light across the block scene, a shared parameter α is used to represent transparency:

[0085]

[0086] In the formula, σ i is the density of the i-th light segment, Δt i is the length of the i-th light segment.

[0087] In the embodiment of the present invention, each section of the medium will absorb light to a certain extent, and this absorption effect is expressed by σ i ·l i To quantify. The exponential function exp(-σ i ·l i ) represents the remaining intensity ratio of the light in each medium. By multiplying the transmittance of all light segments, the transmittance T of the light after passing through all media is calculated. final Based on the principle of exponential decay, where σ i ·Δt i represents the amount of light absorbed in the i-th segment. The entire expression The total absorption of the light along the entire path is calculated. Then, the final transparency is calculated using 1-exp(-x), where x is the total absorption. This ensures that the intensity and color of the light will be correctly attenuated according to the absorption and scattering properties when passing through volumes of different densities.

[0088] It should be noted that, in the embodiment of the present invention, if the above-mentioned rendering method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a magnetic disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0089] Accordingly, an embodiment of the present invention provides a storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps in the rendering method provided in the above embodiment are implemented.

[0090] An embodiment of the present invention provides an electronic device; Figure 2 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 2As shown, the electronic device 400 includes: a processor 401, at least one communication bus 402, a user interface 403, at least one external communication interface 404, and a memory 405. The communication bus 402 is configured to realize the connection and communication between these components. The user interface 403 may include a display screen, and the external communication interface 404 may include a standard wired interface and a wireless interface. The processor 401 is configured to execute the program of the rendering method stored in the memory to implement the steps in the rendering method provided in the above embodiment.

[0091] It should be noted that the description of the above storage medium and electronic device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding.

[0092] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0093] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, object or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, object or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, object or device including the element.

[0094] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0095] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0096] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0097] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read On ly Memory), disks or optical disks, etc. Various media that can store program codes.

[0098] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product, which is stored in a storage medium and includes several instructions for a controller to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0099] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A segmented rendering method based on three-dimensional Gaussian sputtering, characterized in that: include: Use axis-aligned bounding boxes to block the point cloud data of the scene to obtain several non-overlapping block scenes, where one block scene corresponds to one 3DGS module; detecting intersections of the ray with the boundary of the axis-aligned bounding box, and segmenting the ray based on the intersections to obtain a plurality of ray segments; Distribute the part of the scene that the segmented light segment passes through to the corresponding 3DGS module for rendering to obtain the block scene rendering result; Merge the rendering results of each block scene to get the overall rendering result of the scene.

2. The method according to claim 1, characterized in that: include: Load the point cloud data of the scene and preprocess the point cloud data; Assigning initial Gaussian distribution parameters to the preprocessed point cloud data, wherein the Gaussian distribution parameters include position, scale matrix, color value and density value; The point cloud data assigned with Gaussian distribution parameters is divided into blocks using an axis-aligned bounding box to obtain several non-overlapping block scenes.

3. The method according to claim 2, characterized in that include: The preprocessing includes denoising, simplification or resampling.

4. The method according to claim 1, characterized in that include: If the intersection points of the ray r and the axis-aligned bounding box boundary are defined as t0 and t1 respectively, the formula is expressed as follows: r(t)=o+t·d,t0≤t≤t1; Where t is a scalar parameter, r(t) is the point on the ray at the scalar parameter t, o is the starting point of the ray, and d is the direction vector.

5. The method according to claim 1, characterized in that include: The 3DGS module calculates the color contribution of the light segment through the piecewise integration formula: C(r)=∑ k T(t k )·σ(t k )·C(t k )·Δt; Where C(r) is the color of a pixel in the block scene, T(t k ) is the transmittance, σ(t k ) is the density, C(t k ) is the color value, and Δt is the distance.

6. The method according to claim 1, characterized in that include: For a ray that spans multiple tiled scenes, the final color of a pixel in the tiled scene is obtained by accumulating the contribution of each ray segment of the ray: In the formula, C final (r) is the final color of a pixel in the block scene, T k is the transmittance of the kth segment, C k The color value of this segment.

7. The method according to claim 1, characterized in that include: Independent distributed training is performed on the 3DGS module corresponding to each block scene. The 3DGS module corresponding to each block scene processes the light segment that it is responsible for. Among them, the 3DGS modules corresponding to all light segments of the light across multiple block scenes share the transmittance and transparency parameters. After the training is completed, the rendering results of all block scenes are summarized.

8. The method according to claim 7, characterized in that include: For all ray segments of a ray across a tiled scene, the 3DGS modules corresponding to the ray use the shared parameter T final To express the transmittance: T final =∏ i exp(-σ i ·l i ); In the formula, σ i is the density of the i-th light segment, l i is the length of the i-th light segment; For the 3DGS modules corresponding to all ray segments of the ray across the tiled scene, a shared parameter α is used to represent transparency: In the formula, σ i is the density of the i-th light segment, Δt i is the length of the i-th light segment.

Images