Scene rendering method and device, equipment, storage medium and program product

Abstract

The invention provides a scene rendering method and device, equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence, and the scene rendering method comprises the steps: obtaining a target sparse point cloud of a scene according to a plurality of view images of the scene; segmenting the scene according to the target sparse point clouds to obtain different target scene blocks and target sparse point clouds in the different target scene blocks; for each target scene block, converting the target sparse point cloud in the target scene block into a Gaussian point cloud by using a 3DGS model; the scene is rendered according to the Gaussian point cloud of the target LOD in a Gaussian level of detail tree constructed by the Gaussian point cloud, and the Gaussian point cloud comprises Gaussian point clouds of a plurality of LOD. According to the method, the consumption of computing resources required during large-scale scene modeling is reduced, and the rendering effect is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a scene rendering method, apparatus, device, storage medium, and program product. Background Technology

[0002] VastGaussian is a high-quality reconstruction and real-time rendering method for large scenes based on 3D Gaussian Splatting (3DGS). In the VastGaussian method, a progressive partitioning strategy is used to divide the complete scene into multiple sub-blocks, which enables detailed modeling of large scenes, reduces memory requirements and optimization time, and is particularly effective when using multiple graphics processing units (GPUs) for parallel optimization.

[0003] While VastGaussian can achieve 3D reconstruction of large-scale scenes, its progressive partitioning strategy does not comprehensively consider the optimal partitioning scheme based on scene layout, cell count, and training camera distribution. This may lead to unreasonable scene partitioning in practice, affecting subsequent optimization and rendering results. Furthermore, when the scene is huge, there will be a large number of 3D Gaussian distributions. Processing such a large number of Gaussians will significantly reduce rendering speed and increase computational resource consumption. Summary of the Invention

[0004] This invention provides a scene rendering method, apparatus, device, storage medium, and program product to solve the problems of high resource consumption and poor rendering effect in existing scene reconstruction and rendering methods.

[0005] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, embodiments of the present invention provide a scene rendering method, including:

[0007] Obtain the target sparse point cloud of the scene based on multiple view images of the scene;

[0008] The scene is segmented based on the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in different target scene blocks;

[0009] For each target scene block, the sparse point cloud of the target scene block is converted into a Gaussian point cloud using a 3D Gaussian splash 3DGS model.

[0010] The scene is rendered based on the Gaussian point cloud in the Gaussian level of detail tree constructed from the Gaussian point cloud, wherein the Gaussian point cloud includes Gaussian point clouds with multiple levels of detail.

[0011] Optionally, obtaining the target sparse point cloud of the scene based on multiple view images of the scene includes:

[0012] The sparse point cloud of the scene is obtained from multiple view images of the scene;

[0013] Based on the average value and standard deviation of the sparse point cloud in the first direction, the sparse point cloud is filtered in the first direction to obtain the first filtering result of the sparse point cloud.

[0014] Based on the average value and standard deviation of the sparse point cloud in the second direction, the sparse point cloud is filtered in the second direction to obtain the second filtering result of the sparse point cloud.

[0015] Based on the preset distribution range of the third direction, the sparse point cloud is filtered in the third direction to obtain the third filtering result of the sparse point cloud.

[0016] The target sparse point cloud is obtained based on the first screening result, the second screening result, the third screening result, and the sparse point cloud.

[0017] Wherein, the first direction is perpendicular to the second direction, and the third direction is perpendicular to both the first direction and the second direction.

[0018] Optionally, the step of segmenting the scene based on the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in different target scene blocks includes:

[0019] The target sparse point cloud is mapped to a two-dimensional space to obtain an initial two-dimensional point cloud;

[0020] The scene is segmented according to the preset number of segments and the spatial range of the initial two-dimensional point cloud to obtain an initial scene block;

[0021] Based on the preset overlap ratio between adjacent initial scene blocks and the initial scene blocks and the initial two-dimensional point cloud, obtain the first scene block and the first point cloud in the first scene block;

[0022] Based on a preset first point count threshold and the first point cloud in the first scene block, the first scene block is segmented to obtain a second scene block and a second point cloud in the second scene block;

[0023] Based on the preset second point count threshold, the preset third point count threshold, and the second point cloud in the second scene block, adjacent second scene blocks are merged to obtain the target scene block and the target sparse point cloud in the target scene block.

[0024] Wherein, the second point threshold is less than the third point threshold.

[0025] Optionally, the step of segmenting the first scene block according to a preset first point count threshold and a first point cloud in the first scene block to obtain a second scene block and a second point cloud in the second scene block includes:

[0026] If the number of points in the first point cloud in the first target scene block is greater than or equal to the first point number threshold, the segmentation direction of the first target scene block is determined based on the number of points in the first point cloud mapped to the first direction and the number of points in the first target scene block mapped to the second direction, wherein the first direction is perpendicular to the second direction.

[0027] Based on the position corresponding to the first median, the first target scene block is segmented to obtain a second target scene block and a first point cloud in the second target scene block, wherein the first median is the median of the number of points in the first target scene block mapped to the first point cloud in the segmentation direction;

[0028] If the first condition is met, the second target scene block is used as the second scene block, and the first point cloud in the second target scene block is used as the second point cloud in the second scene block;

[0029] If the first condition is not met, the second target scene block is used as the first target scene block, and the step of determining the segmentation direction of the first target scene block based on the number of points in the first target scene block mapped to the first point cloud in the first direction and the number of points in the first target scene block mapped to the first point cloud in the second direction is returned until the first condition is met.

[0030] The first condition includes at least one of the following:

[0031] The first target scene block is segmented more times than a preset number;

[0032] The number of points in the first point cloud in the second target scene block is less than the first point count threshold;

[0033] Wherein, the first target scene block is any one of the scene blocks in the first scene block.

[0034] Optionally, the step of merging adjacent second scene blocks according to a preset second point count threshold, a preset third point count threshold, and the second point cloud in the second scene block to obtain the target scene block and the target sparse point cloud in the target scene block includes:

[0035] If the number of points in the second point cloud in the third target scene block is less than the second point count threshold, and the second condition is met, the third target scene block and the fourth target scene block are merged to obtain a third scene block and a third point cloud in the third scene block. The third target scene block is any scene block in the second scene block, and the fourth target scene block is a scene block in the second scene block that is adjacent to the third target scene block. The second condition includes that the number of points in the merged second point cloud in the third target scene block and the second point cloud in the fourth target scene block is less than the third point count threshold.

[0036] Obtain the view image of the scene corresponding to the third point cloud in the third scene block;

[0037] The camera corresponding to the view image is determined based on the Laplacian operator value of the view image;

[0038] When the number of cameras is greater than a preset number, the third scene block corresponding to the view image is taken as the target scene block, and the third point cloud in the third scene block corresponding to the view image is taken as the target sparse point cloud in the target scene block.

[0039] Optionally, before rendering the scene based on the Gaussian point cloud containing the target level of detail (LOD) in the Gaussian level of detail tree constructed from the Gaussian point cloud, the method further includes:

[0040] Based on the spatial extent of the Gaussian point cloud, obtain the Gaussian bounding box of the Gaussian point cloud;

[0041] Using the bisection median segmentation method, the Gaussian point cloud is split into leaf nodes of the Gaussian detail tree within the Gaussian bounding box, and each leaf node includes one Gaussian point cloud.

[0042] Starting from the last leaf node of the Gaussian detail tree, the Gaussian point cloud in each leaf node is sequentially fused into the Gaussian point cloud in the leaf node above the previous layer, in order from bottom to top, to obtain the fused Gaussian point cloud corresponding to each leaf node.

[0043] If the boundary granularity value of the fused Gaussian point cloud corresponding to the leaf node of the first target detail level is less than the preset granularity value, and the boundary granularity value of the fused Gaussian point cloud corresponding to the leaf node of the second target detail level is greater than or equal to the preset granularity value, then the fused Gaussian point cloud corresponding to the leaf node of the first target detail level is taken as the Gaussian point cloud of the target detail level. In the Gaussian detail level tree, the leaf node of the second target detail level is the parent node of the leaf node of the first target detail level.

[0044] Secondly, embodiments of the present invention also provide a scene rendering apparatus, comprising:

[0045] The first processing module is used to obtain the target sparse point cloud of the scene based on multiple view images of the scene;

[0046] The second processing module is used to segment the scene according to the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in different target scene blocks.

[0047] The third processing module is used to convert the target sparse point cloud in the target scene block into a Gaussian point cloud using a 3D Gaussian splash 3DGS model for each target scene block.

[0048] The fourth processing module is used to render the scene based on the Gaussian point cloud of the target level of detail (LOD) in the Gaussian level of detail tree constructed from the Gaussian point cloud, wherein the Gaussian point cloud includes Gaussian point clouds with multiple LODs.

[0049] Thirdly, embodiments of the present invention also provide a scene rendering device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the file sending method as described in any one of the first aspects.

[0050] Fourthly, embodiments of the present invention also provide a readable storage medium storing a program, which, when executed by a processor, implements the scene rendering method as described in any one of the first aspects.

[0051] Fifthly, embodiments of the present invention also provide a computer program product, including computer instructions, which, when executed by a processor, implement the scene rendering method as described in any one of the first aspects.

[0052] The beneficial effects of this invention are:

[0053] The scene rendering method provided by this invention obtains a target sparse point cloud of the scene based on multiple view images of the scene. The scene is then segmented based on the target sparse point cloud to obtain different target scene blocks and target sparse point clouds within each target scene block. For each target scene block, a 3DGS model is used to convert the target sparse point cloud within the target scene block into a Gaussian point cloud. The Gaussian point cloud includes multiple levels of detail (LODs) of Gaussian point clouds. The scene is rendered based on the target level of detail (LOD) Gaussian point clouds constructed from these Gaussian point clouds. The LOD Gaussian point cloud includes multiple levels of detail (LODs) of Gaussian point clouds. By designing a scene segmentation strategy combined with the LOD concept, efficient modeling and visualization of large-scale scenes are achieved, reducing the computational resource consumption required for large-scale scene modeling. Utilizing the inherent characteristics of 3D Gaussian, a level-of-detail structure is constructed, and the scene is rendered based on the target LOD Gaussian point cloud, thus improving the rendering effect. Attached Figure Description

[0054] Figure 1 A schematic diagram illustrating the scene reconstruction process of the 3DGS model improved by this invention;

[0055] Figure 2 A flowchart illustrating the scene rendering method provided in an embodiment of the present invention;

[0056] Figure 3 This represents a technical roadmap of the scene rendering method provided in the embodiments of the present invention;

[0057] Figure 4 This is a schematic diagram of the structure of the scene rendering device provided in an embodiment of the present invention;

[0058] Figure 5 This is a schematic diagram showing the structure of the scene rendering device provided in an embodiment of the present invention. Detailed Implementation

[0059] To make the technical problems, technical solutions, and advantages of this application clearer, a detailed description will be provided below in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as particular configurations and components are provided merely to aid in a comprehensive understanding of the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Furthermore, for clarity and brevity, descriptions of known functions and structures have been omitted.

[0060] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0061] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0062] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, "A or B" covers three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0063] The term "instruction" in this application can be either a direct instruction (or explicit instruction) or an indirect instruction (or implicit instruction). A direct instruction can be understood as one in which the sender explicitly informs the receiver of specific information, the operation to be performed, or the requested result, etc.; an indirect instruction can be understood as one in which the receiver determines the corresponding information based on the instruction sent by the sender, or makes a judgment and determines the operation to be performed or the requested result, etc., based on the judgment result.

[0064] Before describing specific embodiments of the present invention, the following explanation is provided:

[0065] The scene reconstruction process of 3DGS models is as follows: Figure 1As shown, the 3D Gaussian Splatting (3DGS) model was proposed by Kerbl et al. in 2023. It takes point clouds (SfM Points) obtained by sparse reconstruction of multi-view images (Camera) and Structure from Motion (SfM) as input, initializes and generates millions of 3D Gaussians in 3D space, and obtains Gaussian clouds through Gaussian splitting and replication. The Gaussians are then projected onto the imaging space, and differentiable tile rendering is performed in the form of rasterization based on alpha-blending. The red, green, and blue (RGB) loss is calculated and transferred using the rendering results and ground truth (Image) (Adapitive Density Control), and finally, high-quality 3D reconstruction is achieved.

[0066] To address the problems of high resource consumption and poor rendering effects in existing scene reconstruction and rendering methods, embodiments of the present invention provide a scene rendering method, apparatus, device, storage medium, and program product.

[0067] like Figure 2 As shown, an embodiment of the present invention provides a scene rendering method, including:

[0068] Step 201: Obtain the target sparse point cloud of the scene based on multiple view images of the scene.

[0069] The multiple view images are multi-view images taken by a drone.

[0070] In this step, the target sparse point cloud is obtained by processing the sparse point cloud of the scene.

[0071] Step 202: Segment the scene according to the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in different target scene blocks.

[0072] In this step, based on the three-dimensional distribution characteristics of the target sparse point cloud, scene segmentation is performed by three steps: constructing an initial grid, iterative grid segmentation, and merging adjacent grids block by block, resulting in different target scene blocks and target sparse point clouds in different target scene blocks.

[0073] Step 203: For each target scene block, use the 3D Gaussian splash 3DGS model to convert the target sparse point cloud in the target scene block into a Gaussian point cloud.

[0074] In this step, the 3DGS model is used to convert the sparse point cloud of the target scene into a Gaussian point cloud for each segmented target scene block.

[0075] Step 204: Render the scene based on the Gaussian point cloud of the target level of detail (LOD) in the Gaussian level of detail tree constructed from the Gaussian point cloud, wherein the Gaussian level of detail tree includes Gaussian point clouds with multiple LODs.

[0076] In this step, a Gaussian level of detail tree (or Gaussian structure tree) is constructed based on the Gaussian point cloud. This Gaussian level of detail tree includes Gaussian point clouds with multiple levels of detail (LOD), i.e., multiple levels of Gaussian point clouds. The target LOD Gaussian point cloud is determined from these multiple LOD Gaussian point clouds, and rendering is performed based on this target LOD Gaussian point cloud to obtain the rendering result of the scene. The above steps, by designing a scene partitioning strategy combined with the LOD concept, achieve efficient modeling and visualization of large-scale scenes, reducing the computational resource consumption required for large-scale scene modeling. By utilizing the inherent characteristics of 3D Gaussians to construct a level of detail structure and rendering the scene based on the target LOD Gaussian point cloud, the rendering effect is improved.

[0077] In some embodiments, obtaining the target sparse point cloud of the scene based on multiple view images of the scene includes:

[0078] The sparse point cloud of the scene is obtained from multiple view images of the scene.

[0079] This involves performing geographic scene data preprocessing. Specifically, the input is multi-view images from a UAV. The multi-view images are downsampled, and the depth values ​​of the downsampled multi-view images are obtained using the Depth Pixel Transport (DPT) framework. Feature extraction and feature matching are performed on the depth values, along with camera pose calculation, to obtain the scene features. Colmap is then used to reconstruct the scene features, resulting in a sparse point cloud of the scene.

[0080] Based on the average value and standard deviation of the sparse point cloud in a first direction, the sparse point cloud is filtered in the first direction to obtain a first filtering result of the sparse point cloud; based on the average value and standard deviation of the sparse point cloud in a second direction, the sparse point cloud is filtered in the second direction to obtain a second filtering result of the sparse point cloud.

[0081] Based on a preset distribution range in a third direction, the sparse point cloud is filtered in the third direction to obtain a third filtering result of the sparse point cloud. The first direction is perpendicular to the second direction, and the third direction is perpendicular to both the first and second directions.

[0082] In this embodiment, the first direction is used as the constructed three-dimensional coordinate system. Taking the axial direction as an example, in the three-dimensional coordinate system constructed with the second direction as the reference, Taking the axial direction as an example, in a three-dimensional coordinate system constructed with the third direction as the reference, Let's take the axial direction as an example for explanation.

[0083] It should be noted that the process of filtering the sparse point cloud in the first direction based on its average value and standard deviation in the first direction to obtain a first filtering result is consistent with the process of filtering the sparse point cloud in the second direction based on its average value and standard deviation in the second direction to obtain a second filtering result. In this embodiment, the process of filtering the sparse point cloud in the first direction based on its average value in the first direction will be used for illustration.

[0084] Sparse point clouds Axial direction, It exhibits characteristics of a normal distribution along the axis, and the Z-score method is used to analyze it. Axial direction, The sparse point clouds along the axial direction are filtered separately. The sparse point clouds are then mapped to... axis, The axis is used for filtering in two directions.

[0085] against Sparse point cloud mapped along the axis, for each data point Calculate its average value with the point cloud set. The difference between (i.e., the average value of the sparse point cloud in the first direction) and the difference divided by the standard deviation of the point cloud set. (i.e., the standard deviation of the sparse point cloud in the first direction) yields the position of the data points relative to the center of the data distribution. (That is, the center position of the sparse point cloud in the first direction), the calculation formula is as follows:

[0086]

[0087] Will Z-scores with an absolute value exceeding 2 are considered outliers. These outliers deviate from the mean by more than two standard deviations. Based on the characteristics of a normal distribution, a sufficient number of center points can be retained, i.e., along the first direction. Sparse point clouds with a value greater than 2 are removed, resulting in the first screening result for sparse point clouds in the first direction.

[0088] Optionally, the preset distribution range is a preset percentile distribution range of the sparse point cloud in the third direction, for example, the preset distribution range is the range between the 2.5 percentile and the 99th percentile.

[0089] It should be noted that the mapping to In a sparse point cloud along the axial direction, most data points are concentrated at the cluster center, with only a very small number of data points extremely far from the cluster center. Therefore, this embodiment uses percentile filtering to remove these loose points and map them to... The data points in the sparse point cloud along the axis are arranged in descending order. The data points between the 2.5th percentile and the 99th percentile after the arrangement are taken as the third filtering result of the sparse point cloud, as shown below:

[0090]

[0091] in, This represents the third screening result of the sparse point cloud. Indicates the 2.5 percentile. This represents the 99th percentile.

[0092] Based on the first, second, and third screening results, the target sparse point cloud is selected from the sparse point cloud.

[0093] In this embodiment, before scene segmentation and training based on sparse point clouds, filtering based on Z-Score and percentiles is performed to improve scene reconstruction results.

[0094] In some embodiments, segmenting the scene based on the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in different target scene blocks includes:

[0095] The target sparse point cloud is mapped to a two-dimensional space to obtain an initial two-dimensional point cloud.

[0096] Specifically, the 3D target sparse point cloud is mapped to... From a plane (i.e., two-dimensional space), we construct an initial two-dimensional point cloud (or initial two-dimensional point cloud set), which is represented as: .

[0097] Calculate the initial 2D point cloud (or initial 2D point cloud set). In the first direction (i.e.) axial direction) and second direction (i.e. The maximum and minimum values ​​in the two axes are used to obtain the spatial range of the initial two-dimensional point cloud.

[0098] The scene is segmented according to the preset number of segments (or preset number of blocks) and the spatial range of the initial two-dimensional point cloud to obtain the initial scene block.

[0099] The preset number of segments includes the first direction (i.e. Number of segments (in the axial direction) Second direction (i.e.) Number of segments (in the axial direction) Optionally, and They are 2 and 3 respectively. According to the first direction (i.e. Number of segments (in the axial direction) Second direction (i.e.) Number of segments (in the axial direction) The spatial extent of the 2D point cloud is segmented to obtain initial scene blocks. (or initial submesh), width of the initial scene block (i.e., dimensions in the first direction) and height (i.e., the dimension in the second direction).

[0100] Based on the preset overlap ratio between adjacent initial scene blocks and the initial scene blocks and the initial two-dimensional point cloud, obtain the first scene block and the first point cloud in the first scene block.

[0101] Specifically, a preset overlap ratio is set between two adjacent initial scene blocks. It will be based on the preset overlap ratio The first scene block is obtained by overlapping the initial scene blocks, based on the width of the initial scene blocks. (i.e., dimensions in the first direction) and height (i.e., the dimension in the second direction) and the preset overlap ratio Calculate each first scene block The coordinates of the lower left corner ( ) and the coordinates of the lower right corner ( ), which is represented as follows:

[0102]

[0103] Based on the coordinates of the lower left corner ( ) and the coordinates of the lower right corner ( ), count the initial two-dimensional point cloud contained in each first scene block, that is, obtain the first point cloud in each first scene block.

[0104] The above method, through projection, allows the point set of the first point cloud to better reflect the distribution of the plane, while also taking into account the third direction (i.e., The distribution characteristics along the axis direction were used to complete the mesh initialization of the point cloud.

[0105] It should be noted that, assuming a uniform point cloud distribution, the ideal scenario would be that the total number of points in the point cloud is... The number of scene blocks (or sub-mesh) is Therefore, the point cloud within each scene block (or each sub-grid) should be: In order to ensure that the number of sparse point clouds of targets in the target scene block is distributed as evenly as possible, in this embodiment:

[0106] Based on a preset first point threshold and the first point cloud in the first scene block, the first scene block is segmented to obtain a second scene block and a second point cloud in the second scene block.

[0107] Based on the preset second point count threshold, the preset third point count threshold, and the second point cloud in the second scene block, adjacent second scene blocks are merged to obtain the target scene block and the target sparse point cloud in the target scene block; wherein, the second point count threshold is less than the third point count threshold.

[0108] In this embodiment, the constructed first scene block is subjected to grid iteration segmentation and adjacent grids are merged block by block to obtain different target scene blocks and target sparse point clouds with uniform distribution in different target scene blocks.

[0109] Specifically, the first scene block, whose number of first point clouds exceeds the first point count threshold, is segmented to obtain the second scene block and the second point cloud in the second scene block.

[0110] And merge second scene blocks whose number of second point clouds is less than the second point count threshold, ensuring that the number of point clouds in the merged scene block does not exceed the third point count threshold. For example, the third point count threshold is twice the second point count threshold.

[0111] The following example illustrates the process of segmenting the first scene block, using the segmentation of the first target scene block (which can be any scene block within the first scene block) as an example. Optionally, the segmentation of the first scene block based on a preset first point count threshold and the first point cloud in the first scene block to obtain a second scene block and a second point cloud in the second scene block includes:

[0112] If the number of points in the first point cloud of the first target scene block is greater than or equal to the first point count threshold, the segmentation direction of the first target scene block is determined based on the number of points in the first point cloud mapped to the first direction and the number of points in the first point cloud mapped to the second direction. The first direction (i.e., ...) The axial direction is perpendicular to the second direction (i.e., the axial direction) (Axial direction).

[0113] Specifically, if the number of first point clouds in the first target scene block exceeds a first point count threshold, the first target scene block is divided (or split) into two scene blocks. It is necessary to ensure that the split scene blocks (or sub-mesh) are still rectangular, and the difference in point count between the scene blocks (or sub-mesh) is minimal. Before splitting, the splitting direction needs to be determined. The number of first point clouds in the first target scene block mapped to the first direction is obtained. The variance of the first point cloud in the first direction is determined based on the number of first point clouds in the first direction, and the variance of the first point cloud in the second direction is determined based on the number of first point clouds in the first target scene block mapped to the second direction. The direction with the larger variance between the variances in the first and second directions is selected as the splitting direction.

[0114] Based on the position corresponding to the first median, the first target scene block is segmented to obtain a second target scene block and a first point cloud in the second target scene block, wherein the first median is the median number of points in the first target scene block mapped to the first point cloud in the segmentation direction.

[0115] On the selected segmentation direction (or splitting direction), select the median of the number of points mapped sequentially to the first point cloud on the segmentation direction. Use the mapping position of this median on the mapping direction to segment (or split) the first target scene block on the segmentation direction (or splitting direction) to obtain two second target scene blocks and the first point cloud in each second target scene block.

[0116] If the first condition is met, the second target scene block is used as the second scene block, and the first point cloud in the second target scene block is used as the second point cloud in the second scene block; wherein, the first condition includes at least one of the following:

[0117] The first target scene block was segmented more times than the preset number of times, that is, the maximum number of iterations was reached. Optional, The value is 10;

[0118] The number of points in the first point cloud in the second target scene block is less than the first point count threshold.

[0119] If the first condition is met, namely, the maximum number of iterations is reached or the number of points in the first point cloud of the segmented second target scene block is less than the first point number threshold, then the segmentation (or splitting) is stopped, and the first point cloud in the second target scene block obtained by the segmentation (or splitting) is taken as the second point cloud in the second scene block.

[0120] If the first condition is not met, the second target scene block is used as the first target scene block, and the step of determining the segmentation direction of the first target scene block based on the number of points in the first target scene block mapped to the first point cloud in the first direction and the number of points in the first target scene block mapped to the first point cloud in the second direction is returned until the first condition is met.

[0121] If the first condition is not met, the overlapping is used for segmentation (or splitting) until the number of point clouds in the segmented (or split) scene block is less than the first point number threshold and / or the number of segmentations is greater than the preset number.

[0122] Following the above process, each first scene block with a first point cloud quantity greater than or equal to the first point number threshold is segmented (or split) until the segmentation of each first scene block with a first point cloud quantity greater than or equal to the first point number threshold satisfies the first condition.

[0123] For example, in the determined segmentation direction (or splitting direction) is In the axial direction, the representations of the second target scene blocks on the left and right sides after segmentation (or splitting) and the first point clouds included in the second target scene blocks on the left and right sides are as follows:

[0124]

[0125] in, This represents the coordinates of the bottom left corner of the first target scene block. This represents the coordinates of the bottom right corner of the first target scene block. This indicates the position corresponding to the first median. This represents the second target scene block on the left. This represents the second target scene block on the right.

[0126]

[0127] in, This represents the first point cloud in the second target scene block on the left. This represents the first point cloud in the second target scene block on the right. This represents the first point cloud in the first target scene block.

[0128] It should also be noted that, in order to avoid the point cloud being too densely distributed in some areas, leading to mesh segmentation failure, excessively large differences in the area of ​​the two sub-mesh, or even the appearance of extremely small sub-mesh, it is necessary to make a judgment during segmentation to avoid this problem. In this embodiment, if the difference in the number of first point clouds between the second target scene blocks obtained by splitting the first target scene block exceeds a preset threshold, the splitting behavior is stopped, and the original mesh remains unchanged. Optionally, the preset threshold is equal to 0.7 times the first point cloud threshold.

[0129] The following example illustrates the process of merging the second scene blocks, using the segmentation of the first target scene block (where the third target scene block is any scene block within the second scene block, and the fourth target scene block is a scene block adjacent to the third target scene block) within the second scene block. Optionally, the process involves merging adjacent second scene blocks based on a preset second point count threshold, a preset third point count threshold, and the second point cloud within the second scene block to obtain the target scene block and the target sparse point cloud within the target scene block.

[0130] If the number of points in the second point cloud within the third target scene block is less than the second point count threshold, and if the second condition is met, the third target scene block and the fourth target scene block are merged to obtain a third scene block and a third point cloud within the third scene block. The second condition includes that the number of points in the merged second point cloud of the third target scene block and the second point cloud of the fourth target scene block is less than the third point count threshold. Optionally, the second point count threshold is... The third point threshold is .

[0131] That is, the number of points in the second point cloud within the third target scene block. Less than the second point threshold In this case, attempt to merge the third target scene block with the adjacent fourth target scene block. If the number of points after merging is less than the third point threshold... If so, then perform the merge operation; otherwise, leave the original third target scene block unchanged.

[0132] Optionally, after merging, each merged third scene block is shrunk to make it more compactly surround its internal points, with a shrinkage rate of 99%.

[0133] Obtain the view image of the scene corresponding to the third point cloud in the third scene block.

[0134] The camera corresponding to the view image is determined based on the Laplacian operator value of the view image.

[0135] Specifically, camera images are assigned to each third scene block. First, all cameras are traversed, and valid cameras are selected based on the position of the third point cloud and the position of the camera center. If a camera is within one size of the third scene and has a previous matching relationship with the third point cloud of the third scene, then the camera is a candidate.

[0136] The Laplacian operator is incorporated as a pre-evaluation, and images of poor quality are removed based on their mean and standard deviation. The Laplacian operator is defined as follows:

[0137]

[0138] The size of the Laplacian operator can reflect the sharpness of an image in a local region. The larger the Laplacian operator, the richer the edge information in that region and the sharper the image. Conversely, the smaller the Laplacian operator, the blurrier the image.

[0139] In this embodiment, a standard 3×3 Laplacian convolution kernel is used to calculate the Laplacian operator, as shown below:

[0140]

[0141] Calculate the Laplacian operator value for all view images, based on the mean of the Laplacian operator. and the standard deviation of the Laplace operator Confirmed, with The Laplacian operator value representing the k-th image is used to select the effective camera for the view image according to the following formula, expressed as follows:

[0142]

[0143] When the number of cameras is greater than a preset number, the third scene block corresponding to the view image is taken as the target scene block, and the third point cloud in the third scene block corresponding to the view image is taken as the target sparse point cloud in the target scene block.

[0144] Specifically, the system determines whether a third scene block is the target scene block based on the number of cameras. If the number of cameras is greater than a preset number, the third scene block is considered the target scene block, and the third point cloud in the third scene block is the target sparse point cloud in the target scene block. Otherwise, the third scene block is skipped. Optionally, the preset number is 50.

[0145] The following details the construction process of the Gaussian level of detail tree (or Gaussian structure tree). In some embodiments, before rendering the scene based on the Gaussian point cloud containing the target level of detail (LOD) in the Gaussian level of detail tree constructed from the Gaussian point cloud, the method further includes:

[0146] Based on the spatial extent of the Gaussian point cloud, obtain the Gaussian bounding box of the Gaussian point cloud.

[0147] Specifically, an axis-aligned bounding box (AABB) is constructed for the Gaussian point cloud, which is the smallest bounding box in the spatial range of the Gaussian point cloud where all faces are parallel to the coordinate axes, and this bounding box is used as the root node for subsequent segmentation and merging operations.

[0148] Using the bisection median segmentation method, the Gaussian point cloud is split into leaf nodes of the Gaussian detail tree within the Gaussian bounding box, and each leaf node includes one Gaussian point cloud.

[0149] Specifically, the Gaussian set within the bounding box is recursively split into leaf nodes based on a binary median split, ensuring that each leaf node contains only one Gaussian. First, the major axis of the current node's Gaussian bounding box is chosen as the splitting axis, better representing the main distribution direction of the data and ensuring that the two subsets after splitting do not overlap excessively in this direction. Then, the mean of all Gaussian point clouds within the current node is projected onto the splitting axis, and the median of these projections is calculated. Finally, the projected values ​​are compared with the median; those above the median are considered a subset, while the rest are all below the median. This operation is recursively performed until each leaf node contains only one or fewer Gaussians, thus constructing a binary tree-structured Gaussian detail hierarchy structure, where each internal node represents a spatial region, and each leaf node represents a trained Gaussian point cloud.

[0150] Starting from the last leaf node of the Gaussian detail tree, the Gaussian point clouds in each leaf node are sequentially merged into the Gaussian point clouds in the leaf node of the previous layer, in order from bottom to top, to obtain the merged Gaussian point cloud corresponding to each leaf node.

[0151] Specifically, a bottom-up Gaussian merging process is performed. After splitting, an N-level binary tree is obtained, where each leaf node in the Nth level is a Gaussian point cloud, and each Gaussian point cloud has specific parameters, including position. Covariance ,transparency spherical harmonic coefficient However, the first N-1 layers are sets rather than individual Gaussian point clouds. To construct Gaussian point clouds with such intermediate nodes, merging and fusing is performed sequentially upwards from the leaf nodes to form new Gaussian point clouds. When merging child nodes, the weights of each node are... , For opacity, This is the covariance matrix. Based on this weight, the size, transparency, and spherical harmonic coefficients of the parent node are calculated, and this step is performed recursively until the entire tree is completed. It is represented as follows:

[0152]

[0153] If the boundary granularity value of the fused Gaussian point cloud corresponding to the leaf node of the first target detail level is less than the preset granularity value, and the boundary granularity value of the fused Gaussian point cloud corresponding to the leaf node of the second target detail level is greater than or equal to the preset granularity value, then the fused Gaussian point cloud corresponding to the leaf node of the first target detail level is taken as the Gaussian point cloud of the target detail level.

[0154] Specifically, when selecting a level, the bounding box of the node's Gaussian shape is first obtained. Using the long side (maximum dimension) of the bounding box and the viewing direction, the size projected onto the screen is calculated. This size is the size of the node. (i.e., the leaf nodes of the first target detail level) hierarchical granularity (i.e., boundary granularity value), the selection logic is as follows: if the node The boundary satisfies the grain size condition (smaller than a given grain size). (i.e., the preset granularity value), but its parent node (i.e., the leaf node of the second target detail level) does not satisfy it, then the node will be... Selected as the target point, optionally in this embodiment... The value is 1.

[0155] The selected Gaussian layer is rendered through the 3DGS rendering pipeline to obtain the final modeling result.

[0156] In summary, the technical approach of the scene rendering method provided in the embodiments of the present invention is as follows: Figure 3 As shown, a large-scale scene reconstruction technology based on 3DGS combined with scene segmentation and LOD strategy is presented. 3DGS is used to model large-scale scenes, and the main steps include: geographic scene data preprocessing, uniform scene segmentation, and view rendering. Specifically, geographic scene data preprocessing includes: inputting UAV multi-view images, downsampling the images, obtaining depth using the DPT algorithm, performing feature extraction, feature matching, and camera pose calculation; uniform scene segmentation includes: reconstructing sparse point clouds, filtering detached points in the sparse point clouds, constructing an initial mesh, iteratively segmenting the mesh, and merging adjacent meshes block by block; view rendering includes: loss optimization based on RGB and depth maps, constructing a hierarchy tree, constructing a hierarchy selection scheme, and rendering.

[0157] Specifically, to demonstrate the effectiveness of this method, comparative experiments were conducted with multiple methods on the Building and Rubble scenes in the Mill19 dataset and the Campus, Residence, and SciArt scenes in the Urban 3D Scenic dataset. Quantitative numerical results are shown in Table 1.

[0158] Table 1: Comparison of Quantitative Evaluation Results of Public Data Test Set Indicators for Complete Scenarios

[0159]

[0160] When segmenting a scene, this invention needs to consider the three-dimensional distribution characteristics of the sparse point cloud in order to construct a uniform scene segmentation method that integrates the distribution characteristics of the point cloud and the camera. It also needs to consider how to make full use of the features of the three-dimensional Gaussian itself to construct a hierarchical structure. This invention proposes a mesh-based scene segmentation method based on the three-dimensional distribution characteristics of the point cloud. This invention considers the distribution in the z-axis direction more, and it does not require too many parameter adjustments, resulting in a higher degree of automation and universality.

[0161] This invention proposes an automatic scene segmentation method based on meshing and the 3D distribution characteristics of sparse point clouds. This method provides more reasonable scene segmentation and has stronger universality. Furthermore, a Level of Detail (LOD) strategy is incorporated into the rendering process, constructing a Gaussian hierarchical tree and a hierarchy selection scheme. This reduces the computational resources required for large-scale scene rendering and improves the view rendering quality.

[0162] This invention can be applied to the iterative upgrade of existing data center inspection robots, aiming to solve the problems of rough environmental modeling and insufficient data processing efficiency of existing inspection robots. It can provide robots with high-precision scene modeling and hierarchical data processing capabilities, helping them to accurately perceive complex data center environments and providing support for efficient operation and maintenance of data centers.

[0163] like Figure 4 As shown, this embodiment of the invention also provides a scene rendering apparatus, including:

[0164] The first processing module 401 is used to obtain the target sparse point cloud of the scene based on multiple view images of the scene.

[0165] The second processing module 402 is used to segment the scene according to the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in different target scene blocks.

[0166] The third processing module 403 is used to convert the target sparse point cloud in the target scene block into a Gaussian point cloud using a three-dimensional Gaussian splash 3DGS model for each target scene block.

[0167] The fourth processing module 404 is used to render the scene based on the Gaussian point cloud of the target level of detail (LOD) in the Gaussian level of detail tree constructed from the Gaussian point cloud, wherein the Gaussian point cloud includes Gaussian point clouds with multiple LODs.

[0168] Optionally, the first processing module 401 includes:

[0169] The first processing unit is used to obtain a sparse point cloud of the scene based on multiple view images of the scene.

[0170] The second processing unit is used to filter the sparse point cloud in the first direction based on the average value and standard deviation of the sparse point cloud in the first direction, and obtain the first filtering result of the sparse point cloud.

[0171] The third processing unit is used to filter the sparse point cloud in the second direction based on the average value and standard deviation of the sparse point cloud in the second direction, and obtain the second filtering result of the sparse point cloud.

[0172] The fourth processing unit is used to filter the sparse point cloud in the third direction according to the preset distribution range in the third direction, so as to obtain the third filtering result of the sparse point cloud.

[0173] The fifth processing unit is used to obtain the target sparse point cloud based on the first filtering result, the second filtering result, the third filtering result, and the sparse point cloud.

[0174] Wherein, the first direction is perpendicular to the second direction, and the third direction is perpendicular to both the first direction and the second direction.

[0175] Optionally, the second processing module 402 includes:

[0176] The sixth processing unit is used to map the target sparse point cloud to a two-dimensional space to obtain an initial two-dimensional point cloud;

[0177] The seventh processing unit is used to segment the scene according to the preset number of segments and the spatial range of the initial two-dimensional point cloud to obtain an initial scene block;

[0178] The eighth processing unit is used to obtain a first scene block and a first point cloud in the first scene block based on a preset overlap ratio between adjacent initial scene blocks and the initial scene block and the initial two-dimensional point cloud.

[0179] The ninth processing unit is used to segment the first scene block according to a preset first point count threshold and the first point cloud in the first scene block to obtain a second scene block and a second point cloud in the second scene block.

[0180] The tenth processing unit is used to merge adjacent second scene blocks according to a preset second point count threshold, a preset third point count threshold, and the second point cloud in the second scene block, to obtain the target scene block and the target sparse point cloud in the target scene block.

[0181] Wherein, the second point threshold is less than the third point threshold.

[0182] Optionally, the ninth processing unit is specifically used for:

[0183] If the number of points in the first point cloud in the first target scene block is greater than or equal to the first point number threshold, the segmentation direction of the first target scene block is determined based on the number of points in the first point cloud mapped to the first direction and the number of points in the first target scene block mapped to the second direction, wherein the first direction is perpendicular to the second direction.

[0184] Based on the position corresponding to the first median, the first target scene block is segmented to obtain a second target scene block and a first point cloud in the second target scene block, wherein the first median is the median of the number of points in the first target scene block mapped to the first point cloud in the segmentation direction;

[0185] If the first condition is met, the second target scene block is used as the second scene block, and the first point cloud in the second target scene block is used as the second point cloud in the second scene block;

[0186] If the first condition is not met, the second target scene block is used as the first target scene block, and the step of determining the segmentation direction of the first target scene block based on the number of points in the first target scene block mapped to the first point cloud in the first direction and the number of points in the first target scene block mapped to the first point cloud in the second direction is returned until the first condition is met.

[0187] The first condition includes at least one of the following:

[0188] The first target scene block is segmented more times than a preset number;

[0189] The number of points in the first point cloud in the second target scene block is less than the first point count threshold;

[0190] Wherein, the first target scene block is any one of the scene blocks in the first scene block.

[0191] Optionally, the tenth processing unit is specifically used for:

[0192] If the number of points in the second point cloud in the third target scene block is less than the second point count threshold, and the second condition is met, the third target scene block and the fourth target scene block are merged to obtain a third scene block and a third point cloud in the third scene block. The third target scene block is any scene block in the second scene block, and the fourth target scene block is a scene block in the second scene block that is adjacent to the third target scene block. The second condition includes that the number of points in the merged second point cloud in the third target scene block and the second point cloud in the fourth target scene block is less than the third point count threshold.

[0193] Obtain the view image of the scene corresponding to the third point cloud in the third scene block;

[0194] The camera corresponding to the view image is determined based on the Laplacian operator value of the view image;

[0195] When the number of cameras is greater than a preset number, the third scene block corresponding to the view image is taken as the target scene block, and the third point cloud in the third scene block corresponding to the view image is taken as the target sparse point cloud in the target scene block.

[0196] Optionally, the device further includes:

[0197] The fifth processing module is used to obtain the Gaussian bounding box of the Gaussian point cloud based on the spatial range of the Gaussian point cloud.

[0198] The sixth processing module is used to split the Gaussian point cloud into the leaf nodes of the Gaussian detail tree within the Gaussian bounding box using the bisection median segmentation method, wherein each leaf node includes one Gaussian point cloud.

[0199] The seventh processing module is used to merge the Gaussian point cloud in each leaf node of the Gaussian detail tree into the Gaussian point cloud in the leaf node of the previous layer in a bottom-up order, starting from the last leaf node of the Gaussian detail tree, to obtain the merged Gaussian point cloud corresponding to each leaf node.

[0200] The eighth processing module is used to take the fused Gaussian point cloud corresponding to the leaf node of the first target detail level as the Gaussian point cloud of the target detail level when the boundary granularity value of the fused Gaussian point cloud corresponding to the leaf node of the first target detail level is less than the preset granularity value, and the boundary granularity value of the fused Gaussian point cloud corresponding to the leaf node of the second target detail level is greater than or equal to the preset granularity value. In this case, the leaf node of the second target detail level is the parent node of the leaf node of the first target detail level in the Gaussian detail level tree.

[0201] It should be noted that the scene rendering device provided in the embodiments of the present invention is a device capable of executing the above-described scene rendering method. Therefore, all embodiments of the above-described scene rendering method are applicable to this device and can achieve the same or similar technical effects.

[0202] like Figure 5 As shown, this embodiment of the invention also provides a scene rendering device, including: a processor 501; and a memory 503 connected to the processor 501 via a bus interface 502. The memory 503 is used to store programs and data used by the processor 501 when performing operations, and the processor 501 calls and executes the programs and data stored in the memory 503.

[0203] The transceiver 504 is connected to the bus interface 502 and is used to receive and send data under the control of the processor 501. Specifically, the processor 501 is used to read the program in the memory 503 and to execute the following processes:

[0204] Obtain the target sparse point cloud of the scene based on multiple view images of the scene;

[0205] The scene is segmented based on the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in different target scene blocks;

[0206] For each target scene block, the sparse point cloud of the target scene block is converted into a Gaussian point cloud using a 3D Gaussian splash 3DGS model.

[0207] The scene is rendered based on the Gaussian point cloud in the Gaussian level of detail tree constructed from the Gaussian point cloud, wherein the Gaussian point cloud includes Gaussian point clouds with multiple levels of detail.

[0208] Optionally, the processor 501 is configured to:

[0209] The sparse point cloud of the scene is obtained from multiple view images of the scene;

[0210] Based on the average value and standard deviation of the sparse point cloud in the first direction, the sparse point cloud is filtered in the first direction to obtain the first filtering result of the sparse point cloud.

[0211] Based on the average value and standard deviation of the sparse point cloud in the second direction, the sparse point cloud is filtered in the second direction to obtain the second filtering result of the sparse point cloud.

[0212] Based on the preset distribution range of the third direction, the sparse point cloud is filtered in the third direction to obtain the third filtering result of the sparse point cloud.

[0213] The target sparse point cloud is obtained based on the first screening result, the second screening result, the third screening result, and the sparse point cloud.

[0214] Wherein, the first direction is perpendicular to the second direction, and the third direction is perpendicular to both the first direction and the second direction.

[0215] Optionally, the processor 501 is configured to:

[0216] The target sparse point cloud is mapped to a two-dimensional space to obtain an initial two-dimensional point cloud;

[0217] The scene is segmented according to the preset number of segments and the spatial range of the initial two-dimensional point cloud to obtain an initial scene block;

[0218] Based on the preset overlap ratio between adjacent initial scene blocks and the initial scene blocks and the initial two-dimensional point cloud, obtain the first scene block and the first point cloud in the first scene block;

[0219] Based on a preset first point count threshold and the first point cloud in the first scene block, the first scene block is segmented to obtain a second scene block and a second point cloud in the second scene block;

[0220] Based on the preset second point count threshold, the preset third point count threshold, and the second point cloud in the second scene block, adjacent second scene blocks are merged to obtain the target scene block and the target sparse point cloud in the target scene block.

[0221] Wherein, the second point threshold is less than the third point threshold.

[0222] Optionally, the processor 501 is specifically used for:

[0223] If the number of points in the first point cloud in the first target scene block is greater than or equal to the first point number threshold, the segmentation direction of the first target scene block is determined based on the number of points in the first point cloud mapped to the first direction and the number of points in the first target scene block mapped to the second direction, wherein the first direction is perpendicular to the second direction.

[0224] Based on the position corresponding to the first median, the first target scene block is segmented to obtain a second target scene block and a first point cloud in the second target scene block, wherein the first median is the median of the number of points in the first target scene block mapped to the first point cloud in the segmentation direction;

[0225] If the first condition is met, the second target scene block is used as the second scene block, and the first point cloud in the second target scene block is used as the second point cloud in the second scene block;

[0226] If the first condition is not met, the second target scene block is used as the first target scene block, and the step of determining the segmentation direction of the first target scene block based on the number of points in the first target scene block mapped to the first point cloud in the first direction and the number of points in the first target scene block mapped to the first point cloud in the second direction is returned until the first condition is met.

[0227] The first condition includes at least one of the following:

[0228] The first target scene block is segmented more times than a preset number;

[0229] The number of points in the first point cloud in the second target scene block is less than the first point count threshold;

[0230] Wherein, the first target scene block is any one of the scene blocks in the first scene block.

[0231] Optionally, the processor 501 is specifically used for:

[0232] If the number of points in the second point cloud in the third target scene block is less than the second point count threshold, and the second condition is met, the third target scene block and the fourth target scene block are merged to obtain a third scene block and a third point cloud in the third scene block. The third target scene block is any scene block in the second scene block, and the fourth target scene block is a scene block in the second scene block that is adjacent to the third target scene block. The second condition includes that the number of points in the merged second point cloud in the third target scene block and the second point cloud in the fourth target scene block is less than the third point count threshold.

[0233] Obtain the view image of the scene corresponding to the third point cloud in the third scene block;

[0234] The camera corresponding to the view image is determined based on the Laplacian operator value of the view image;

[0235] When the number of cameras is greater than a preset number, the third scene block corresponding to the view image is taken as the target scene block, and the third point cloud in the third scene block corresponding to the view image is taken as the target sparse point cloud in the target scene block.

[0236] Optionally, the processor 501 is further configured to:

[0237] Based on the spatial extent of the Gaussian point cloud, obtain the Gaussian bounding box of the Gaussian point cloud;

[0238] Using the bisection median segmentation method, the Gaussian point cloud is split into leaf nodes of the Gaussian detail tree within the Gaussian bounding box, and each leaf node includes one Gaussian point cloud.

[0239] Starting from the last leaf node of the Gaussian detail tree, the Gaussian point cloud in each leaf node is sequentially fused into the Gaussian point cloud in the leaf node above the previous layer, in order from bottom to top, to obtain the fused Gaussian point cloud corresponding to each leaf node.

[0240] If the boundary granularity value of the fused Gaussian point cloud corresponding to the leaf node of the first target detail level is less than the preset granularity value, and the boundary granularity value of the fused Gaussian point cloud corresponding to the leaf node of the second target detail level is greater than or equal to the preset granularity value, then the fused Gaussian point cloud corresponding to the leaf node of the first target detail level is taken as the Gaussian point cloud of the target detail level. In the Gaussian detail level tree, the leaf node of the second target detail level is the parent node of the leaf node of the first target detail level.

[0241] Among them, Figure 5 In this context, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 501) and memory (memory 503). The bus architecture may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides a user interface 505. A transceiver 504 may be multiple elements, including transmitters and receivers, providing units for communicating with various other devices over a transmission medium. Processor 501 is responsible for managing the bus architecture and general processing, and memory 503 may store data used by processor 501 during operation.

[0242] In addition, specific embodiments of the present invention also provide a readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps in the scene rendering method as described above.

[0243] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0244] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically comprised separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or in the form of hardware plus software functional units.

[0245] The integrated unit implemented as a software functional unit described above can be stored in a computer-readable storage medium. This software functional unit, stored in a storage medium, includes several instructions that cause a computer device (which may be a personal computer, server, or network device, etc.) to execute partial steps of the resource selection method described in the various embodiments of the present invention, or to execute partial steps of the information transmission method described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0246] A specific embodiment of the present invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described functionality. Figure 2 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0247] The above describes the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also within the scope of protection of the present invention.

Claims

1. A method of scene rendering, characterized by, The method comprises the following steps: acquiring a target sparse point cloud of a scene according to multiple view images of the scene; segmenting the scene according to the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in different target scene blocks; for each target scene block, converting the target sparse point cloud in the target scene block into a Gaussian point cloud by using a three-dimensional Gaussian splash (3DGS) model; rendering the scene according to Gaussian point clouds of target levels of detail (LODs) in a Gaussian level of detail tree constructed by the Gaussian point clouds, wherein the Gaussian point clouds comprise Gaussian point clouds of multiple LODs.

2. The method of claim 1, wherein, The method of acquiring the target sparse point cloud of the scene according to the multiple view images of the scene comprises the following steps: acquiring a sparse point cloud of the scene according to the multiple view images of the scene; screening the sparse point cloud in a first direction according to the average value and the standard deviation of the sparse point cloud in the first direction to obtain a first screening result of the sparse point cloud; screening the sparse point cloud in a second direction according to the average value and the standard deviation of the sparse point cloud in the second direction to obtain a second screening result of the sparse point cloud; screening the sparse point cloud in a third direction according to a preset distribution range in the third direction to obtain a third screening result of the sparse point cloud; obtaining the target sparse point cloud according to the first screening result, the second screening result, the third screening result and the sparse point cloud; wherein the first direction is perpendicular to the second direction, and the third direction is perpendicular to the first direction and the second direction.

3. The method of claim 1, wherein, The method of segmenting the scene according to the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in different target scene blocks comprises the following steps: mapping the target sparse point cloud to a two-dimensional space to obtain an initial two-dimensional point cloud; segmenting the scene according to a preset segmentation number and a spatial range of the initial two-dimensional point cloud to obtain initial scene blocks; obtaining a first scene block and a first point cloud in the first scene block according to a preset overlap ratio between adjacent initial scene blocks and the initial scene blocks and the initial two-dimensional point cloud; segmenting the first scene block according to a preset first point threshold and the first point cloud in the first scene block to obtain a second scene block and a second point cloud in the second scene block; merging adjacent second scene blocks according to a preset second point threshold, a preset third point threshold and the second point cloud in the second scene block to obtain the target scene blocks and target sparse point clouds in the target scene blocks; wherein the second point threshold is smaller than the third point threshold.

4. The method of claim 3, wherein, The method of segmenting the first scene block according to the preset first point threshold and the first point cloud in the first scene block to obtain a second scene block and a second point cloud in the second scene block comprises the following steps: In a case where the number of points of the first point cloud in the first target scene block is greater than or equal to the first point number threshold, a segmentation direction of the first target scene block is determined according to the number of points of the first point cloud in the first target scene block mapped to a first direction and the number of points of the first point cloud in the first target scene block mapped to a second direction, wherein the first direction is perpendicular to the second direction; The first target scene block is segmented according to a position corresponding to a first median, to obtain a second target scene block and a first point cloud in the second target scene block, wherein the first median is a median of the number of points of the first point cloud in the first target scene block mapped to the segmentation direction; In a case where a first condition is met, the second target scene block is taken as the second scene block, and the first point cloud in the second target scene block is taken as the second point cloud in the second scene block; In a case where the first condition is not met, the second target scene block is taken as the first target scene block, and the step of determining the segmentation direction of the first target scene block according to the number of points of the first point cloud in the first target scene block mapped to the first direction and the number of points of the first point cloud in the first target scene block mapped to the second direction is returned until the first condition is met; The first condition includes at least one of the following: The number of times of segmentation of the first target scene block is greater than a preset number of times; The number of points of the first point cloud in the second target scene block is less than the first point number threshold; The first target scene block is any one of the first scene blocks.

5. The method of claim 3, wherein, The merging of adjacent second scene blocks according to the preset second point number threshold, the preset third point number threshold and the second point cloud in the second scene block to obtain the target scene block and a target sparse point cloud in the target scene block includes: In a case where the number of points of the second point cloud in a third target scene block is less than the second point number threshold and a second condition is met, the third target scene block and a fourth target scene block are merged to obtain a third scene block and a third point cloud in the third scene block, wherein the third target scene block is any one of the second scene blocks, the fourth target scene block is a scene block adjacent to the third target scene block in the second scene blocks, and the second condition includes that the number of points of the second point cloud in the third target scene block and the second point cloud in the fourth target scene block after merging is less than the third point number threshold; A view image of the scene corresponding to the third point cloud in the third scene block is acquired; A camera corresponding to the view image is determined according to a Laplacian value of the view image; In a case where the number of cameras is greater than a preset number, the third scene block corresponding to the view image is taken as the target scene block, and the third point cloud in the third scene block corresponding to the view image is taken as a target sparse point cloud in the target scene block.

6. The method of claim 1, wherein, Before rendering the scene according to the target level of detail (LOD) of the Gaussian point cloud in the Gaussian level of detail tree constructed from the Gaussian point cloud, the method further comprises: obtaining a Gaussian bounding box of the Gaussian point cloud according to a spatial range of the Gaussian point cloud; splitting the Gaussian point cloud into leaf nodes of the Gaussian level of detail tree in the Gaussian bounding box by using a binary median split method, each leaf node comprising a Gaussian point cloud; fusing the Gaussian point cloud in each leaf node to the Gaussian point cloud in the upper leaf node in turn according to a bottom-up order to obtain a fused Gaussian point cloud corresponding to each leaf node; in a case where a boundary granularity value of the fused Gaussian point cloud corresponding to a leaf node of a first target level of detail is less than a preset granularity value, and a boundary granularity value of the fused Gaussian point cloud corresponding to a leaf node of a second target level of detail is greater than or equal to the preset granularity value, wherein the leaf node of the second target level of detail is a parent node of the leaf node of the first target level of detail, taking the fused Gaussian point cloud corresponding to the leaf node of the first target level of detail as the target level of detail of the Gaussian point cloud.

7. A scene rendering apparatus, characterized by comprising: comprise: a first processing module configured to obtain a target sparse point cloud of a scene according to multiple view images of the scene; a second processing module configured to segment the scene according to the target sparse point cloud to obtain different target scene blocks and target sparse point clouds in the different target scene blocks; a third processing module configured to convert the target sparse point cloud in each target scene block into a Gaussian point cloud by using a three-dimensional Gaussian splash (3DGS) model for each target scene block; a fourth processing module configured to render the scene according to a target level of detail (LOD) of the Gaussian point cloud in a Gaussian level of detail tree constructed from the Gaussian point cloud, wherein the Gaussian point cloud comprises Gaussian point clouds of multiple LODs.

8. A scene rendering device, characterized by comprise: a processor, a memory, and a program stored on the memory and executable on the processor, the program being executed by the processor to implement the scene rendering method according to any one of claims 1 to 6.

9. A readable storage medium, characterized by, a readable storage medium having a program stored thereon, the program being executed by a processor to implement the scene rendering method according to any one of claims 1 to 6.

10. A computer program product, characterised in that, computer instructions, the computer instructions being executed by a processor to implement the scene rendering method according to any one of claims 1 to 6.

Images