Large-scale scene multi-level-of-detail cloud rendering processing method and device based on 3DGS

Abstract

The invention provides a large-scale scene multi-level-of-detail cloud rendering processing method and device based on 3DGS, and relates to the technical field of three-dimensional modeling, and the method comprises the steps: dividing a target modeling scene into a plurality of sub-blocks; performing particle redundancy reconstruction and overlapping region marking on boundary regions between adjacent sub-blocks of each sub-block to obtain processed sub-blocks; performing multi-detail level division on each processing sub-block to generate a particle level set; determining a current visual area according to the user motion data, and scheduling a target hierarchy of a particle hierarchy set in the current visual area; the rendering tasks of all the processing sub-blocks of the target hierarchy are distributed to a plurality of rendering nodes to execute real-time rendering operation; and performing video stream coding on pictures rendered by each rendering node, decoding and displaying received video stream data, and performing particle level updating and re-rendering operation according to an interaction instruction. According to the invention, high-quality detail rendering can be realized for a model of a large scene.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional modeling, and in particular to a method and device for cloud rendering processing of large-scale scenes with multiple levels of detail based on 3DGS. Background Art

[0002] With the increasing application of 3D scene reconstruction and novel perspective synthesis technologies in fields such as augmented reality, virtual reality, autonomous driving, and 3D city generation, the construction of large-scale 3D models of urban scenes faces multiple challenges, including complex spatial structures and massive data volumes. Although existing reconstruction technologies have achieved initial breakthroughs in accuracy and expressiveness, practical deployments still struggle to simultaneously meet the dual requirements of high-quality rendering and efficient access services. Achieving low-latency, high-smoothness terminal access while maintaining visual accuracy has become a key bottleneck and research hotspot in current technological development.

[0003] As a novel explicit 3D representation method, 3D Gaussian splatter modeling constructs a 3D Gaussian point cloud from a set of Gaussian ellipsoids and efficiently rasterizes this point cloud into an image using a splatting algorithm, effectively improving rendering speed during new perspective synthesis. Compared to traditional discrete meshes or conventional point cloud models, 3D Gaussian splatter modeling significantly reduces the computational complexity of high-quality rendering while maintaining the accuracy of scene geometry and lighting details. Its structured representation also naturally supports a variety of downstream applications such as dynamic reconstruction, geometry editing, and physical simulation, demonstrating comprehensive performance advantages over traditional model structures.

[0004] As 3D Gaussian splatter modeling technology is widely adopted in applications such as digital twins and the metaverse, which require extremely high precision and real-time performance, the ultra-large-scale data structures and continuously updated rendering requirements involved pose severe challenges to the computing performance and storage capabilities of terminal devices. Existing rendering solutions mainly rely on the client to load and calculate high-precision 3D Gaussian splatter modeling models locally. Although this method can achieve good results in applications targeting small or local target areas, when it is expanded to large-scale scenarios such as city-level scenarios, it is difficult to achieve high-quality and efficient unified rendering due to the video memory capacity of the terminal device, the model optimization iteration time, and the ability to manage multi-region visual consistency. This has become a key factor restricting the current expansion of 3D Gaussian splatter modeling to macro-scale applications. Summary of the Invention

[0005] The present application provides a large-scale scene multi-detail level cloud rendering processing method and device based on 3DGS, which can achieve high-quality detail rendering of large-scale scene models.

[0006] In a first aspect of the present application, a method for processing large-scale scene multi-level-of-detail cloud rendering based on 3DGS is provided, the method comprising:

[0007] Performing image segmentation on the target modeling scene to divide the target modeling scene into multiple sub-blocks;

[0008] Performing spatial registration and boundary clipping processing on each of the sub-blocks, and performing particle redundancy reconstruction and overlapping area marking on the boundary areas between adjacent sub-blocks to obtain processed sub-blocks;

[0009] Performing multi-level-of-detail division on each of the processing sub-blocks to generate a particle level set including multiple particle retention ratios and geometric error thresholds;

[0010] Build a world partitioning mechanism on the server side, determine the current visible area based on the user motion data transmitted by the client, and schedule the target level of the particle level set within the current visible area;

[0011] Building a distributed rendering node cluster on the server side, and assigning rendering tasks of each processing sub-block of the target layer to multiple rendering nodes to perform real-time rendering operations;

[0012] The images rendered by each rendering node are encoded into video streams, and the received video stream data is decoded and displayed. At the same time, user interaction instructions are collected in real time and particle level updates and re-rendering operations are triggered, thereby achieving re-rendering of the target modeling scene.

[0013] Based on the above technical solution, preferably, performing spatial registration and boundary clipping processing on each sub-block, and performing particle redundancy reconstruction and overlapping area marking on the boundary areas between adjacent sub-blocks to obtain processed sub-blocks specifically includes:

[0014] Performing spatial registration on the 3DGS model according to the partition coordinates of each of the sub-blocks, and clipping the 3DGS model according to the partition boundary and the restricted height to eliminate distant point clouds of the 3DGS model;

[0015] determining a complex block in which the Gaussian unit density in the plurality of sub-blocks exceeds a preset standard;

[0016] Redundant adjustments are made to the boundaries of adjacent sub-blocks of the complex block and dependencies are marked, and overlapping areas of boundary Gaussian units are marked to ensure visual continuity during streaming loading.

[0017] Based on the above technical solution, preferably, performing multi-level detail division on each of the processing sub-blocks to generate a particle level set including multiple particle retention ratios and geometric error thresholds specifically includes:

[0018] Multiple levels of detail are divided according to the density and radius of the Gaussian distribution to generate a hierarchical particle level set, wherein the first level higher than the preset level among the multiple particle levels in the particle level set retains the complete particle simulation, and the second level lower than the preset level among the multiple particle levels in the particle level set is simplified or represented as a static billboard. The simplification rule is density and radius clustering simplification based on Gaussian distribution, Gaussian units are merged for sparse areas, and density sampling is retained for dense areas. The static billboard only retains position and rotation properties, and the simplification process retains the error threshold.

[0019] Based on the above technical solution, preferably, the world partition mechanism is constructed on the server side, and the current visible area is determined according to the user motion data transmitted by the client, and the target level of the particle level set in the current visible area is scheduled, which specifically includes:

[0020] Performing spatial index management on the plurality of processing sub-blocks based on the world partition mechanism of the Unreal Engine on the server side, and dividing the 3DGS model into a plurality of streaming partitions based on a user viewing distance parameter transmitted by the client;

[0021] Prioritizing the loading of the first level within the viewing cone based on the user's current viewing direction and field of view, and only loading the second level for areas outside the viewing cone, and determining the target level based on the first level and the second level;

[0022] When it is detected that any one of the sub-blocks is outside the current visible area, skipping data loading of the sub-block outside the current visible area through the layered detail pipeline;

[0023] When the client's bandwidth is lower than a preset network latency threshold, a forced degradation operation of the particle system's level of detail is triggered, and a particle instance merging strategy is adopted to merge particle units of the same material into a unified draw call to reduce real-time transmission overhead;

[0024] Based on the user's viewing angle distance change rate and historical behavior records, the second-level hierarchical data of adjacent sub-blocks is preloaded in the user movement trajectory prediction area, and fading transition particles are inserted during the level switching process to achieve visually smooth transition between levels through Alpha channel control.

[0025] Based on the above technical solution, preferably, the step of building a distributed rendering node cluster on the server side and allocating the rendering tasks of each processing sub-block of the target layer to multiple rendering nodes to perform real-time rendering operations specifically includes:

[0026] determining a rendering complexity of the sub-block based on a Gaussian unit density of each sub-block;

[0027] Monitor the rendering load status of each rendering node in real time, adjust the task allocation among the multiple rendering nodes in real time according to the rendering complexity, and preferentially migrate the sub-blocks with a rendering complexity higher than a preset complexity to the rendering nodes with an idle rendering load status;

[0028] During the rendering process of the rendering node, for the particles of the first level, volume fog effect and real-time ray tracing mechanism are enabled, volume lighting cache is generated according to radiosity attributes, and shadow casting effect is calculated;

[0029] For the second level of particles, we use pre-baked lightmaps combined with a simplified signed distance field model and distance field ambient occlusion to approximate global illumination.

[0030] The rendered images generated by each rendering node are encoded in H.264 and transmitted through the WebRTC protocol for low-latency pixel streaming. At the same time, the server dynamically matches the bit rate and resolution based on the current particle level and the type of the client, and performs particle level degradation and bit rate reduction processing when it detects that network fluctuations exceed a set threshold.

[0031] Based on the above technical solution, preferably, encoding the images rendered by each rendering node into a video stream, decoding and displaying the received video stream data, and simultaneously collecting user interaction instructions in real time and triggering particle level update and re-rendering operations specifically include:

[0032] A multi-user rendering instance management mechanism is established on the server side. Based on the dedicated server architecture of Unreal Engine, a unique instance identifier is generated for each user request, and an independent rendering process is allocated according to the instance identifier to isolate particle detail level loading and computing resources.

[0033] When the client initiates access, the corresponding video stream content is received by binding the instance identifier and decoding and presenting the target modeling scene;

[0034] The user's terminal parameters and interactive inputs, including perspective switching, zoom instructions, and control commands, are collected based on the socket protocol, and the collected results are transmitted back to the server in real time to trigger the dynamic switching of the particle detail level of the current visible area and the re-rendering operation of the target processing sub-block.

[0035] On the basis of the above technical solution, preferably, the target modeling scene is cut into pieces to divide the target modeling scene into a plurality of sub-blocks, specifically including:

[0036] According to the regional coordinates set for the target modeling scene, the scene is cut according to the two-dimensional geographical relationship of the world coordinates to form a continuous standard map and introduce the basic terrain and building height data in different areas as supervision to divide the target modeling scene into multiple sub-blocks.

[0037] In a second aspect of the present application, a large-scale scene multi-level of detail cloud rendering processing device based on 3DGS is provided, wherein the device is used to execute any one of the above-mentioned large-scale scene multi-level of detail cloud rendering processing methods based on 3DGS. The device includes an acquisition module, a processing module, and an output module, wherein:

[0038] The acquisition module is used to cut the target modeling scene into multiple sub-blocks;

[0039] The processing module is configured to perform spatial registration and boundary clipping processing on each of the sub-blocks, and perform particle redundancy reconstruction and overlapping area marking on the boundary areas between adjacent sub-blocks to obtain processed sub-blocks;

[0040] The processing module is configured to perform multi-level detail division on each of the processing sub-blocks to generate a particle level set including multiple particle retention ratios and geometric error thresholds;

[0041] The processing module is used to build a world partition mechanism on the server side, determine the current visible area based on the user motion data transmitted by the client, and schedule the target level of the particle level set within the current visible area;

[0042] The processing module is used to build a distributed rendering node cluster on the server side, and distribute the rendering tasks of each processing sub-block of the target layer to multiple rendering nodes to perform real-time rendering operations;

[0043] The output module is used to encode the images rendered by each rendering node into a video stream, decode and display the received video stream data, and at the same time collect user interaction instructions in real time and trigger particle level updates and re-rendering operations, thereby achieving re-rendering of the target modeling scene.

[0044] Based on the above technical solution, preferably, the processing module is used to perform spatial registration on the 3DGS model according to the partition coordinates of each of the sub-blocks, and to crop the 3DGS model according to the partition boundary and the restricted height to eliminate the distant point cloud of the 3DGS model;

[0045] The processing module is used to determine a complex block in which the Gaussian unit density in the plurality of sub-blocks exceeds a preset standard;

[0046] The processing module is used to perform redundancy adjustment on the boundaries of adjacent sub-blocks of the complex block and mark the dependency relationship, and mark the overlapping area of ​​the boundary Gaussian units to ensure visual continuity during streaming loading.

[0047] On the basis of the above technical solution, preferably, the output module is used to perform multi-detail level division according to the density and radius of the Gaussian distribution to generate a hierarchical particle level set, wherein the first level higher than the preset level among the multiple particle levels in the particle level set retains the complete particle simulation, and the second level lower than the preset level among the multiple particle levels in the particle level set is simplified or represented as a static billboard, and the simplification rule is density and radius clustering simplification based on Gaussian distribution, Gaussian units are merged for sparse areas, and density sampling is retained for dense areas, the static billboard only retains position and rotation properties, and the simplification process retains the error threshold.

[0048] Based on the above technical solution, preferably, the acquisition module is used to perform spatial index management on the multiple processing sub-blocks based on the world partition mechanism of the Unreal Engine on the server side, and divide the 3DGS model into multiple streaming partitions based on the user perspective distance parameter transmitted by the client;

[0049] The processing module is configured to preferentially load the first level within the visual cone according to the user's current viewing direction and field of view, load only the second level for areas outside the visual cone, and determine the target level based on the first level and the second level;

[0050] The processing module is configured to, when detecting that any of the sub-blocks is outside the current visible area, skip loading data of the sub-blocks outside the current visible area through the layered detail pipeline;

[0051] The processing module is configured to trigger a forced degradation of the particle system level of detail when the bandwidth of the client is lower than a preset network latency threshold, and adopt a particle instance merging strategy to merge particle units of the same material into a unified draw call to reduce real-time transmission overhead;

[0052] The processing module is used to preload the second-level hierarchical data of adjacent sub-blocks in the user movement trajectory prediction area based on the user perspective distance change rate and historical behavior records, and insert fade-out transition particles during the level switching process to achieve visually smooth transition between levels through alpha channel control.

[0053] On the basis of the above technical solution, preferably, the processing module is used to determine the rendering complexity of the sub-block based on the Gaussian unit density of each sub-block;

[0054] The acquisition module is configured to monitor the rendering load status of each rendering node in real time, adjust task allocation among the plurality of rendering nodes in real time according to the rendering complexity, and preferentially migrate sub-blocks with a rendering complexity higher than a preset complexity to rendering nodes with an idle rendering load status;

[0055] The processing module is configured to enable volumetric fog effect and real-time ray tracing mechanism for particles of the first level during the rendering process of the rendering node, generate a volumetric lighting cache based on radiosity attributes, and calculate shadow casting effects;

[0056] The processing module is configured to approximately simulate global illumination for particles of the second level by using a pre-baked light map combined with a simplified signed distance field model and a distance field ambient occlusion algorithm;

[0057] The processing module is used to encode the rendered images generated by each rendering node using H.264 and transmit low-latency pixel streams through the WebRTC protocol. At the same time, the server dynamically matches the bit rate and resolution based on the current particle level and the type of the client, and performs particle level degradation and bit rate reduction processing when it detects that network fluctuations exceed a set threshold.

[0058] Based on the above technical solution, preferably, the processing module is used to build a multi-user rendering instance management mechanism on the server side, generate a unique instance identifier for each user request based on the dedicated server architecture of Unreal Engine, and allocate an independent rendering process according to the instance identifier to isolate particle detail level loading and computing resources;

[0059] The processing module is configured to receive the corresponding video stream content by binding the instance identifier and decode and present the target modeling scene when the client initiates access;

[0060] The processing module is used to collect user terminal parameters and interactive inputs based on the socket protocol, including perspective switching, zoom instructions and control commands, and transmit the collected results back to the server in real time to trigger the dynamic switching of the particle detail level of the current visible area and the re-rendering operation of the target processing sub-block.

[0061] On the basis of the above technical solution, preferably, the processing module is used to perform scene cutting according to the regional coordinates set for the target modeling scene and the two-dimensional geographical relationship of the world coordinates, to form a continuous standard map and introduce the basic terrain and building height data in different areas as supervision, and divide the target modeling scene into multiple sub-blocks.

[0062] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs any of the methods described above.

[0063] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed, any one of the methods described above is executed.

[0064] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0065] 1. This application performs image segmentation and spatial registration of large scenes based on the world coordinate system, combines building height supervision to achieve precise cropping and redundant reconstruction, constructs a multi-detail level particle set that supports multi-particle retention ratio and error threshold control, and relies on the Unreal Engine's world partitioning mechanism and distributed rendering node cluster to achieve dynamic scheduling and parallel rendering of particle levels in the visible area. At the same time, it adopts video stream encoding transmission and real-time interactive feedback mechanism to achieve high-quality, low-latency cloud rendering and terminal decoding display of complex city-level models while ensuring the accuracy of particle detail expression, thereby effectively supporting the fine visualization needs of large-scale three-dimensional Gaussian splash modeling scenes.

[0066] 2. By performing spatial registration and boundary clipping operations on each sub-block, combined with redundant particle reconstruction and overlapping area marking at complex region boundaries, distant artifacts and abnormal high-altitude point clouds can be effectively removed, improving the neatness of the model structure. At the same time, during the splicing process of multiple sub-blocks, the dependency relationship and overlapping particle retention strategy are used to achieve visual continuity of the particle system under streaming loading, avoiding geometric breaks or detail mutations caused by boundary switching.

[0067] 3. By dividing particles into multiple levels of detail based on Gaussian distribution density and radius characteristics, a particle hierarchy set that supports adaptive precision scheduling is established, allowing for flexible loading of appropriate particle retention levels in different rendering scenarios. High-level particles achieve high-fidelity scene reproduction, while low-level particles are clustered and merged using billboard representation to reduce computational and transmission overhead, effectively striking a balance between visual quality and system load, and supporting dynamic resolution control from high-precision near-shots to simplified far-shots.

[0068] 4. By building a world partitioning mechanism on the server and dynamically determining the visible area and particle target level based on the perspective parameters transmitted by the client, on-demand scheduling of scene space resources and particle loading control are achieved. Occlusion culling and network adaptation strategies are combined to significantly reduce invalid data transmission and redundant rendering tasks, improving the system's real-time responsiveness. At the same time, trajectory prediction and fade-out transition mechanisms are introduced to effectively avoid sudden changes in the level switching process, improving rendering consistency and natural interaction when the user's perspective moves.

[0069] 5. By building a distributed rendering node cluster on the server side and dynamically adjusting task allocation based on the rendering complexity of sub-blocks, elastic migration of high-load sub-blocks and node load balancing are achieved, fully unleashing the cluster's parallel rendering capabilities. High-level particles enable real-time lighting simulation to ensure image realism, while low-level particles use lighting approximation technology to compress computing resources. Combining coding optimization and network adaptation strategies, stable video streaming transmission and high-frame-rate, low-latency rendering display are achieved for highly complex scenes under multi-terminal conditions.

[0070] 6. Through the instance management mechanism, an independent rendering process is allocated to each client request on the server side, realizing isolated control of particle system loading, calculation and transmission logic, and improving system stability under concurrent access by multiple users; the client receives the video stream based on the instance identifier and synchronously collects interactive input, which can drive the switching of particle detail levels and regional re-rendering in real time, forming an input-feedback closed-loop mechanism, enhancing the terminal control sensitivity and the immediacy of scene response, and effectively supporting a smooth cross-platform interactive experience.

[0071] 7. By combining the world coordinate system and terrain and building height supervision information to perform standard map division and height limit control during the map cutting process, the three-dimensional Gaussian splash modeling model is accurately cut into multiple sub-blocks with clear structures and regular boundaries, facilitating subsequent spatial alignment, distributed scheduling, and particle-level layered processing; the unified map standard provides basic support for the controllable organization of large-scene particle systems and parallel loading on the server, realizing efficient city-level modeling data organization and distributed particle rendering scheduling capabilities for high-density scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flowchart of a large-scale scene multi-level of detail cloud rendering processing method based on 3DGS disclosed in an embodiment of the present application;

[0073] Figure 2 This is a schematic diagram of a resource dynamic scheduling optimization process for a large-scale scenario of a 3DGS disclosed in an embodiment of the present application;

[0074] Figure 3 This is a module diagram of a large-scale scene multi-level of detail cloud rendering processing device based on 3DGS disclosed in an embodiment of the present application;

[0075] Figure 4 This is a structural diagram of an electronic device disclosed in an embodiment of the present application.

[0076] Explanation of the reference numerals: 301, acquisition module; 302, processing module; 303, output module; 401, processor; 402, communication bus; 403, user interface; 404, network interface; 405, memory. DETAILED DESCRIPTION

[0077] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0078] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0079] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0080] The widespread application of 3D scene reconstruction and new perspective synthesis technologies in fields such as augmented reality, autonomous driving, and 3D city modeling has driven the demand for rendering of large-scale 3D city-level models, but faces the dual challenges of complex spatial structure and huge data size. As an emerging explicit representation method, 3D Gaussian splattering modeling significantly improves the efficiency of new perspective rendering through Gaussian point clouds and splatting algorithms, and has expansion capabilities such as dynamic reconstruction and physical simulation. It has obvious advantages over traditional models in expression accuracy and computational efficiency. However, its promotion in high-precision real-time scenarios such as digital twins and metaverses has exposed the high dependence of local rendering solutions on terminal computing power and storage. In particular, it is difficult to overcome video memory limitations, rendering consistency, and optimization efficiency bottlenecks in macro-scale environments, which have become key technical obstacles restricting its large-scale application.

[0081] This embodiment discloses a large-scale scene multi-level of detail cloud rendering processing method based on 3DGS, referring to Figure 1 , including the following steps S110-S160:

[0082] S110 , performing image segmentation on the target modeling scene to divide the target modeling scene into multiple sub-blocks.

[0083] The embodiments of this application disclose a large-scale scene multi-level-of-detail cloud rendering processing method based on 3DGS, which is applied to a server. The server includes but is not limited to electronic devices such as mobile phones, tablets, wearable devices, and PCs (personal computers), and can also be a background server running a large-scale scene multi-level-of-detail cloud rendering processing method based on 3DGS. The server can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0084] In one possible implementation, the target modeling scene is cut into multiple sub-blocks, specifically including: according to the regional coordinates set for the target modeling scene, the scene is cut according to the two-dimensional geographical relationship of the world coordinates to form a continuous standard map and introduce the basic terrain and building height data in different areas as supervision to divide the target modeling scene into multiple sub-blocks.

[0085] Specifically, when dividing the target modeling scene into multiple sub-blocks, the starting point for the division is first set based on the target modeling scene's regional coordinates in the Unreal Engine system. These regional coordinates serve as the spatial reference in the world coordinate system for subsequent two-dimensional map layout. Integrating the two-dimensional geographic relationships in the world coordinate system, a grid-based equal division method is used to uniformly divide the entire modeling area, forming multiple standard map sub-blocks with a side length of two kilometers. This map division operation adheres to the requirements of equidistant coordinate alignment and spatial continuity, ensuring that all sub-blocks have no overlap or omissions in the overall structure, forming a complete and controllable set of spatial partitions.

[0086] To enhance the regional adaptability and vertical clipping accuracy of map sheet cutting, supervisory information is introduced within each map subblock. This involves collecting basic terrain and building height data corresponding to that subblock, preferably using global elevation data with a 30-meter resolution as the terrain foundation. Building height information is obtained by combining it with remote sensing inversion or mapping data. For each subblock, the maximum value of all terrain and building heights within it is extracted. The larger of the two values ​​is used as the reference, and ten meters is added to the height limit of the subblock to set the three-dimensional clipping threshold for that map subblock.

[0087] The standard map sub-block set constructed based on the above-mentioned cutting plane and height limit constraints serves as the processing unit for the subsequent spatial alignment and boundary clipping of the 3D Gaussian splash modeling model, and also has complete clipping parameters such as world coordinate identification, horizontal boundary constraints and vertical truncation height. Under this clipping system, not only can the regular grid division of the urban-level target modeling scene be achieved in the horizontal direction, but it can also efficiently peel off multiple regions in the vertical direction in conjunction with the geographical height difference and building scale information, providing a spatial boundary foundation for the subsequent particle structure construction, partition LOD processing and streaming loading management of the 3D Gaussian splash modeling data. The entire map cutting process uses the Unreal Engine world coordinates and supervisory height data as the cutting basis to generate a standard, continuous and computable sub-block set to ensure that the modeling structure has engineering usability and real-time rendering compatibility in large-scale urban environments.

[0088] S120 , performing spatial registration and boundary clipping processing on each sub-block, and performing particle redundancy reconstruction and overlapping area marking on boundary areas between adjacent sub-blocks to obtain processed sub-blocks.

[0089] In one possible implementation, spatial registration and boundary clipping are performed within each subblock, and particle redundancy reconstruction and overlapping area marking are performed on the boundary areas between adjacent subblocks to obtain processed subblocks. Specifically, the following steps are performed: spatial registration is performed on the 3DGS model according to the partition coordinates of each subblock, and the 3DGS model is clipped according to the partition boundaries and restricted heights to eliminate distant point clouds of the 3DGS model; complex blocks in which the Gaussian unit density exceeds a preset standard in multiple subblocks are determined; redundant adjustments are made to the boundaries of adjacent subblocks of the complex block and dependencies are marked, and overlapping areas of the boundary Gaussian units are marked to ensure visual continuity during streaming loading.

[0090] Specifically, when performing spatial registration and boundary clipping within each sub-block, the original three-dimensional Gaussian splash model is first spatially registered under the Unreal Engine system based on the partition coordinates of each sub-block. This spatial registration operation is based on the two-dimensional map coordinates of the sub-block. By spatially positioning the three-dimensional Gaussian point cloud data in a unified world coordinate system, the three-dimensional Gaussian splash model from multiple data sources is logically aligned with the map grid, ensuring that the spatial position of each Gaussian particle unit is completely consistent with its sub-block. After the spatial registration is completed, the clipping operation is performed based on the boundary range and height limit corresponding to each sub-block. The clipping process is constrained by the two-dimensional bounding box of the sub-block and the vertical height limit threshold. The particle units located outside the boundary and above the height limit in the three-dimensional Gaussian point cloud are eliminated, especially the aerial suspension artifacts and high-altitude distant point clouds (such as the sky, drone scanning anomalies, etc.) are completely eliminated, laying a neat structural foundation for subsequent particle simplification and rendering continuity processing.

[0091] Subsequently, a particle density statistical analysis is performed on all sub-blocks that have completed spatial registration and boundary clipping, focusing on identifying areas where the Gaussian unit density exceeds a set threshold and is therefore determined to be complex blocks. This complex block identification operation is based on counting the number of three-dimensional Gaussian point clouds within a cubic meter of each sub-block boundary area. If the density exceeds the preset density standard, it is marked as a complex block. This operation can be efficiently implemented through GPU-accelerated point cloud voxelization and density clustering. The existence of complex blocks means that their boundary particles vary dramatically and are large in number. Directly cutting according to the map boundary may cause visual discontinuities or detail jumps during streaming loading.

[0092] Therefore, for these complex blocks, boundary redundancy processing and overlapping area marking are implemented. The specific method is to expand the boundary area between the complex block and its adjacent sub-blocks by a certain spatial range, for example, extending the buffer zone inward and outward by 50 meters. The Gaussian particle units at the boundary are copied within the buffer zone and synchronously retained in the adjacent sub-blocks, thereby forming a boundary particle redundancy structure. This particle redundancy structure adds a cross-block identifier to the particle metadata through a dependency marking mechanism, so that the server can automatically pull the overlapping area data according to the user's perspective coverage when executing particle system loading, and execute visual continuity guarantee logic. Through this reconstruction mechanism, it is ensured that during the splicing and loading process of multiple sub-blocks, even if there is switching or loading delay, there will be no breakage, jump or density drop in the user's field of view, thereby achieving visual seamless transition and dynamic splicing consistency of the three-dimensional Gaussian splash modeling scene.

[0093] S130 , performing multi-level detail division on each processing sub-block to generate a particle level set including multiple particle retention ratios and geometric error thresholds.

[0094] In one possible implementation, multiple levels of detail are divided into multiple processing sub-blocks to generate a particle level set including multiple particle retention ratios and geometric error thresholds, specifically including: performing multiple levels of detail division according to the density and radius of the Gaussian distribution to generate a hierarchical particle level set, wherein the first level higher than the preset level among the multiple particle levels of the particle level set retains the complete particle simulation, and the second level lower than the preset level among the multiple particle levels of the particle level set is simplified or represented as a static billboard, the simplification rule is density and radius clustering simplification based on Gaussian distribution, Gaussian units are merged for sparse areas, density sampling is retained for dense areas, the static billboard only retains position and rotation attributes, and the simplification process retains the error threshold.

[0095] Specifically, during the implementation of LOD partitioning for each processing sub-block to generate a particle hierarchy set, the three-dimensional Gaussian splatter modeling particle units in each processing sub-block are first used as input data. The LOD construction operation is then performed based on the Gaussian distribution density and Gaussian distribution radius of each particle unit as the decision dimension. This LOD construction process utilizes a density-radius joint clustering algorithm to spatially partition the particle units into multiple hierarchical clusters based on their local density distribution and scale properties. A particle hierarchy set is then constructed, with each particle level representing a rendering accuracy level that can be scheduled and loaded.

[0096] In the particle level set, a preset level is set as the accuracy dividing point. The particle level above the preset level is defined as the first level. The first level retains the original particle unit set and does not perform any aggregation or simplification operations to ensure the integrity of high-fidelity particle simulation and is used for the visual expression of close-range high-precision rendering areas. The particle level below the preset level is defined as the second level, which is simplified step by step according to the distance and GPU load conditions. Specifically, for sparse areas, that is, particle sets with low Gaussian distribution density per unit space, a Gaussian unit merging operation is performed, and a new aggregated Gaussian particle unit is generated using the average radius weighted fusion algorithm. For dense areas, that is, areas with high Gaussian distribution density, the density sampling interval is kept unchanged, and only edge particles are removed to avoid distortion of boundary morphology caused by over-simplification.

[0097] When a particle level meets the criteria for further simplification, the simplification strategy shifts from Gaussian aggregation to static representation, constructing billboard particle units using static patch representation. Billboard particle units no longer have a Gaussian distribution structure, retaining only their position and rotation properties without carrying volume information. Visual approximation is achieved through patch texture and orientation alignment techniques. During the simplification process, a geometric error threshold parameter is set for each particle level. This geometric error threshold is calculated by evaluating the difference in visual projection of the particle set before and after simplification in a given viewing direction, ensuring that the simplified model maintains a tolerable error within the primary viewing angle.

[0098] Finally, a complete particle level set is generated for each processing sub-block. Each particle level contains parameters such as the corresponding particle retention ratio, minimum particle density, simplified trigger conditions, and geometric error threshold. This particle level set will serve as the basic data structure for subsequent server-side dynamic loading scheduling and perspective adaptive rendering, realizing the multi-precision dynamic rendering capability of three-dimensional Gaussian splash modeling under different distances, different device performance, and different network bandwidth conditions.

[0099] S140 , building a world partition mechanism on the server side, determining the current visible area based on the user motion data transmitted by the client, and scheduling the target level of the particle level set within the current visible area.

[0100] In one possible implementation, a world partitioning mechanism is constructed on the server side, and the current visible area is determined according to the user motion data transmitted by the client, and the target level of the particle level set within the current visible area is scheduled. Specifically, the server side performs spatial index management on multiple processing sub-blocks based on the world partitioning mechanism of the Unreal Engine, and divides the 3DGS model into multiple streaming partitions based on the user viewing distance parameter transmitted by the client; the first level within the viewing cone is preferentially loaded according to the user's current viewing direction and field of view, and only the second level is loaded for the area outside the viewing cone, and the target level is determined according to the first and second levels; when any When a sub-block is outside the current visible area, the data loading of the sub-block outside the current visible area is skipped through the layered detail pipeline; when the client's bandwidth is lower than the preset network delay threshold, the forced degradation of the particle system detail level is triggered, and the particle instance merging strategy is used to merge the particle units of the same material into a unified drawing call to reduce the real-time transmission overhead; based on the user's viewing distance change rate and historical behavior records, the second-level layer data of the adjacent sub-blocks is preloaded in the user's movement trajectory prediction area, and fade-out transition particles are inserted during the layer switching process to achieve visually smooth transition between layers through Alpha channel control.

[0101] Specifically, during the implementation process of constructing a world partitioning mechanism on the server side and determining the current visible area and the target particle level for scheduling based on the user motion data transmitted by the client, the server first performs spatial index management for all processing sub-blocks based on the Unreal Engine system's world partitioning mechanism. This spatial index management is based on the sub-block coordinates generated by the map partitioning. Each sub-block in the three-dimensional Gaussian splash model is registered as an independent data unit in the partition structure, and a region mapping table consistent with the Unreal Engine World Partition system is established. After receiving the user perspective distance parameter uploaded in real time by the client, the server divides the three-dimensional Gaussian splash model into multiple streaming partitions based on this distance parameter and the spatial index structure. Each streaming partition is associated with one or more processing sub-blocks, forming the basic spatial unit for particle scheduling.

[0102] Next, the viewing cone range is constructed based on the user's viewing direction and field of view parameters transmitted by the client. The server preferentially loads the first-level particle level set for the processing sub-blocks within the viewing cone range to ensure that the particles in this area retain the complete particle simulation and are rendered with the highest precision. For processing sub-blocks outside the viewing cone range but still within the near field range, only the second-level particle level set is loaded, and a simplified expression is used to reduce loading bandwidth and computing resource overhead. The target particle level of each processing sub-block in the current visible area is determined through the joint distribution of the first-level and second-level particles, combined with the current viewing position, direction, and streaming radius.

[0103] When the server detects that a processing sub-chunk is outside the current visible area, meaning it's outside the projection bounds of the view frustum and doesn't meet the loading requirements, it skips loading the sub-chunk's data through the Layered Detail pipeline in its hierarchical scheduling process. This skipping mechanism, implemented based on the Unreal Engine HLOD loading architecture, uses node status tags to remove redundant sub-chunks from the data scheduling queue, thereby reducing the amount of data transferred during particle system initialization and improving overall loading efficiency.

[0104] At the same time, the server monitors the client's bandwidth in real time. If it detects that the client's current transmission bandwidth falls below a preset network latency threshold, it immediately triggers a forced downgrade of the particle system's level of detail. This downgrade involves downgrading from the first or second level to a particle level set with a lower particle retention ratio, and enabling a particle instance merging strategy. This strategy uses material identifiers as the aggregation basis, merging all particle units of the same material into a unified draw call, reducing the GPU draw load at the rendering level and significantly compressing the bandwidth of the encoded video stream.

[0105] In addition, to further enhance the foresight of particle system scheduling, the server calculates the rate of change of view distance based on the user view position change data continuously transmitted by the client, and performs trajectory prediction based on the historical movement trajectory. This prediction operation builds a prediction model based on the long short-term memory network. After determining the area that the user's future view position may pass through, it actively loads the second-level particle level set of the adjacent sub-blocks within the predicted trajectory range and caches it in the rendering buffer pool. In order to avoid visual jumps caused by particle mutations when switching detail levels, the server inserts fading transition particles during the level switching process. The transition particles use the Alpha channel to control the construction of gradient transparency between the two levels of particle sets, smoothly completing the visual transition and ensuring the continuity and stability of the three-dimensional Gaussian splash modeling scene under the dynamic switching of multi-level particles.

[0106] S150: Building a distributed rendering node cluster on the server side, and allocating rendering tasks of each processing sub-block of the target layer to multiple rendering nodes to perform real-time rendering operations.

[0107] In one possible implementation, a distributed rendering node cluster is constructed on the server side, and rendering tasks for each processing sub-block of the target level are distributed to multiple rendering nodes to perform real-time rendering operations, specifically including: determining the rendering complexity of the sub-block based on the Gaussian unit density of each sub-block; monitoring the rendering load status of each rendering node in real time, adjusting the task allocation among multiple rendering nodes in real time according to the rendering complexity, and preferentially migrating sub-blocks with a rendering complexity higher than a preset complexity to rendering nodes with an idle rendering load status; during the rendering process of the rendering nodes, for the first-level particles, volumetric fog effects and real-time ray tracing mechanisms are enabled, and a volumetric lighting cache is generated and shadow casting effects are calculated based on radiosity properties; for the second-level particles, pre-baked light maps are used in combination with a simplified signed distance field model and a distance field ambient occlusion algorithm to approximately simulate global illumination; the rendered images generated by each rendering node are encoded using H.264 and transmitted through the WebRTC protocol for low-latency pixel streaming. At the same time, the server dynamically matches the bitrate and resolution based on the current particle level and the client type, and performs particle level degradation and bitrate reduction processing when it detects that network fluctuations exceed a set threshold.

[0108] Specifically, during the implementation process of building a distributed rendering node cluster on the server side and assigning the rendering tasks of the processing sub-blocks corresponding to the target particle level to multiple rendering nodes for real-time rendering operations, the rendering complexity of each processing sub-block is first determined based on the density of three-dimensional Gaussian cells within the sub-block. This rendering complexity is comprehensively evaluated by counting the number of Gaussian cells per unit volume and combining it with the particle radius and lighting properties to generate a rendering complexity score for each sub-block, which is used to prioritize subsequent rendering tasks.

[0109] Subsequently, the server continuously monitors the rendering load status of all rendering nodes in real time, including but not limited to node GPU utilization, rendering frame time, memory usage, and the number of concurrent tasks. Based on the aforementioned rendering complexity score, the server dynamically adjusts the task allocation relationship between multiple rendering nodes through the scheduling management module. If the rendering complexity of a sub-block is found to be higher than the set complexity threshold and the load of its original bound node is saturated, the rendering task of the sub-block is automatically migrated to a rendering node that is currently idle or has a lower load, thereby improving overall cluster resource utilization and avoiding overloading of a single node.

[0110] During the rendering process, different particle levels utilize differentiated lighting processing mechanisms. For particles at the first level, the render node uses volumetric fog and real-time ray tracing modules to calculate each particle's lighting response based on its radiosity properties, generating a volumetric lighting cache. This cache then calculates shadow casting results based on the current ambient light configuration, achieving high-precision volumetric lighting simulation. For particles at the second level, the render node no longer calculates lighting paths in real time. Instead, it uses pre-baked lightmaps, directly mapping light intensity to pre-generated map data. This, combined with a simplified signed distance field model and distance field ambient occlusion algorithms, approximates the global lighting environment, ensuring rendering efficiency while maintaining visual consistency.

[0111] After rendering is complete, each rendering node inputs its generated image frame data into the video encoding module, which uses the H.264 encoding standard for compression encoding and low-latency pixel streaming transmission via the WebRTC protocol. This pixel streaming transmission features a frame-level synchronization mechanism to ensure that image data generated by different nodes reaches the client on time and is stitched together into the rendered image, maintaining the stability of continuous frame display. During the transmission process, the server dynamically matches the bitrate and resolution strategy based on particle level information and client type (such as web, mobile, or VR terminal). High-level particles use high bitrate and high resolution transmission by default, while low-level particles use low bitrate and medium-to-low resolution configuration. When the network status monitoring module detects that indicators such as bandwidth, latency, jitter, or packet loss exceed preset thresholds, the server immediately executes particle level degradation operations and bitrate compression adjustment strategies, prioritizing ensuring a stable transmission frame rate of more than 30 frames per second and controlling interactive response delays within 50 milliseconds, thereby maintaining the stable and smooth presentation of complex three-dimensional Gaussian splash modeling scenes under dynamic network conditions.

[0112] S160 , encoding the images rendered by each rendering node into a video stream, decoding and displaying the received video stream data, and simultaneously collecting user interaction instructions in real time and triggering particle level update and re-rendering operations.

[0113] In one possible implementation, the images rendered by each rendering node are encoded into a video stream, the received video stream data is decoded and displayed, and the user's interactive instructions are collected in real time to trigger particle level updates and re-rendering operations. Specifically, the following steps are performed: a multi-user rendering instance management mechanism is established on the server side. Based on the dedicated server architecture of the Unreal Engine, a unique instance identifier is generated for each user request, and an independent rendering process is allocated according to the instance identifier to isolate particle detail level loading and computing resources; when the client initiates access, the corresponding video stream content is received by binding to the instance identifier and decoded to present the target modeling scene; based on the socket protocol, the user's terminal parameters and interactive input, including perspective switching, zoom instructions and control commands, are collected, and the collected results are transmitted back to the server side in real time to trigger dynamic switching of the particle detail level in the current visible area and re-rendering operations of the target processing sub-block.

[0114] Specifically, in the process of encoding the images rendered by each rendering node into video streams and completing client decoding and display as well as real-time interactive responses, a multi-user rendering instance management mechanism is first established on the server side. This mechanism is based on the Unreal Engine's dedicated server architecture to separately schedule user requests. Specifically, when any client initiates an access request, the server-side instance management module generates a unique instance identifier for the request and, based on the instance identifier, starts an independent rendering process in the rendering node cluster. This rendering process runs in isolation from other instances in terms of resource allocation, ensuring that its particle detail level loading process and calculation tasks will not be interfered with by other rendering requests, thereby ensuring independence, isolation and response consistency under multi-user access.

[0115] After the client completes instance binding, it connects to the corresponding rendering process using the instance identifier, receiving and decoding the video stream content pushed in real time by the rendering node. The rendering results are then encoded in H.264 on the server side and pushed to the client via the WebRTC protocol with low latency. The client then calls the video decoding module to convert the received compressed video stream data into a sequence of frames for continuous playback, thus achieving remote visual presentation of the target modeling scene without the need for local rendering. This provides a cross-platform, highly efficient, and lightweight terminal rendering experience.

[0116] At the same time, the client continuously performs real-time acquisition of interactive input, specifically encoding and transmitting the terminal's current state and user operation commands via a socket protocol. This acquisition includes, but is not limited to, the current camera viewing angle parameters (position, direction, elevation), zoom scale, mouse dragging, touch sliding, and other input behaviors, as well as additional interactive control commands such as function calls and menu selections. This interactive data packet is sent to the server in real time in a structured format, where it is parsed by the server's interactive scheduling module to drive the recalculation process of the visible area particle system.

[0117] When the server receives the latest interactive input from the client, it recalculates the visible area and particle detail level configuration in real time based on the changed terminal parameters. If the viewing angle offset causes the rendering coverage of the currently processed sub-block to change, the particle level update of the corresponding sub-block and the re-rendering of the target area will be triggered immediately. The server re-schedules the target particle level set based on the latest interactive state, completes particle reloading, lighting resampling, and image recoding through the node process bound to the rendering instance, forms new video frame content and transmits it back to the client for continuous display, thus forming a closed-loop feedback mechanism of "interactive input-particle update-image recoding-video replay", achieving low-latency response and high-quality dynamic display capabilities for three-dimensional Gaussian splash modeling scenes in multi-terminal environments.

[0118] In one possible implementation, refer to Figure 2 First, a two-dimensional map partitioning operation is performed based on the world coordinate system of the target modeling scene, dividing the original modeling area into multiple standard map units at equal intervals, forming a spatially continuous sub-block structure, which serves as the basic unit for all subsequent spatial processing. Secondly, building height data is introduced as vertical clipping supervision information. By integrating remote sensing elevation, floor structure, and relevant fields in the three-dimensional geographic database, the extreme height values ​​of buildings or terrain within each map sub-block are counted, and a 10-meter redundant value is added to form a height constraint. This step provides the basis for subsequent high-altitude artifact removal and aerial point cloud cleaning.

[0119] Subsequently, a spatial boundary framework was established based on the map distribution and building density information, completing the sub-block division planning of the scene. During the division process, boundary clipping was performed to eliminate irregular edges around the model, aerial anomalies, and unreasonable distant cloud data (such as sky, fog layer, and aerial photography errors) in the original model. Each sub-block model was vertically truncated according to the preset limit height. The structural results after regional model registration and boundary clipping shown in the figure indicate that spatial registration has aligned each regional model with the standard coordinate system, and has independent loading and particle continuity.

[0120] Finally, the 3D Gaussian splatter model within each sub-block is loaded into the Unreal Engine environment, where it is combined with UE5's sky atmosphere and 2D imagery to perform a distant rendering simulation. After cropping, each sub-block's 3DGS particle model is loaded separately, supporting Level of Detail (LOD) and multi-partition dynamic streaming. This provides comprehensive pre-processing spatial data support for efficient scheduling and distributed rendering of city-scale 3D modeling content. This overall process embodies the solution's optimized design philosophy for Gaussian particle spatial organization, cropping accuracy control, and rendering compatibility during the data preprocessing phase.

[0121] This embodiment also discloses a large-scale scene multi-level of detail cloud rendering processing device based on 3DGS, referring to Figure 3 , comprising an acquisition module 301, a processing module 302 and an output module 303, the device is used to execute any of the above-mentioned large-scale scene multi-level of detail cloud rendering processing methods based on 3DGS, wherein:

[0122] The acquisition module 301 is used to cut the target modeling scene into multiple sub-blocks.

[0123] The processing module 302 is used to perform spatial registration and boundary clipping processing on each sub-block, and perform particle redundancy reconstruction and overlapping area marking on the boundary areas between adjacent sub-blocks to obtain processed sub-blocks.

[0124] The processing module 302 is configured to perform multiple levels of detail division on each processing sub-block to generate a particle level set including multiple particle retention ratios and geometric error thresholds.

[0125] The processing module 302 is used to build a world partition mechanism on the server side, determine the current visible area according to the user motion data transmitted by the client, and schedule the target level of the particle level set in the current visible area.

[0126] The processing module 302 is used to build a distributed rendering node cluster on the server side, and distribute the rendering tasks of each processing sub-block of the target layer to multiple rendering nodes to perform real-time rendering operations.

[0127] The output module 303 is used to encode the images rendered by each rendering node into video streams, decode and display the received video stream data, and at the same time collect user interaction instructions in real time and trigger particle level updates and re-rendering operations, thereby achieving re-rendering of the target modeling scene.

[0128] In a possible implementation, the processing module 302 is configured to perform spatial registration on the 3DGS model according to the partition coordinates of each sub-block, and to crop the 3DGS model according to the partition boundaries and the restricted height to eliminate distant point clouds of the 3DGS model.

[0129] The processing module 302 is configured to determine a complex block in which the Gaussian unit density in the plurality of sub-blocks exceeds a preset standard.

[0130] The processing module 302 is used to perform redundancy adjustment on the boundaries of adjacent sub-blocks of the complex block and mark the dependency relationship, and mark the overlapping area of ​​the boundary Gaussian units to ensure visual continuity during streaming loading.

[0131] In one possible embodiment, the output module 303 is used to perform multi-level detail division according to the density and radius of the Gaussian distribution to generate a hierarchical particle level set, wherein the first level higher than the preset level among the multiple particle levels of the particle level set retains the complete particle simulation, and the second level lower than the preset level among the multiple particle levels of the particle level set is simplified or represented as a static billboard, and the simplification rule is density and radius clustering simplification based on the Gaussian distribution, Gaussian units are merged for sparse areas, and density sampling is retained for dense areas. The static billboard only retains position and rotation properties, and the simplification process retains the error threshold.

[0132] In one possible implementation, the acquisition module 301 is configured to perform spatial index management on multiple processing sub-blocks based on the world partitioning mechanism of the Unreal Engine on the server side, and to divide the 3DGS model into multiple streaming partitions based on the user viewing distance parameter transmitted by the client.

[0133] The processing module 302 is used to preferentially load the first level within the viewing cone according to the user's current viewing direction and field of view, only load the second level for the area outside the viewing cone, and determine the target level based on the first and second levels.

[0134] The processing module 302 is configured to skip loading data of the sub-block outside the current visible area through the layered detail pipeline when it is detected that any sub-block is outside the current visible area.

[0135] The processing module 302 is used to trigger a forced degradation operation of the particle system detail level when the client bandwidth is lower than a preset network delay threshold, and adopt a particle instance merging strategy to merge particle units of the same material into a unified drawing call to reduce real-time transmission overhead.

[0136] Processing module 302 is used to preload the second-level hierarchical data of adjacent sub-blocks in the user movement trajectory prediction area based on the user's viewing angle distance change rate and historical behavior records, and insert fade-out transition particles during the level switching process to achieve visually smooth transition between levels through alpha channel control.

[0137] In a possible implementation, the processing module 302 is configured to determine the rendering complexity of each sub-block based on the Gaussian unit density of each sub-block.

[0138] The acquisition module 301 is used to monitor the rendering load status of each rendering node in real time, adjust the task allocation among multiple rendering nodes in real time according to the rendering complexity, and preferentially migrate sub-blocks with a rendering complexity higher than a preset complexity to rendering nodes with an idle rendering load status.

[0139] The processing module 302 is used to enable volumetric fog effect and real-time ray tracing mechanism for the first level of particles during the rendering process of the rendering node, generate volumetric lighting cache according to radiosity attributes, and calculate shadow casting effect.

[0140] The processing module 302 is configured to approximately simulate global illumination for the second level of particles by using a pre-baked light map combined with a simplified signed distance field model and a distance field ambient occlusion algorithm.

[0141] Processing module 302 is used to encode the rendered images generated by each rendering node using H.264 and transmit them through the WebRTC protocol for low-latency pixel streaming. At the same time, the server dynamically matches the bitrate and resolution based on the current particle level and client type, and performs particle level degradation and bitrate reduction processing when it detects that network fluctuations exceed a set threshold.

[0142] In one possible implementation, processing module 302 is used to build a multi-user rendering instance management mechanism on the server side, based on the dedicated server architecture of Unreal Engine, to generate a unique instance identifier for each user request, and to allocate an independent rendering process according to the instance identifier to isolate particle detail level loading and computing resources.

[0143] The processing module 302 is configured to receive the corresponding video stream content through the binding instance identifier and decode and present the target modeling scene when the client initiates access.

[0144] Processing module 302 is used to collect user terminal parameters and interactive input based on the socket protocol, including perspective switching, zoom instructions and control commands, and transmit the collected results back to the server in real time to trigger dynamic switching of the particle detail level of the current visible area and re-rendering operations of the target processing sub-block.

[0145] In one possible implementation, the processing module 302 is used to cut the scene according to the regional coordinates set for the target modeling scene and the two-dimensional geographic relationship of the world coordinates, forming a continuous standard map and introducing basic terrain and building height data in different areas as supervision, thereby dividing the target modeling scene into multiple sub-blocks.

[0146] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0147] This embodiment also discloses an electronic device, referring to Figure 4 The electronic device may include: at least one processor 401 , at least one communication bus 402 , a user interface 403 , a network interface 404 , and at least one memory 405 .

[0148] The communication bus 402 is used to implement the connection and communication between these components.

[0149] The user interface 403 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 403 may also include a standard wired interface and a wireless interface.

[0150] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0151] The processor 401 may include one or more processing cores. The processor 401 utilizes various interfaces and lines to connect various parts within the entire server, and executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 405, as well as calling data stored in the memory 405. Optionally, the processor 401 may be implemented in the form of at least one hardware component selected from digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 401 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs. The GPU is responsible for rendering and drawing the content to be displayed on the display screen. The modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 401 and may be implemented separately on a single chip.

[0152] Among them, the memory 405 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory 405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 405 may also optionally be at least one storage device located away from the aforementioned processor 401. The memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface 403 module and an application for a large-scale scene multi-level of detail cloud rendering processing method based on 3DGS.

[0153] exist Figure 4 In the electronic device shown, user interface 403 is primarily used to provide an input interface for the user and obtain user input data. Processor 401 can be used to invoke an application stored in memory 405 that describes a method for processing large-scale scene multi-level-of-detail cloud rendering based on 3DGS. When executed by one or more processors 401, the electronic device executes one or more of the methods described in the aforementioned embodiments.

[0154] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0155] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0157] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0158] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory 405 and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory 405 includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disk.

[0160] The present application also discloses a computer-readable storage medium storing instructions, which, when executed by one or more processors 401 , enable an electronic device to execute one or more methods in the above embodiments.

[0161] The above are merely exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A large-scale scene multi-level of detail cloud rendering processing method based on 3DGS, characterized in that: The method comprises: Performing image segmentation on the target modeling scene to divide the target modeling scene into multiple sub-blocks; Performing spatial registration and boundary clipping processing on each of the sub-blocks, and performing particle redundancy reconstruction and overlapping area marking on the boundary areas between adjacent sub-blocks to obtain processed sub-blocks; Performing multi-level-of-detail division on each of the processing sub-blocks to generate a particle level set including multiple particle retention ratios and geometric error thresholds; Build a world partitioning mechanism on the server side, determine the current visible area based on the user motion data transmitted by the client, and schedule the target level of the particle level set within the current visible area; Building a distributed rendering node cluster on the server side, and assigning rendering tasks of each processing sub-block of the target layer to multiple rendering nodes to perform real-time rendering operations; The images rendered by each rendering node are encoded into video streams, and the received video stream data is decoded and displayed. At the same time, user interaction instructions are collected in real time and particle level updates and re-rendering operations are triggered, thereby achieving re-rendering of the target modeling scene.

2. A large-scale scene multi-level of detail cloud rendering processing method based on 3DGS according to claim 1, characterized in that: The performing of spatial registration and boundary clipping processing on each sub-block, and performing particle redundancy reconstruction and overlapping area marking on the boundary areas between adjacent sub-blocks to obtain processed sub-blocks specifically includes: Performing spatial registration on the 3DGS model according to the partition coordinates of each of the sub-blocks, and clipping the 3DGS model according to the partition boundary and the restricted height to eliminate distant point clouds of the 3DGS model; determining a complex block in which the Gaussian unit density in the plurality of sub-blocks exceeds a preset standard; Redundant adjustments are made to the boundaries of adjacent sub-blocks of the complex block and dependencies are marked, and overlapping areas of boundary Gaussian units are marked to ensure visual continuity during streaming loading.

3. The method for large-scale scene multi-level of detail cloud rendering processing based on 3DGS according to claim 1, characterized in that: The performing of multi-level detail division on each of the processing sub-blocks to generate a particle level set including multiple particle retention ratios and geometric error thresholds specifically includes: Multiple levels of detail are divided according to the density and radius of the Gaussian distribution to generate a hierarchical particle level set, wherein the first level higher than the preset level among the multiple particle levels in the particle level set retains the complete particle simulation, and the second level lower than the preset level among the multiple particle levels in the particle level set is simplified or represented as a static billboard. The simplification rule is density and radius clustering simplification based on Gaussian distribution, Gaussian units are merged for sparse areas, and density sampling is retained for dense areas. The static billboard only retains position and rotation properties, and the simplification process retains the error threshold.

4. The method for large-scale scene multi-level of detail cloud rendering processing based on 3DGS according to claim 3, characterized in that: The world partition mechanism is constructed on the server side, and the current visible area is determined based on the user motion data transmitted by the client, and the target level of the particle level set in the current visible area is scheduled, specifically including: Performing spatial index management on the plurality of processing sub-blocks based on the world partition mechanism of the Unreal Engine on the server side, and dividing the 3DGS model into a plurality of streaming partitions based on a user viewing distance parameter transmitted by the client; Prioritizing the loading of the first level within the viewing cone based on the user's current viewing direction and field of view, and only loading the second level for areas outside the viewing cone, and determining the target level based on the first level and the second level; When it is detected that any one of the sub-blocks is outside the current visible area, skipping data loading of the sub-block outside the current visible area through the layered detail pipeline; When the client's bandwidth is lower than a preset network latency threshold, a forced degradation operation of the particle system's level of detail is triggered, and a particle instance merging strategy is adopted to merge particle units of the same material into a unified draw call to reduce real-time transmission overhead; Based on the user's viewing angle distance change rate and historical behavior records, the second-level hierarchical data of adjacent sub-blocks is preloaded in the user movement trajectory prediction area, and fading transition particles are inserted during the level switching process to achieve visually smooth transition between levels through Alpha channel control.

5. The method for large-scale scene multi-level of detail cloud rendering processing based on 3DGS according to claim 3, characterized in that: The step of constructing a distributed rendering node cluster on the server side and allocating rendering tasks of each processing sub-block of the target layer to multiple rendering nodes to perform real-time rendering operations specifically includes: determining a rendering complexity of the sub-block based on a Gaussian unit density of each sub-block; Monitor the rendering load status of each rendering node in real time, adjust the task allocation among the multiple rendering nodes in real time according to the rendering complexity, and preferentially migrate the sub-blocks with a rendering complexity higher than a preset complexity to the rendering nodes with an idle rendering load status; During the rendering process of the rendering node, for the particles of the first level, volume fog effect and real-time ray tracing mechanism are enabled, volume lighting cache is generated according to radiosity attributes, and shadow casting effect is calculated; For the second level of particles, we use pre-baked lightmaps combined with a simplified signed distance field model and distance field ambient occlusion to approximate global illumination. The rendered images generated by each rendering node are encoded in H.264 and transmitted through the WebRTC protocol for low-latency pixel streaming. At the same time, the server dynamically matches the bit rate and resolution based on the current particle level and the type of the client, and performs particle level degradation and bit rate reduction processing when it detects that network fluctuations exceed a set threshold.

6. The method for large-scale scene multi-level of detail cloud rendering processing based on 3DGS according to claim 1, characterized in that: The video stream encoding of the images rendered by each rendering node, decoding and displaying the received video stream data, and simultaneously collecting user interaction instructions in real time and triggering particle level update and re-rendering operations specifically include: A multi-user rendering instance management mechanism is established on the server side. Based on the dedicated server architecture of Unreal Engine, a unique instance identifier is generated for each user request, and an independent rendering process is allocated according to the instance identifier to isolate particle detail level loading and computing resources. When the client initiates access, the corresponding video stream content is received by binding the instance identifier and decoding and presenting the target modeling scene; The user's terminal parameters and interactive inputs, including perspective switching, zoom instructions, and control commands, are collected based on the socket protocol, and the collected results are transmitted back to the server in real time to trigger the dynamic switching of the particle detail level of the current visible area and the re-rendering operation of the target processing sub-block.

7. The method for large-scale scene multi-level of detail cloud rendering based on 3DGS according to claim 1, characterized in that: The target modeling scene is cut into multiple sub-blocks, specifically including: According to the regional coordinates set for the target modeling scene, the scene is cut according to the two-dimensional geographical relationship of the world coordinates to form a continuous standard map and introduce the basic terrain and building height data in different areas as supervision to divide the target modeling scene into multiple sub-blocks.

8. A large-scale scene multi-level of detail cloud rendering processing device based on 3DGS, characterized in that: The device is used to execute a large-scale scene multi-level of detail cloud rendering processing method based on 3DGS according to any one of claims 1 to 7, and the device comprises an acquisition module (301), a processing module (302) and an output module (303), wherein: The acquisition module (301) is used to cut the target modeling scene into multiple sub-blocks; The processing module (302) is used to perform spatial registration and boundary clipping processing on each sub-block, and perform particle redundancy reconstruction and overlapping area marking on the boundary areas between adjacent sub-blocks to obtain processed sub-blocks; The processing module (302) is used to perform multi-level detail division on each of the processing sub-blocks to generate a particle level set including multiple particle retention ratios and geometric error thresholds; The processing module (302) is used to build a world partition mechanism on the server side, determine the current visible area according to the user motion data transmitted by the client, and schedule the target level of the particle level set in the current visible area; The processing module (302) is used to build a distributed rendering node cluster on the server side, and distribute the rendering tasks of each processing sub-block of the target layer to multiple rendering nodes to perform real-time rendering operations; The output module (303) is used to encode the images rendered by each rendering node into a video stream, decode and display the received video stream data, and simultaneously collect user interaction instructions in real time and trigger particle level update and re-rendering operations, thereby achieving re-rendering of the target modeling scene.

9. An electronic device, characterized in that: The electronic device comprises a processor (401), a communication bus (402), a user interface (403), a network interface (404) and a memory (405), wherein the memory (405) is used to store instructions, the user interface (403) and the network interface (404) are both used to communicate with other devices, the communication bus (402) is used to realize connection and communication between components in the electronic device, and the processor (401) is used to execute the instructions stored in the memory (405) so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images