Rendering method and system for real-time interactive three-dimensional gaussian splatting scene, and storage medium

Abstract

The present disclosure relates to a data rendering method for a real-time interactive three-dimensional Gaussian splatting scene. In the method, longitudinal and lateral block-wise partitioning is performed on a three-dimensional Gaussian splatting scene on the basis of a non-solidifying progressive construction technique, so as to optimize streaming media loading performance; furthermore, by means of scene block compression and a priority-based transmission and loading technique based on a user visual perception center, the data transmission efficiency and online real-time rendering efficiency of the three-dimensional Gaussian splatting scene can be effectively improved even under bandwidth-limited conditions, thereby satisfying real-time interaction requirements and improving the user experience even in network fluctuation environments.

Description

Real-time interactive 3D Gaussian sputtering scene rendering method, storage medium and system Technical Field

[0001] This disclosure relates to computer technology, and in particular to a real-time interactive three-dimensional Gaussian sputtering scene data rendering method, storage medium, and system. Background Technology

[0002] 3D Gaussian sputtering technology performs well in high-fidelity rendering, but generates a large amount of scene data. Especially when network bandwidth is limited and device rendering capabilities vary, transmitting this data to users via streaming media will face severe challenges.

[0003] To improve the rendering and transmission efficiency of 3D scenes, existing technologies employ a layered progressive construction method. This method constructs a layered structure for stacked representation, allowing different quality levels of the scene to be loaded layer by layer according to network conditions in real-time interactive and streaming media scenarios. This progressively refines the 3D scene representation, achieving dynamic adaptive transmission efficiency and viewpoint quality. However, this method still has the following drawbacks.

[0004] First, this method primarily relies on layer-by-layer accumulation of Hoskiy units to add detail, thus passively reducing the number of Hoskiy units in the entire scene, but failing to achieve sufficiently efficient compression. This results in a still large overall data volume for the constructed scene, limiting transmission efficiency in low-bandwidth network environments. If external programs are used to compress and decompress the scene transmitted layer by layer, the large data volume divided by layer will cause rendering delays, leading to a poor user experience.

[0005] Secondly, each layer in the hierarchical structure is built independently. During the accumulation process, the upper enhancement layers are only responsible for adding extra details to the existing base layers, and cannot optimize or adjust the properties of the lower-level Gaussian units other than transparency. This lack of inter-layer interaction easily leads to the scene representation getting stuck in local optima; that is, each layer only optimizes its own Gaussian units, without making comprehensive adjustments based on the global scene. This local optima problem not only affects compression efficiency but also leads to data redundancy, failing to fully exploit the global characteristics of the scene to achieve more efficient compression.

[0006] Finally, its layer-by-layer transmission method ignores the user's focus on specific areas in 3D space, failing to dynamically prioritize the transmission of detailed information from the user's visual perception center. As a result, the system wastes bandwidth when transmitting large amounts of data from low-priority areas. This design not only leads to unnecessary transmission overhead but also reduces loading efficiency in real-time interactive scenarios, failing to meet the needs of dynamic adjustments.

[0007] Summary of the Invention

[0008] Based on this, the purpose of this disclosure is to propose a method, storage medium, and system for constructing and transmitting a 3D Gaussian sputtering scene for real-time interaction. The method utilizes a non-fixed progressive construction technique to perform vertical and horizontal block-based partitioning of the 3D Gaussian sputtering scene to optimize streaming media loading performance. Furthermore, through scene block compression technology and a priority transmission loading technique based on the user's visual perception center, the data transmission efficiency and online real-time rendering efficiency of the 3D Gaussian sputtering scene can be effectively improved even with limited bandwidth. Even in environments with fluctuating network conditions, it can meet the needs of real-time interaction and enhance the user experience. The specific technical solution is as follows.

[0009] In a first aspect, this disclosure proposes a real-time interactive 3D Gaussian sputtering scene rendering method, the method comprising the following steps: on the client side, the mesh vertices are projected onto a 2D screen, and the center of the rendered screen is determined based on the real-time acquired current user visual perception center; according to the distance between the center of the rendered screen and the center point of the mesh projection, the client determines or adjusts the transmission request priority of each layer of scene blocks under the current user visual perception center; from low-quality layers to high-quality layers, after a corresponding transmission request for one layer is responded to, the client sends the corresponding transmission request for the next layer to the server; after receiving the compressed scene blocks transmitted by the server, the client uses a renderer to load the scene layer by layer from low-quality layers to high-quality layers, progressively enhancing the 3D scene; wherein, the mesh is generated by vertically and horizontally splitting the layered 3D Gaussian sputtering scene into blocks.

[0010] In one implementation of the above technical solution, the client uses BlockMap to record the highest level of synchronized scene blocks under different grids; when the client and server disconnect and reconnect successfully, the client initiates transmission requests to the server layer by layer for the scene blocks that have not been received, according to the records in BlockMap.

[0011] In one embodiment of the above technical solution, the priority of transmission requests is determined as follows: requests for scene blocks that include the center of the rendered screen are given the first priority, and requests for scene blocks that do not include the center of the rendered screen are given the second priority based on their distance from the center point of the grid projection, from closest to farthest.

[0012] In one embodiment of the above technical solution, if two adjacent grids have different quality levels at the grid boundary, the transparency of the transition zone is determined by interpolation by setting a transition zone between the two grids.

[0013] In one embodiment of the above technical solution, the three-dimensional Gaussian sputtering scene is constructed using a non-solidified layered progressive construction method. This non-solidified layered progressive construction method, based on the layered progressive construction method, allows the current layer to update the transparency and scale of Gaussian elements in the previous layer's scene during the training process.

[0014] In one embodiment of the above technical solution, a scale threshold and a transparency threshold are set so that the ratio of the size update to the original scale does not exceed the scale threshold, and if the transparency of a Gaussian cell after the update is lower than the transparency threshold, the Gaussian cell is deleted.

[0015] In one embodiment of the above technical solution, the transmission request is stored in a queue, and a sliding queue window is used to ensure timeout retransmission and orderly transmission. The size of the sliding queue window is adjustable, and a multi-threaded concurrency mechanism is adopted to simultaneously transmit the 3D Gaussian data of the scene blocks corresponding to the center position of the rendered screen and its surrounding areas.

[0016] In one embodiment of the above technical solution, at the start of rendering, the client obtains metadata about the 3D Gaussian sputtering scene from the server, including the total scene size, supported quality levels, scene segmentation method, number of scene segments, and compression standard used.

[0017] Secondly, this disclosure proposes a computer-readable storage medium storing a computer program that can be loaded by a processor and executed by any of the methods described above.

[0018] Thirdly, this disclosure proposes a real-time interactive 3D Gaussian sputtering scene rendering system, comprising a client, a server, and a network module. The server constructs a 3D Gaussian sputtering scene using a non-fixed layered progressive construction method, and segments the constructed 3D Gaussian sputtering scene to generate a mesh. The client projects the mesh vertices onto a 2D screen, determines the center of the rendered screen based on the real-time acquired current user visual perception center, and determines or adjusts the transmission request priority of each scene block under the current user visual perception center based on the distance between the rendered screen center and the mesh projection center point. After receiving the scene blocks transmitted by the server, the client uses a renderer to progressively enhance the 3D scene by loading layers from low-quality to high-quality layers. The network module, from low-quality to high-quality layers, sends the next layer's corresponding transmission request to the server after receiving a response to the transmission request for one layer. The non-fixed layered progressive construction method, based on the layered progressive construction method, allows the current layer to update the transparency and scale of Gaussian elements in the previous layer's scene during training.

[0019] The beneficial technical effects of this disclosure are as follows: By constructing a 3D Gaussian sputtering scene using a non-fixed, layered, progressive approach, introducing efficient compression technology and a dynamically adaptive transmission strategy, and employing an adaptive, block-based smoothing renderer, the transmission efficiency and rendering performance of the 3D scene are significantly improved. This effectively expands its applications in virtual reality, augmented reality, cloud gaming, 3D streaming media, and online digital archive systems, providing support for fields such as autonomous driving simulation, remote collaborative design, smart city modeling, and 3D content generation. This technology will also promote the widespread application of 3D Gaussian sputtering technology to common consumer devices, such as mobile terminals and lightweight hardware devices. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 is a schematic diagram of the system architecture in one embodiment. In Figure 1, 100 is a non-solidified layered progressive construction of a 3D Gaussian sputtering scene, 200 is a mesh block LOD layered scene, 300 is a transmission strategy management module, 400 is user visual feedback, 500 is a byte stream, and 600 is a renderer.

[0022] Figure 2. Schematic diagram of a non-solidified layered progressive construction method in one embodiment.

[0023] Figure 3. A schematic diagram of a scene transmission strategy prioritizing visual perception in one implementation. Detailed Implementation

[0024] The relevant terms are defined as follows.

[0025] 3D Gaussian Splatting: 3D Gaussian sputtering is a technique for representing and rendering 3D scenes. It maps the scene's geometry and attributes to a series of Gaussian primitives with properties such as position, color, and transparency by using Gaussian function primitives in 3D space. Each Gaussian primitive's attributes include position, size, color, opacity, and rotation; these attribute combinations represent object information in 3D space. Compared to traditional 3D mesh models, 3D Gaussian sputtering can construct realistic and complex scenes at a lower cost and with less equipment, making it particularly suitable for online browsing services of 3D digital collections.

[0026] Adaptive streaming (Structure from Motion): Adaptive streaming is a technology used to transmit audio and video content in a network environment. It can dynamically adjust the quality of streaming media based on network bandwidth, device performance, and current network conditions to ensure streaming continuity and the best user experience.

[0027] Level of Detail (LOD) is a technique in computer graphics used to optimize 3D scene rendering and improve performance. By creating multiple models of different complexities for the same object, the system can dynamically select the appropriate model version for rendering based on factors such as viewing distance and screen resolution. LOD technology is widely used in games, virtual reality, 3D modeling, and online 3D digital archive systems to significantly improve rendering frame rates and save computing resources.

[0028] Referring to Figure 1, this scheme uses a non-fixed, layered, progressive construction method to create a 3D Gaussian sputtering scene (100), and then applies a grid-block LOD layered scene (200) to the constructed 3D Gaussian sputtering scene. Next, the scene is segmented and compressed using a priority strategy based on the user's visual perception center, and the data is transmitted to the renderer (600) in the form of a byte stream (500). The user's visual perception center in the priority strategy based on the user's visual perception center can be represented as (view, x, y), where x and y are the positions of the user's visual perception center, and view is the viewpoint.

[0029] The following will describe clearly and completely how the technical solution of this case is implemented, with reference to Figures 2-3. Obviously, the described implementation method is only a part of the implementation method of this case, and not all of the implementation methods.

[0030] (I) Construction Method

[0031] Traditional layer-by-layer progressive 3D Gaussian sputtering algorithms only build details by accumulating Gaussian primitives layer by layer. Since only transparency can be adjusted, the properties of the lower-level Gaussian primitives are fixed, ultimately leading to the model easily getting trapped in local optima and failing to fully represent high-frequency details. Furthermore, the lack of interactive optimization between layers easily generates redundant Gaussian primitives.

[0032] This solution employs a non-fixed, layered, progressive approach to construct a 3D Gaussian sputtering scene. By introducing a local dynamic optimization mechanism into the layered structure, this method allows for refinement adjustments to the bottom-level primitives, particularly edge regions. This effectively optimizes high-frequency details within the scene, avoiding excessive reliance on subsequent layers for refinement and improving the overall scene performance from a global perspective. Simultaneously, this adjustment mechanism reduces the generation of redundant Gaussian primitives during the densification process in subsequent layers, significantly improving data compression ratio and global efficiency, achieving a balance between high-efficiency compression and detail.

[0033] First, based on the input multi-view image set {I i Construct a multi-resolution image set version D. Based on the hierarchical design of quality levels 0 to N, where N is the highest quality level, downsample each image to obtain an image pyramid with resolutions ranging from smallest to largest. in Let represent the k-th quality level image from the i-th viewpoint.

[0034] The scene is built starting from the lowest quality level, level=0, using a set of images at the corresponding resolution. As input, the distribution of a set of initial Gaussian elements is optimized to form scene G0.

[0035] in, It is the optimized scene base layer. It is the foundational layer that needs optimization during the training process. This represents the process of rendering the base layer to be optimized into a 2D image.

[0036] Let represent the i-th real input image at resolution level 0, and M represent the number of viewpoints in the input image. arg represents the expression that makes the function... Variable that achieves minimum value

[0037] As training progresses, each layer uses a higher resolution view. As input to train the enhancement layer Enhancement layer Scenes of the previous quality level Combined, to form a new quality level scenario This stacking construction method makes full use of the structural consistency between LOD scene models. The stacking formula for scenes is as follows:

[0038] base layer This represents the lowest quality 3D Gaussian sputtering model, containing basic geometric and attribute information of the scene. Each enhancement layer... This is achieved through incremental optimization of Gaussian spectra, further improving scene resolution and detail. Through this layer-by-layer accumulation, the model can achieve higher quality at each subsequent quality level. The intermediate pre-sequence level The information is then used to add new details. See Figure 2 for illustration.

[0039] During the training of each subsequent enhancement layer, not only are all attributes of the Gaussian primitives in the current enhancement layer optimized, but some attributes of the Gaussian primitives from previous layers are also frozen. Updates to the opacity and scale of the Gaussian primitives from previous layers are allowed. If the opacity of a Gaussian primitive after the update is lower than a transparency threshold, that Gaussian primitive is deleted, thus reducing the number of redundant Gaussian primitives. If the scale update of a Gaussian primitive from a previous layer does not exceed 80% of its original scale, then the violation of global consistency in low-quality scenes can be avoided. 80% is an instance value of a set scale threshold, which is adjustable.

[0040] in, It is a globally optimized enhancement layer. It is the previous layer after optimizing the transparency and scale of Gaussian elements only. This is an enhancement layer that needs optimization. This is the previous layer that needs optimization. This represents the process of merging scenes to be optimized and rendering them into a 2D image. Let M represent the i-th real input image at resolution level k, and M represent the number of input images.

[0041] Because traditional algorithms use multi-layered resolution images to train the base layer, synthesizing low-resolution images inevitably introduces problems such as over-smoothing or artificial distortion, resulting in the low-level Gaussian primitives being insensitive to image edge details during the modeling process.

[0042] By dynamically adjusting the size of Gaussian primitives in previous layers during the layering process, the common edge distortion problem in layer-by-layer training of multi-resolution images is effectively solved. Simultaneously, freezing the updates of other parameters avoids the risk of model bloat caused by the splitting of lower-level Gaussian primitives. This optimization strategy preserves the global descriptive power of lower-level primitives while improving the detail enhancement and repair effects of subsequent layers, reducing the generation of redundant Gaussian primitives, and ensuring that the model can be refined layer by layer without sacrificing overall accuracy, thus significantly improving the final modeling performance.

[0043] (II) Compressed Transmission Strategy

[0044] This solution adopts a priority strategy based on the user's visual perception center, which is the core transmission mechanism designed in this disclosure to improve the real-time interactive experience and optimize bandwidth utilization.

[0045] This strategy divides the 3D space into multiple grids, combining this with the scene's layering to create horizontally and vertically segmented scene slices. These slices form fine-grained network transmission units, enabling data compression at the grid level and ensuring efficient transmission and decoding. Simultaneously, this strategy supports visual perception, allowing the system to prioritize transmission based on grid priority, focusing on the spatial block where the user's visual center is located, further improving resource utilization and response speed. Specifically, by analyzing the user's perspective and interaction behavior in the 3D scene, the system identifies the user's current visual perception center. Based on the priority of this focal area, scene data is sliced ​​into multiple independent grids, each with a priority set according to its importance. This ensures the user receives high-quality rendered content of the relevant portion as quickly as possible, while low-priority background or non-focal areas are transmitted at lower resolution or loaded gradually when bandwidth is sufficient.

[0046] As can be seen, this strategy fully leverages scene layering capabilities, utilizes block segmentation to achieve finer-grained network transmission and interactive processing, and dynamically senses the user's visual perception center area within the 3D scene, prioritizing the transmission of important details in these areas. This ensures that users can quickly obtain high-quality rendered content during real-time interaction, while reducing bandwidth and computing resource waste in non-critical areas. Furthermore, during transmission, the priority of scene block transmission is dynamically adjusted based on the client's visual center, achieving a user experience that prioritizes visual perception.

[0047] Therefore, this transmission strategy can improve transmission efficiency under limited bandwidth, ensure that the core visual areas that users care about can be loaded with the highest quality, reduce the transmission burden on secondary areas, and achieve efficient utilization of network resources.

[0048] In one implementation, a renderer client executing a priority strategy based on the user's visual perception center dynamically evaluates the priority of the queue of slices to be transmitted by tracking the user's viewpoint and position. It detects the region where the user's visual perception center is located in the 3D scene, thereby determining the priority of the scene blocks to be transmitted, ensuring that the most important visual content receives high-quality compression and transmission first. That is, meshes closer to the user's visual perception center are more important and assigned higher transmission priority and quality levels. Meshes farther from the user's visual perception center or located at the edge of the viewpoint are assigned lower priority. The implementation steps are detailed below with reference to Figure 3.

[0049] First, for the 3D Gaussian sputtering scene Perform vertical and horizontal block-based division to generate several square grids M. i Due to the layered structure of the scene, each mesh can be viewed as a scene block with multiple quality levels. k represents the quality level, N represents the highest quality level, and i is the grid position number. The server uses lossless compression technology to preprocess the scene blocks to save bandwidth usage during subsequent transmission.

[0050] When rendering begins, the client communicates with the server to obtain the scene from the server. Meta information includes: total scene size, supported quality levels, scene segmentation method, number of scene segments, and compression standard used.

[0051] Next, the client initializes the BlockMap and the sliding window queue. The BlockMap structure records the quality level of received scene blocks, indicating the data status that the client has received. For received scene blocks, the highest quality level at their current position is recorded as 0 ≤ level ≤ N. For grid positions where no blocks have been received, a not-received flag is set, for example, using "-1" to indicate that no blocks have been received. In one implementation, the BlockMap is initialized using the not-received flag.

[0052] The sliding window queue (Queue) is used to manage network transmission requests and responses. Clients' requests to transmit scene blocks to the server are stored in the queue according to priority, and a separate network transmission thread manages the sending of requests and the receiving of scene blocks.

[0053] At this point, the client generates all blocks at level 0. Transmission request Placed in queue Queue0. In one implementation, the vertices of the grid are projected onto the 2D screen through the current user's visual perception center, by transmitting the request. Prioritization is achieved by dividing the data into two parts based on conditions, in order to save computational resources: the center position containing the current rendering interface will be prioritized. Scene block transmission request Requests with the highest priority are directly inserted at the beginning of queue Queue0, while other requests are based on the grid projection center point and the screen center point. Distance is used as the second priority for sorting.

[0054] Then, the client sends requests in sequence. If a response is received from the server, slide the queue window and send the next request. `WindowSlide(Queue0, i)`

[0055] If the client sends a message containing the center of the screen... First priority scene block request If all responses are received, a new thread and a queue (Queue) for the next level will be created after a delay of t. k The next scene block of quality level k is sent in parallel. `NewThread(InitLevel=k)`

[0056] Finally, if the perspective in the client changes, the order of requests in the queue will be readjusted to update the current request priority.

[0057] If the client loses connection with the server, it does not discard the current transmission state but retains the current BlockMap. After successfully reconnecting, the client restarts the queue layer by layer according to the scene block reception status recorded in the BlockMap, and initiates requests for scene blocks that have not yet been received.

[0058] In the above implementation steps, priority queues and BlockMaps are used to manage scene block transmission requests. The user's visual perception center is detected in real time, and the priority of requests in the queue is dynamically adjusted, prioritizing the transmission of these areas. This enables the ability to dynamically adjust the transmitted content based on the user's perspective, optimizing the user's visual experience in 3D space. This is particularly beneficial in frequently interactive applications, ensuring the timeliness and continuity of scene loading. A multi-threaded concurrency mechanism allows for the simultaneous transmission of high and low quality data of the central mesh and its surrounding meshes, avoiding transmission bottlenecks. A sliding window ensures timeout retransmission and ordered transmission of scene blocks in the queue. Furthermore, by adjusting the sliding window size, pipelined transmission within a single thread can be achieved. This multi-threaded, multi-queue sliding window transmission effectively improves the adaptability of the transmission strategy under different network conditions. Parallel transmission of scene blocks allows for full utilization of network bandwidth, accelerating scene loading speed, especially in high-frequency interactive scenarios, enabling users to obtain rendering results faster.

[0059] In summary, the prioritization strategy based on the user's visual perception center fully leverages scene layering capabilities and utilizes segmentation to achieve finer-grained network transmission and interaction processing. During transmission, the priority of scene block compression transmission is dynamically adjusted according to the client's visual perception center, achieving a user experience that prioritizes visual perception. In addition, it also possesses the following capabilities:

[0060] (1) Disconnection recovery capability: Through the BlockMap's saving and loading mechanism, the client can continue the previous transmission task after the disconnection is restored, reducing data transmission redundancy and duplicate requests.

[0061] (2) Sliding window optimized transmission: The client manages the orderly transmission of blocks through a sliding window. The window size can be expanded as needed to achieve pipelined transmission and make full use of network bandwidth.

[0062] (3) Multi-queue parallel transmission: Always ensure the priority of the user's vision, and the scene block in the visual center will be loaded with higher quality level first.

[0063] (III) Renderer

[0064] The renderer treats each mesh as an independent rendering unit. To ensure a smooth user experience under low bandwidth or resource constraints, the renderer can load LOD (Level of Detail) data of different resolutions in layers, from low-quality layers to high-quality layers. It first transmits and renders the low-resolution scene of the base layer, then progressively loads higher-resolution detail layers, achieving a gradual enhancement of the scene.

[0065] To adapt to scenarios with multi-quality level layered transmission and avoid abrupt transitions between different resolutions during visual rendering, the renderer disclosed herein performs quality level interpolation on the edges of the tiles to generate an adaptive, smooth transition effect. The interpolation algorithm smooths the transition between different LOD levels, allowing high-resolution details of the scene to load gradually and seamlessly integrate with the low-resolution base layer. This prevents abrupt visual jumps or distortions when the scene gradually loads from low to high resolution during real-time interaction, ensuring a smooth and natural rendering transition. Simultaneously, tile processing and interpolation optimization enable low-quality scenes to gradually transition to high quality, reducing the burden on bandwidth and computing resources.

[0066] Specifically, when rendering at quality level k, each mesh takes into account the quality levels of its surrounding meshes. At the boundary of a mesh, if two adjacent meshes M... i and M j If the quality levels are different, a rendering transition band is set between these two adjacent meshes. The renderer generates a smoother transition effect by interpolating the transition band, which not only reduces visual abruptness but also significantly improves the naturalness of switching between different quality levels.

[0067] In one implementation, the grid M i The quality level is k, and the mesh size is M. i The quality level is k′. The interpolation process occurs in grid M. i and M j A transition zone is set between them. The transition zone is a defined area, for example, the area between grid M. iThe edge 20% region is set as a transition band, with 20% being an adjustable instance value. The transparency of the Gaussian elements in the transition band is then blended according to the interpolation coefficient t, which is an adjustable value, as shown in the following formula:

[0068] in, To improve the transparency of the transition zone high-ski element, For grid M i transparency, For grid M j Transparency.

[0069] In summary, compared to traditional layered progressive techniques that typically freeze low-resolution Hoskiy primitives and cannot dynamically adjust underlying primitives, leading to over-smoothing or aliasing in low-resolution layers, especially at scene edges where they easily get stuck in local optima, making it difficult for subsequent layers to effectively repair these details, this disclosure introduces a non-fixed optimization mechanism that allows dynamic adjustment of the geometric parameters of low-level edge Hoskiy primitives. This optimizes high-frequency details of the scene, particularly the performance of edge regions, significantly reducing the generation of redundant Hoskiy primitives and improving overall scene performance and compression efficiency. Furthermore, traditional layered progressive techniques cannot dynamically adjust transmission priority based on the user's visual perception center, resulting in all regions being transmitted and loaded in the same order, making it difficult to prioritize the scene loading experience of the user's visually focused areas under limited bandwidth. This disclosure introduces a visual perception-priority slice compression transmission strategy, which dynamically divides the scene according to the user's real-time visual perception center, prioritizing the transmission of the core areas of user focus. This ensures that the area where the user's visual perception center is located is transmitted and rendered with the highest quality under bandwidth constraints, improving the user experience while reducing the transmission burden on secondary areas and improving network resource utilization efficiency. Compared to traditional layered progressive rendering techniques, which often produce visual discontinuities when switching between different quality levels and fail to achieve a smooth, continuous visual transition, making users easily perceive abrupt changes in image quality during the transition, this disclosure addresses this issue. Through the design of an adaptive block-based smooth renderer, a linear interpolation method is used to generate smooth transitions, resulting in more natural transitions between different resolution levels and avoiding visual discontinuities.

[0070] The above technical solution has undergone multiple simulation tests, and the experiments have proven that it can meet the expected design indicators, the 3D scene construction effect is good, and it can be presented in real time.

[0071] Through the above description of the embodiments, those skilled in the art can clearly understand that a corresponding system can be implemented according to the method of this disclosure. For example, a real-time interactive 3D Gaussian sputtering scene rendering system is implemented according to the method of this disclosure. The system includes a client, a server, and a network module. The server constructs a 3D Gaussian sputtering scene using a non-fixed layered progressive construction method and segments the constructed 3D Gaussian sputtering scene to generate a mesh. The client projects the mesh vertices onto a 2D screen and determines the center of the rendering screen based on the real-time acquired current user visual perception center. Based on the distance between the center of the rendering screen and the center point of the mesh projection, the client determines or adjusts the transmission request priority of each scene block under the current user visual perception center. After receiving the scene blocks transmitted by the server, the client uses a renderer to progressively enhance the 3D scene by loading layers from low-quality to high-quality layers. The network module sends the corresponding transmission request for the next layer to the server after receiving a response for the corresponding transmission request for each layer from low-quality to high-quality layers. The non-fixed layered progressive construction method, based on the layered progressive construction method, allows the current layer to update the transparency and scale of Gaussian elements in the scene of the previous layer during the training process.

[0072] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods or systems disclosed herein can be implemented using software plus necessary general-purpose hardware, or they can be implemented using dedicated hardware including dedicated integrated circuits, dedicated CPUs, dedicated memory, dedicated components, etc. Generally, any function performed by a computer program can be easily implemented using corresponding hardware, and the specific hardware structure used to implement the same function can be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the purposes of this disclosure, software program implementation is more often a preferred implementation method.

Claims

1. A real-time interactive 3D Gaussian sputtering scene rendering method, characterized in that, The method includes the following steps: On the client side, the mesh vertices are projected onto the 2D screen, and the center of the rendered screen is determined based on the real-time acquisition of the current user's visual perception center. Based on the distance between the center of the rendered image and the center of the grid projection, the client determines or adjusts the transmission request priority of each scene block under the current user's visual perception center. From low-quality layer to high-quality layer, after the client receives a response to the corresponding transmission request at one layer, it sends the corresponding transmission request for the next layer to the server. After receiving the scene blocks compressed and transmitted from the server, the client uses a renderer to progressively enhance the 3D scene by loading layers from low quality to high quality. The mesh is generated by dividing the layered three-dimensional Gaussian sputtering scene into blocks in both the vertical and horizontal directions.

2. The method according to claim 1, characterized in that: On the client side, a BlockMap is used to record the highest level of scene blocks that have been synchronized across different grids; When the client and server disconnect and reconnect successfully, the client initiates transmission requests to the server for the scene blocks that have not been received, layer by layer, according to the records in the BlockMap.

3. The method according to claim 1, characterized in that, The priority of transmission requests is determined as follows: requests for scene blocks that include the center of the rendered screen are given the first priority, and requests for scene blocks that do not include the center of the rendered screen are given the second priority based on their distance from the center point of the grid projection, from closest to farthest.

4. The method according to claim 1, characterized in that, At the grid boundary, if two adjacent grids have different quality levels, the transparency of the transition zone is determined by interpolation by setting a transition zone between the two grids.

5. The method according to claim 1, characterized in that, The 3D Gaussian sputtering scene is constructed using a non-fixed layered progressive construction method. This non-fixed layered progressive construction method, based on the layered progressive construction method, allows the current layer to update the transparency and scale of Gaussian elements in the previous layer's scene during training.

6. The method according to claim 5, characterized in that, Set a scale threshold and a transparency threshold so that the ratio of the updated size to the original scale does not exceed the scale threshold. If the transparency of a Gaussian cell after the update is lower than the transparency threshold, then the Gaussian cell is deleted.

7. The method according to claim 1, characterized in that, Transmission requests are stored in a queue, and a sliding queue window is used to ensure timeout retransmission and orderly transmission. The size of the sliding queue window is adjustable, and a multi-threaded concurrency mechanism is used to transmit 3D Gaussian data of the scene blocks at the center of the rendered screen and its surrounding areas simultaneously.

8. The method according to claim 1, characterized in that, At the start of rendering, the client obtains metadata about the 3D Gaussian sputtering scene from the server, including the total scene size, supported quality levels, scene tiling method, number of scene tilings, and compression standard used.

9. A computer-readable storage medium, characterized in that: The computer program is stored that can be loaded by a processor and executed according to any one of claims 1 to 8.

10. A real-time interactive 3D Gaussian sputtering scene rendering system, characterized in that, The system includes a client, a server, and a network module; The server uses a non-fixed layered progressive construction method to construct a three-dimensional Gaussian sputtering scene, and then segments the constructed three-dimensional Gaussian sputtering scene to generate a mesh. The client projects the mesh vertices onto a two-dimensional screen, based on real-time acquired data. The current user's visual perception center is used to determine the center of the rendered screen; based on the distance between the center of the rendered screen and the center point of the mesh projection, the transmission request priority of each scene block under the current user's visual perception center is determined or adjusted. After receiving the scene blocks transmitted from the server, the renderer loads them layer by layer from low quality to high quality, progressively enhancing the 3D scene. The network module, from low-quality layer to high-quality layer, sends the corresponding transmission request of the next layer to the server after receiving a response to the corresponding transmission request of the next layer. The non-fixed layered progressive construction method, based on the layered progressive construction method, allows the current layer to update the transparency and scale of Gaussian elements in the scene of the previous layer during the training process.

Images