Real-time interactive three-dimensional Gaussian sputtering scene rendering method, storage medium and system

Abstract

The invention relates to a real-time interactive three-dimensional Gaussian sputtering scene data rendering method, and the method carries out the longitudinal and transverse block splitting of a three-dimensional Gaussian sputtering scene based on a non-curing progressive construction technology, so as to optimize the streaming media loading performance. Through scene block compression and a priority transmission loading technology based on a user visual perception center, the data transmission efficiency and the online real-time rendering efficiency of the three-dimensional Gaussian sputtering scene can be effectively improved even if the bandwidth is limited, the real-time interaction requirement can be met even in a network fluctuation environment, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to computer technology, and in particular to a real-time interactive three-dimensional Gaussian sputtering scene data rendering method, storage medium and system. Background Art

[0002] Three-dimensional Gaussian sputtering technology performs well in high-fidelity rendering, but generates a large amount of scene data. Especially when network bandwidth is limited and device rendering capabilities vary, transmitting this data to users through streaming media poses severe challenges.

[0003] In order to improve the rendering and transmission efficiency of 3D scenes, the existing technology adopts a hierarchical progressive construction method, which constructs a hierarchical structure for stacking representation. This hierarchical structure allows the loading of scenes of different quality levels layer by layer according to network conditions in real-time network interaction and streaming media scenarios, and gradually refines the 3D scene representation to achieve dynamic adaptive transmission efficiency and viewing quality. However, this method still has the following defects.

[0004] First, this method mainly relies on accumulating Gaussian primitives layer by layer to add details, thereby passively reducing the number of Gaussian primitives in the entire scene, but fails to fully achieve efficient compression. This makes the overall data volume of the constructed scene still large, limiting the transmission efficiency in a low-bandwidth network environment. If you rely on external programs to compress and decompress the scenes transmitted layer by layer, the large amount of data divided by layers will cause delays in screen rendering, resulting in a poor user experience.

[0005] Secondly, each layer of the hierarchical structure is constructed independently. In the process of accumulation, the upper enhancement layer is only responsible for adding additional details on top of the existing base layer, but cannot optimize or adjust other properties of the lower Gaussian primitives except transparency. This design that lacks interaction between layers can easily cause the scene representation to fall into a local optimum, that is, each layer only optimizes its own Gaussian primitives, but cannot make comprehensive adjustments based on the global scene. This local optimum problem not only affects the compression efficiency, but also leads to redundancy in data representation, and cannot fully exploit the global characteristics of the scene to achieve more efficient compression.

[0006] Finally, its layer-by-layer transmission method ignores the user's attention to specific areas in the 3D space and fails to dynamically prioritize the transmission of detailed information in the center of the user's visual perception. As a result, the system wastes bandwidth when transmitting a large amount of data in low-priority areas. Such a design not only leads to unnecessary transmission overhead, but also reduces the loading efficiency in real-time interactive scenarios and cannot meet the needs of dynamic adjustment. Summary of the invention

[0007] Based on this, the purpose of the present disclosure is to propose a construction and transmission method, storage medium and system for a three-dimensional Gaussian sputtering scene for real-time interaction. The method splits the three-dimensional Gaussian sputtering scene into blocks vertically and horizontally based on the non-solidified progressive construction technology to optimize the streaming media loading performance, and then through the scene block compression technology and the priority transmission loading technology based on the user's visual perception center, the data transmission efficiency and online real-time rendering efficiency of the three-dimensional Gaussian sputtering scene can be effectively improved even if the bandwidth is limited, and even in a network fluctuation environment, it can meet the real-time interaction needs and improve the user experience. The specific technical solution is as follows.

[0008] In a first aspect, the present disclosure proposes a real-time interactive three-dimensional Gaussian sputtering scene rendering method, the method comprising the following steps: at the client, mesh vertices are projected onto a two-dimensional screen, and the center of the rendering screen is determined based on the current user visual perception center obtained in real time; according to the distance between the center of the rendering screen and the center point of the mesh projection, the client determines or adjusts the transmission request priority of each layer of scene blocks under the current user visual perception center; from low-quality layer to high-quality layer, after a corresponding transmission request of one layer is responded to, the client sends a corresponding transmission request of the next layer to the server; after receiving the scene blocks compressed and transmitted by the server, the client uses a renderer to load them in layers from the low-quality layer to the high-quality layer, and progressively enhances the three-dimensional scene; wherein, the mesh is generated by vertically and horizontally block-splitting the hierarchical three-dimensional Gaussian sputtering scene.

[0009] In one implementation of the above technical solution, the client uses BlockMap to record the highest level of synchronized scene blocks under different grids; when the client is disconnected from the server and reconnects successfully, the client initiates transmission requests to the server for the unreceived scene blocks layer by layer according to the records of BlockMap.

[0010] In one implementation of the above technical solution, the priority of the transmission request is determined as follows: the request for the scene block including the center of the rendering screen is taken as the first priority, and for the scene block not including the center of the rendering screen, its request is taken as the second priority according to its distance from the center point of the grid projection from near to far.

[0011] In one implementation of the above technical solution, at the mesh boundary, if the quality levels of two adjacent meshes are different, a transition zone is set between the two meshes, and the transparency of the rendering of the transition zone is determined by an interpolation method.

[0012] In one implementation of the above technical solution, a three-dimensional Gaussian sputtering scene is constructed using a non-solidified layered progressive construction method. The non-solidified layered progressive construction method, based on the layered progressive construction method, allows the current layer to update the transparency and scale of the Gaussian primitives in the scene of the previous layer during the training process.

[0013] In one implementation of the above technical solution, a scale threshold and a transparency threshold are set so that the ratio of the updated size to the original scale does not exceed the scale threshold, and if the updated transparency of a Gaussian primitive is lower than the transparency threshold, the Gaussian primitive is deleted.

[0014] In one implementation of the above technical solution, the transmission request is stored in a queue, and a sliding queue window is used to ensure timeout retransmission and orderly transmission. The sliding queue window size is adjustable, and a multi-threaded concurrent mechanism is used to simultaneously transmit 3D Gaussian data of each layer of scene blocks corresponding to the center position of the rendering picture and its surroundings.

[0015] In one implementation of the above technical solution, when rendering starts, the client obtains metadata of the three-dimensional Gaussian splattering scene from the server, including the total size of the scene, supported quality levels, scene segmentation method, number of scene segments, and compression standard used.

[0016] In a second aspect, the present disclosure proposes a computer-readable storage medium storing a computer program that can be loaded by a processor and execute any of the above methods.

[0017] In a third aspect, the present disclosure proposes a real-time interactive three-dimensional Gaussian sputtering scene rendering system, the system comprising a client, a server and a network module; the server adopts a non-solidified layered progressive construction method to construct a three-dimensional Gaussian sputtering scene, and divides the constructed three-dimensional Gaussian sputtering scene to generate a grid; the client projects the grid vertices onto a two-dimensional screen, and determines the center of the rendering screen based on the current user visual perception center obtained in real time; according to the distance between the center of the rendering screen and the center point of the grid projection, determines or adjusts the transmission request priority of each layer of scene blocks under the current user visual perception center; after receiving the scene blocks transmitted by the server, uses a renderer to load them in layers from low-quality layers to high-quality layers, and progressively enhances the three-dimensional scene; the network module, from low-quality layers to high-quality layers, after a corresponding transmission request of one layer is responded to, sends a corresponding transmission request of the next layer to the server; wherein, the non-solidified layered progressive construction method, based on the layered progressive construction method, allows the current layer to update the transparency and scale of the Gaussian primitives in the scene of the previous layer during the training process.

[0018] The beneficial technical effects of the present disclosure are as follows: by adopting a non-solidified layered progressive method to construct a three-dimensional Gaussian sputtering scene, introducing efficient compression technology and a dynamically adaptive transmission strategy, and adopting an adaptive block smoothing renderer, the transmission efficiency and rendering performance of the three-dimensional scene are significantly improved, and the application in virtual reality, augmented reality, cloud games, 3D streaming media, and online digital archive systems is effectively expanded, providing support for the fields of autonomous driving simulation, remote collaborative design, smart city modeling, and three-dimensional content generation. This technology will also promote the application of three-dimensional Gaussian sputtering technology to common consumer devices, such as mobile terminals and lightweight hardware devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 , one A schematic diagram of the system architecture in the embodiment of the present invention is shown in FIG. Figure 1 In the figure, 100 is a non-solidified layered progressive construction of a three-dimensional Gaussian sputtering scene, 200 is a grid block LOD layered scene, 300 is a transmission strategy management module, 400 is a user visual feedback, 500 is a byte stream, and 600 is a renderer.

[0021] Figure 2 , one Schematic diagram of a non-solidified layered progressive construction method in an embodiment.

[0022] Figure 3 , one Schematic diagram of a scene transmission strategy with visual perception priority in an implementation manner. DETAILED DESCRIPTION

[0023] The relevant terms are defined below.

[0024] 3D Gaussian Splatting: 3D Gaussian Splatting is a technology used to represent and render 3D scenes. It uses Gaussian function primitives in 3D space to represent the scene's geometric structure and attribute information, mapping it into a series of Gaussian primitives with attributes such as position, color, and transparency. The attributes of each Gaussian primitive include position, size, color, opacity, rotation, etc. The combination of these attributes represents the object information in 3D space. Compared with traditional 3D mesh models, 3D Gaussian Splatting technology can build realistic complex scenes at a lower cost and equipment, and is particularly suitable for online browsing services for 3D digital collections.

[0025] Adaptive streaming (Structure from Motion): Adaptive streaming is a technology used to transmit audio and video content in a network environment. It can dynamically adjust the quality of streaming media based on network bandwidth, device performance, and current network conditions to ensure streaming continuity and the best user experience.

[0026] Level of Detail (LOD): It is a technology used in computer graphics to optimize 3D scene rendering and improve performance. By creating multiple models of different complexity for the same object, the system can dynamically select the appropriate model version for rendering based on factors such as viewing distance and screen resolution. LOD technology is widely used in scenes such as games, virtual reality, 3D modeling, and online 3D digital archive systems to significantly improve rendering frame rates and save computing resources.

[0027] See also Figure 1 The present solution uses a non-solidified layered progressive method to construct a three-dimensional Gaussian sputtering scene (100), and uses a grid block LOD layered scene (200) for the constructed three-dimensional Gaussian sputtering scene. Then, a transmission strategy management module (300) is used to compress the scene blocks and transmit the data to a renderer (600) in the form of a byte stream (500) using a priority strategy based on a user visual perception center. The user visual perception center in the priority strategy based on the user visual perception center can be expressed as (view, x, y), where x and y are the positions of the user visual perception center and view is the viewing angle.

[0028] The following will be combined with the attached Figure 2-3 , a clear and complete description is given on how to implement the technical solution of this case. Obviously, the described implementation method is only a part of the implementation method of this case, not all of the implementation methods.

[0029] (I) Construction method

[0030] The traditional layer-by-layer progressive 3D Gaussian sputtering algorithm only constructs details based on the accumulation of Gaussian primitives layer by layer. Since the properties of the underlying Gaussian primitives cannot be adjusted except for transparency, the model is easily trapped in the local optimum and cannot fully express high-frequency details. In addition, there is a lack of interactive optimization between layers, which easily generates redundant Gaussian primitives.

[0031] This solution uses a method of constructing a three-dimensional Gaussian sputtering scene in a non-solidified layered progressive manner. The method introduces a local dynamic optimization mechanism in the layered structure, allowing the underlying primitives, especially the edge areas, to be refined and adjusted, thereby effectively tuning the high-frequency details in the scene and avoiding excessive reliance on subsequent layers for refinement, which is conducive to improving the overall performance of the scene from a global perspective. At the same time, this adjustment mechanism also reduces the generation of redundant Gaussian primitives in the dense refinement process of subsequent layers, significantly improving the data compression rate and global efficiency, and achieving a balance between efficient compression and details.

[0032] First, based on the input multi-view image set {Ii}, a multi-resolution image set version D is constructed. According to the hierarchical quality level 0 to N, N is the highest quality level, each image is downsampled to obtain an image pyramid with resolution from small to large. k∈{0, 1, ..., N}, where Represents the k-th quality level image of the ith viewing angle.

[0033] Build the scene starting from the lowest quality level, level = 0, using the resolution image set of the corresponding quality level As input, the distribution of an initial set of Gaussian primitives is optimized to form the scene G0.

[0034]

[0035] in, It is the optimized scene base layer. It is the basic layer to be optimized during training. Indicates the process of rendering the base layer to be optimized into a 2D image.

[0036] represents the i-th real input image at resolution level 0, M represents the number of viewing angles of the input image. arg represents the function The variable with the minimum value

[0037] As training progresses, each layer uses higher resolution views k∈{0, 1, ..., N} as input to train the enhancement layer Enhancement Layer A scene with a quality level of frying Combined to form a new quality level scene This stacking construction method makes full use of the structural consistency between L0D scene models. The stacking formula for the scene is as follows:

[0038]

[0039] Base Layer Represents the lowest quality 3D Gaussian sputtering model, which contains the basic geometry and attribute information of the scene. The resolution and detail of the scene are further improved by incrementally optimizing the Gaussian spots. Middle comprehensive pre-order level and add new details to it. Figure 2 Hint.

[0040] In the subsequent training process of each enhancement layer, not only all the properties of the Gaussian primitives of the current enhancement layer are optimized, but also the properties of the Gaussian primitives of the previous layer are partially frozen. The transparency and scale of the Gaussian primitives in the previous layer are allowed to be updated. If the transparency of the updated Gaussian primitive is lower than the transparency threshold, the Gaussian primitive is deleted, thereby reducing the number of redundant Gaussian primitives. If the update of the scale of the Gaussian primitives of the previous layer does not exceed 80% of its original scale, the destruction of the global consistency of the low-quality scene can be avoided. 80% is an instance value of a scale threshold set and can be adjusted.

[0041]

[0042] in, is the globally optimized enhancement layer. is the previous layer after optimizing only the transparency and scale of the high primitives, is the enhancement layer to be optimized, is the previous layer to be optimized. It represents the process of merging the scenes to be optimized and rendering them into 2D images. represents the i-th real input image at resolution level k, and M represents the number of input images.

[0043] Since traditional algorithms use multi-level resolution images to train the base layer, synthesizing low-resolution images will inevitably introduce problems of over-smoothing or artificial aliasing, resulting in the underlying Gaussian primitives being insensitive to image edge details during the modeling process.

[0044] By dynamically adjusting the size of the Gaussian primitives in the previous layer during the layering process, the common edge damage problem in the layer-by-layer training of multi-level resolution images is effectively solved. At the same time, freezing the update of other parameters also avoids the risk of model expansion caused by the splitting of the underlying Gaussian primitives. This optimization strategy not only retains the global description ability of the underlying primitives, but also improves the detail enhancement and repair effects of subsequent layers, reduces the generation of redundant Gaussian primitives, and ensures that the model can be refined layer by layer without losing overall accuracy, thereby significantly improving the final modeling effect.

[0045] (II) Compression transmission strategy

[0046] This solution adopts a priority strategy based on the user's visual perception center, which is the core transmission mechanism designed in this disclosure to enhance the real-time interactive experience and optimize bandwidth utilization.

[0047] This strategy divides the three-dimensional space into multiple grids, and combines the layering of the scene itself to form scene slices that are split vertically and horizontally. These scene slices form fine-grained network transmission units to achieve data compression processing at the grid granularity to ensure efficient transmission and decoding processes. At the same time, this strategy supports visual perception strategies, allowing the system to transmit according to the priority of different grids, giving priority to the spatial block where the user's visual center is located, further improving resource utilization efficiency and response speed. Specifically, by analyzing the user's perspective and interactive behavior in the 3D scene, the user's current user visual perception center is identified. According to the priority of this focus area, the scene data is sliced ​​into multiple independent grids, and each grid is prioritized according to its importance to ensure that users can obtain high-quality rendering content of the part of interest as soon as possible, while low-priority background areas or non-focus areas are transmitted at a lower resolution or loaded step by step when bandwidth is sufficient.

[0048] It can be seen that this strategy makes full use of the scene layering capability and uses slicing to achieve finer-grained network transmission interaction processing. It also dynamically senses the area where the user's visual perception center is located in the three-dimensional scene, and prioritizes the transmission of important details in these areas to ensure that users can quickly obtain high-quality rendering content during real-time interaction, while reducing the waste of bandwidth and computing resources in non-key areas. In the transmission process, the priority of scene block transmission is dynamically adjusted according to the client's visual center to achieve a user experience with visual perception priority.

[0049] Therefore, this transmission strategy can improve transmission efficiency when bandwidth is limited, ensure that the core visual area that users are concerned about can be loaded with the highest quality first, reduce the transmission burden on secondary areas, and achieve efficient use of network resources.

[0050] In one implementation, a renderer client that implements a priority strategy based on the user's visual perception center dynamically evaluates the priority of the queue of slices to be transmitted by tracking the user's perspective and position. The area where the user's visual perception center is located in the three-dimensional scene is detected, and then the priority of the scene block to be requested for transmission is determined to ensure that the most important visual content can be compressed and transmitted with high quality first. That is, the closer the grid is to the user's visual perception center, the more important it is, and the higher the transmission priority and quality level are assigned. Grids that are farther away from the user's visual perception center or at the edge of the viewing angle will be assigned a lower priority. The following is combined with Figure 3 The implementation steps are detailed as follows.

[0051] First, the three-dimensional Gaussian sputtering scene g is segmented into blocks in the vertical and horizontal directions to generate several square grids Mi 。 Due to the hierarchical structure of the scene, each mesh can be considered as a scene block with multiple quality levels. k∈{0, 1, ..., N}, k represents the quality level, N represents the highest quality level, and i represents the grid position number. The server uses lossless compression technology to preprocess the scene blocks to save bandwidth in the subsequent transmission process.

[0052] When rendering starts, the client communicates with the server and obtains the meta information of the scene g from the server, including: the total size of the scene, the supported quality levels, the scene segmentation method, the number of scene segments, and the compression standard used.

[0053] Next, the client initializes BlockMap and the sliding window queue Queue. The BlockMap structure is used to record the quality level of the received scene blocks, indicating the data status that the client has received. For the received scene blocks, the highest quality level of the current position is recorded 0≤level≤N. For the grid positions where the blocks have not been received, the unreceived flag is set, such as using "-1" to indicate unreceived. In one embodiment, the BlockMap is initialized using the unreceived flag.

[0054]

[0055] The sliding window queue is a queue used to manage network transmission requests and responses. The scene block transmission request that the client is about to initiate to the server will be stored in the queue according to priority, and a separate network transmission thread will manage the request sending and scene block receiving.

[0056] At this point, the client generates all blocks at layer 0 Transfer request Put it in queue Queue0 中 In one embodiment, the vertices of the mesh division are projected onto the two-dimensional screen through the current user visual perception center, and the transmission request is Divide into two parts according to the conditions to implement priority sorting to save computing resources: the center position of the current rendering interface Scene block transfer request As the first priority, it is directly inserted into the beginning of the queue Queue0, and other requests are based on the center point of the grid projection and the center point of the screen. The distance is sorted as the second priority.

[0057]

[0058] The client then sends requests in order If a response is received from the server, the queue window is slid and the next request is sent.

[0059] WindowSlide(Queue0, i)

[0060] If the client sends a First priority scene block request If all responses are received, a new thread and a queue for the next layer will be created after a delay of t. k , and send the scene blocks of the next quality level k in parallel.

[0061] NewThread(InitLevel=k)

[0062] Finally, if the perspective in the client changes, the order of requests in the queue is re-adjusted to update the current request priority.

[0063]

[0064] If the client is disconnected from the server, the client does not discard the current transmission status, but retains the current BlockMap. After reconnecting successfully, the client restarts the queue layer by layer according to the different levels of scene block reception status recorded in BlockMap, and initiates requests for scene blocks that have not yet been received.

[0065] In the above implementation steps, the transmission request of the scene block is managed by the priority queue and BlockMap, the user's visual perception center is detected in real time, and the priority of the request in the queue is dynamically adjusted. These areas are transmitted first, so that the ability to dynamically adjust the transmission content according to the user's perspective can be realized, and the user's visual experience in three-dimensional space can be optimized, especially in applications with frequent interactions, ensuring the timeliness and consistency of scene loading. The high and low quality level data of the central grid and its surrounding grids are transmitted simultaneously through the multi-threaded concurrent mechanism to avoid the bottleneck of transmission waiting. The sliding window ensures the timeout retransmission and orderly transmission of the scene blocks in the queue. Furthermore, by adjusting the size of the sliding window, pipeline transmission within a single thread can be realized. This sliding window transmission based on multi-threaded and multi-queue can effectively improve the adaptability of the transmission strategy under different network conditions. The parallel transmission of scene blocks enables full use of network bandwidth; it speeds up the loading speed of the scene, especially in scenes with high-frequency interactions, so that users can obtain rendering results faster.

[0066] In summary, the priority strategy based on the user's visual perception center fully utilizes the scene layering capability and uses block segmentation to achieve more fine-grained network transmission interaction processing. During the transmission process, the priority of scene block compression transmission is dynamically adjusted according to the client's visual center to achieve a user experience that prioritizes visual perception. In addition, it also has the following capabilities:

[0067] (1) Disconnection recovery capability: Through the saving and loading mechanism of BlockMap, the client can continue the previous transmission task after the connection is restored, reducing the redundancy and repeated requests of data transmission.

[0068] (2) Sliding window optimized transmission: The client manages the orderly transmission of blocks through a sliding window. The window size can be expanded as needed to achieve pipeline transmission and fully utilize network bandwidth.

[0069] (3) Multi-queue parallel transmission: The user’s visual priority is always guaranteed, and the scene blocks at the visual center will be loaded with higher quality levels first.

[0070] (III) Renderer

[0071] The renderer treats each mesh as an independent rendering unit. To ensure a smooth user experience in low-bandwidth or resource-constrained environments, the renderer can load L0D (Level of Detail) data of different resolutions from low-quality layers to high-quality layers. First, the low-resolution scene of the base layer is transmitted and rendered, and then higher-resolution detail levels are gradually loaded to achieve progressive enhancement of the scene.

[0072] In order to adapt to the scenes of multi-quality hierarchical block transmission and avoid the sense of disconnection when switching between different resolutions during the visual presentation process, the renderer of the present disclosure interpolates the quality level of the block edges to generate an adaptive smooth transition effect. The interpolation algorithm is used to smooth the transition between different LOD levels, so that the high-resolution details of the scene can be gradually loaded and seamlessly integrated with the low-resolution base layer, so that the renderer avoids abrupt visual jumps or distortions when the scene is gradually loaded from low resolution to high resolution during real-time interaction, ensuring smooth and natural transition of rendering. At the same time, block processing and interpolation optimization enable low-quality scenes to gradually transition to high quality, reducing the burden on bandwidth and computing resources.

[0073] Specifically, when rendering each mesh at quality level k, the quality level of the surrounding meshes will be considered. At the boundary of the mesh, if two adjacent meshes M i and M j If the quality levels of the two meshes are different, a rendering transition zone is set between the two adjacent meshes. The renderer generates a smoother transition effect for the transition zone through interpolation, which not only reduces visual abruptness, but also significantly improves the naturalness of switching between different quality levels.

[0074] In one embodiment, the grid M i The quality level is k, the grid M j The quality level of is k′. The interpolation process, in the grid M i and M j A transition zone is set between the grid M and the grid M. i The 20% area at the edge is set as the transition zone, and 20% is an adjustable instance value. Then the transparency of the Gaussian primitives in the transition zone is mixed according to the interpolation coefficient t, which is an adjustable value. See the following formula:

[0075]

[0076] in, is the transparency of the Gaussian element in the transition zone, For the grid M i transparency, For the grid M j transparency.

[0077] In summary, compared with the traditional layered progressive technology, this solution usually freezes the high-resolution primitives of the low-resolution layer, and cannot dynamically adjust the underlying primitives, which leads to the low-resolution layer easily introducing over-smoothing or aliasing, especially the scene edge is easy to fall into the local optimum, resulting in the inability of subsequent layers to effectively repair these details. The present disclosure introduces a non-fixed optimization mechanism, which allows the geometric parameters of the low-level edge high-resolution primitives to be dynamically adjusted. In this way, the high-frequency details of the scene, especially the performance of the edge area, can be optimized, the generation of redundant high-resolution primitives can be significantly reduced, and the overall scene performance and compression efficiency can be improved. Compared with the traditional layered progressive technology, it is impossible to dynamically adjust the transmission priority according to the user's visual perception center, resulting in the same transmission and loading order of all areas, and it is difficult to give priority to the scene loading experience of the user's visual attention area under limited bandwidth. By introducing a slice compression transmission strategy with visual perception priority, the present disclosure can dynamically divide the scene according to the user's real-time visual perception center, and give priority to the transmission of the core area of ​​the user's attention. In this way, when the bandwidth is limited, the area where the user's visual perception center is located can be guaranteed to be transmitted and rendered with the highest quality, which improves the user experience, while reducing the transmission burden on the secondary area and improving the utilization efficiency of network resources. Compared with the traditional layered progressive technology, when switching between different quality levels, it is easy to produce a sense of visual fragmentation and cannot achieve a continuous and smooth visual transition. Users are likely to perceive a sudden change in picture quality during the switching process. The present disclosure uses a linear interpolation method to generate a smooth transition through the design of an adaptive block smoothing renderer, making the switching between different resolution levels more natural and avoiding the sense of picture fragmentation.

[0078] The above technical solution has been subjected to multiple simulation tests, and experiments have proved that it can meet the expected design indicators, the three-dimensional scene construction effect is good, and it can be presented in real time.

[0079] Through the description of the above embodiments, those skilled in the art can clearly understand that a corresponding system can be implemented according to the method disclosed in the present invention. Exemplarily, according to the disclosed method, a real-time interactive three-dimensional Gaussian sputtering scene rendering system is implemented, the system comprising a client, a server and a network module; the server adopts a non-solidified layered progressive construction method to construct a three-dimensional Gaussian sputtering scene, and divides the constructed three-dimensional Gaussian sputtering scene to generate a grid; the client projects the grid vertices onto a two-dimensional screen, and determines the center of the rendering screen based on the current user visual perception center obtained in real time; according to the distance between the center of the rendering screen and the center point of the grid projection, the transmission request priority of each layer of scene blocks under the current user visual perception center is determined or adjusted; after receiving the scene blocks transmitted by the server, a renderer is used to load them in layers from a low-quality layer to a high-quality layer, and the three-dimensional scene is progressively enhanced; the network module, from a low-quality layer to a high-quality layer, sends a corresponding transmission request for the next layer to the server after a corresponding transmission request for one layer is responded to; wherein, the non-solidified layered progressive construction method, based on the layered progressive construction method, allows the current layer to update the transparency and scale of the Gaussian primitives in the scene of the previous layer during the training process.

[0080] Through the description of the above implementation modes, those skilled in the art can clearly understand that the method or system of the present disclosure can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits, etc. However, for the present disclosure, software program implementation is a better implementation mode in most cases.

Claims

1. A real-time interactive three-dimensional Gaussian sputtering scene rendering method, characterized in that: The method comprises the following steps: On the client side, mesh vertices are projected onto a two-dimensional screen, and the center of the rendering screen is determined based on the current user visual perception center obtained in real time; According to the distance between the center of the rendering image and the center of the grid projection, the client determines or adjusts the transmission request priority of each layer of scene blocks under the current user visual perception center; From low-quality layer to high-quality layer, after a corresponding transmission request of one layer is responded, the client sends a corresponding transmission request of the next layer to the server; After receiving the scene blocks compressed and transmitted by the server, the client uses a renderer to load the blocks in layers from a low-quality layer to a high-quality layer, and progressively enhances the three-dimensional scene; The grid is generated by vertically and horizontally splitting a three-dimensional Gaussian sputtering scene with a hierarchical structure into blocks.

2. The method according to claim 1, characterized in that: Use BlockMap on the client to record the highest level of synchronized scene blocks in different grids; When the client is disconnected from the server and reconnects successfully, the client initiates transmission requests to the server for the unreceived scene blocks layer by layer according to the records of BlockMap.

3. The method according to claim 1, characterized in that The priority of transmission requests is determined as follows: the request for the scene block containing the center of the rendering screen is given the first priority, and for the scene block that does not include the center of the rendering screen, its request is given the second priority according to its distance from the center point of the grid projection from near to far.

4. The method according to claim 1, characterized in that At the mesh boundary, if the quality levels of two adjacent meshes are different, a transition zone is set between the two meshes, and the transparency of the transition zone rendering is determined by interpolation method.

5. The method according to claim 1, characterized in that The three-dimensional Gaussian sputtering scene is constructed by a non-solidified layered progressive construction method, which, based on the layered progressive construction method, allows the current layer to update the transparency and scale of the Gaussian primitives in the scene of the previous layer during the training process.

6. The method according to claim 5, characterized in that The scale threshold and transparency threshold are set so that the ratio of the updated size to the original scale does not exceed the scale threshold. If the transparency of a Gaussian primitive after update is lower than the transparency threshold, the Gaussian primitive is deleted.

7. The method according to claim 1, characterized in that The transmission request is stored in a queue, and a sliding queue window is used to ensure timeout retransmission and orderly transmission. The sliding queue window size is adjustable, and a multi-threaded concurrent mechanism is used to simultaneously transmit 3D Gaussian data of each layer of scene blocks corresponding to the center position of the rendering picture and its surroundings.

8. The method according to claim 1, characterized in that At the beginning of rendering, the client obtains the metadata of the 3D Gaussian splatter scene from the server, including the total size of the scene, the supported quality levels, the scene segmentation method, the number of scene segments, and the compression standard used.

9. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 8.

10. A real-time interactive three-dimensional Gaussian sputtering scene rendering system, characterized in that: The system includes a client, a server and a network module; The server constructs a three-dimensional Gaussian sputtering scene by using a non-solidified layered progressive construction method, and divides the constructed three-dimensional Gaussian sputtering scene into grids; The client projects the mesh vertices onto a two-dimensional screen, determines the center of the rendering screen based on the current user visual perception center acquired in real time, and determines or adjusts the transmission request priority of each layer of scene blocks under the current user visual perception center according to the distance between the rendering screen center and the mesh projection center point; After receiving the scene blocks transmitted by the server, a renderer is used to load the blocks in layers from a low-quality layer to a high-quality layer, thereby progressively enhancing the three-dimensional scene; The network module, from the low-quality layer to the high-quality layer, sends the corresponding transmission request of the next layer to the server after the corresponding transmission request of one layer is responded to; The non-solidified layered progressive construction method, based on the layered progressive construction method, allows the current layer to update the transparency and scale of the Gaussian primitives in the scene of the previous layer during the training process.

Images