3D GS model real-time processing method and device based on unreal engine

Abstract

The invention provides a 3DGS model real-time processing method and device based on an unreal engine, and relates to the technical field of three-dimensional modeling, and the method comprises the steps: writing 3DGS data into a partitioned memory pool structure; writing data in the partitioned memory pool structure into a memory buffer area, performing coordinate system alignment processing according to a world coordinate system of an unreal engine, and performing spatial partitioning on the aligned processing data according to a point cloud density and a region density difference coefficient; converting the data in each spatial block into a point cloud data structure compatible with the Niagara particle system, and completing data format conversion in a graphics processor by using a calculation shader to obtain second processing data; and based on the constructed multi-level resource pool management strategy and a runtime memory monitoring mechanism, performing rendering processing on the second processing data according to a track prediction result of the camera, and generating a rendering result under an unreal engine. The processing efficiency of three-dimensional modeling in the unreal engine can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional modeling, and in particular to a real-time processing method and device for a 3DGS model based on Unreal Engine. Background Art

[0002] 3D Gaussian Splatting (3DGS) is an emerging 3D scene representation and rendering technology. 3DGS models the scene using a set of 3D Gaussian functions, each representing a Gaussian ellipsoid with properties such as position, size, orientation, color, and opacity. 6 Efficient 3D scene rendering is achieved by rasterizing these Gaussian ellipsoids into images.

[0003] With the rapid advancement of computer vision and deep learning technologies, the reconstruction and understanding of three-dimensional scenes has become a core research direction in cutting-edge fields such as autonomous driving, augmented reality, and virtual reality. Highly complex applications such as vehicle-road cooperative systems and digital twin environments require processing massive amounts of heterogeneous three-dimensional data. Traditional 3D modeling methods still have significant limitations in terms of accuracy and effectiveness, especially in dynamically changing scenes, where efficient model loading and updating are difficult. This results in systems relying on repetitive processes, severely restricting both the smoothness of user interaction and the intelligence of system operations.

[0004] The current mainstream 3D modeling process typically relies on plugins within the Unreal Engine. This involves manually loading 3D model files in ply format within the editor to generate the corresponding Actors, and then repackaging the entire scene into an executable project before deployment. However, this approach exhibits significant disadvantages in scenarios with frequent model updates or dynamically changing data. Each model replacement requires repetitive editing and packaging steps, creating a complex and inefficient process. It also struggles to meet the demands of systems with extremely high real-time requirements, limiting its applicability in environments with real-time interaction and continuous iteration. Summary of the Invention

[0005] The present application provides a real-time processing method and device for 3DGS models based on Unreal Engine, which can improve the processing efficiency of three-dimensional modeling in Unreal Engine.

[0006] In a first aspect of the present application, a real-time processing method for a 3DGS model based on Unreal Engine is provided, the method comprising:

[0007] Read 3DGS model file and write 3DGS data into block memory pool structure;

[0008] After normalizing and compressing the 3DGS data in the block-type memory pool structure, the data is written into a memory buffer to obtain first processed data;

[0009] Performing coordinate system alignment processing on the first processed data according to the world coordinate system of the Unreal Engine, and spatially dividing the aligned processed data into blocks according to the point cloud density and the regional density difference coefficient;

[0010] Converting the processed data in each of the spatial blocks into a point cloud data structure compatible with the Niagara particle system, and completing the data format conversion using a compute shader on a graphics processor to obtain second processed data;

[0011] Based on the constructed multi-level resource pool management strategy and runtime memory monitoring mechanism, the second processed data is rendered according to the camera trajectory prediction result to generate the rendering result of the 3DGS model under the Unreal Engine.

[0012] Based on the above technical solution, preferably, the multi-level resource pool management strategy and runtime memory monitoring mechanism are constructed, and the second processed data is rendered according to the trajectory prediction result of the camera to generate the rendering result of the 3DGS model under the Unreal Engine, which specifically includes:

[0013] Performing a loading priority calculation on the second processed data, dynamically generating an Actor in the current scene based on the loading priority and attaching a Niagara component to the Actor, forming a rendering object, and loading the object into the world scene within the rendering frustum;

[0014] During the rendering process, based on the world space depth information in the Unreal Engine, the rendering resolution of the 3DGS model in different spatial blocks is dynamically adjusted, and fade-out transition particles are set for the area where the rendering resolution changes to complete the smooth switching between resolution levels, thereby realizing real-time loading and dynamic updating of the 3DGS model under the Unreal Engine.

[0015] Based on the above technical solution, preferably, the 3DGS data in the block-based memory pool structure is sequentially normalized and compressed, and then written into a memory buffer to obtain first processed data, specifically including:

[0016] Normalizing the color vector in the 3DGS data, converting each channel value of the color vector into a floating point number and quantizing it into a fixed-point integer of a preset precision to obtain a processed color vector;

[0017] Converting the covariance matrix into a scale parameter and a rotation parameter, wherein the scale parameter is converted using a logarithmic function and retains a preset accuracy, and the rotation parameter is quantized into an integer form by a spherical linear encoding function based on the relative rotation residual between the current rotation quaternion and the previous rotation quaternion;

[0018] The processed color vector, the scale parameter and the rotation parameter are respectively compressed using a multi-context entropy coding model constructed based on a tabular asymmetric digital system, and the compressed fixed-length data units are continuously written into a memory buffer of a double buffer structure to complete the generation of the first processed data.

[0019] Based on the above technical solution, preferably, the step of performing coordinate alignment processing on the first processed data according to the world coordinate system of the Unreal Engine and spatially dividing the aligned processed data according to the point cloud density and the regional density difference coefficient includes:

[0020] Calculating a difference coefficient between point cloud density and regional density based on the spatial distribution of the first processed data, wherein the regional density difference coefficient is a ratio of a standard deviation of the point cloud density to a mean of the point cloud density;

[0021] When the regional density difference coefficient is less than or equal to the preset threshold, the quadtree algorithm is used for spatial block division; when the regional density difference coefficient is greater than the preset threshold, the KD tree algorithm is used for spatial block division;

[0022] During the spatial block process, an overlapping boundary area of ​​a preset length ratio of the adjacent boundaries is set between each adjacent spatial block, and corresponding spatial index information is generated for each spatial block, the spatial index information including a bounding box range, an offset address in a memory buffer, and a compressed data length;

[0023] The spatial blocks are sequentially encoded based on a Morton encoding method.

[0024] On the basis of the above technical solution, preferably, the step of calculating the difference coefficient between the point cloud density and the regional density based on the spatial distribution of the first processed data specifically includes:

[0025] Divide the spatial range of the entire 3DGS model into a number of regular spatial voxel units, and configure the side length of each of the spatial voxel units according to a preset resolution;

[0026] For each of the spatial voxel units, counting the amount of 3DGS data contained therein;

[0027] Determining the point cloud density according to the number of points within a unit volume;

[0028] The difference coefficient is calculated based on the point cloud density and the area density, and is calculated by the following formula

[0029]

[0030] Among them, CV d is the coefficient of variation, N i is the data quantity, n is the number of the spatial voxel units, and ΔL is the side length of the spatial voxel unit.

[0031] Based on the above technical solution, preferably, the processing data in each of the spatial blocks is converted into a point cloud data structure compatible with the Niagara particle system, and the data format conversion is completed using a compute shader on a graphics processor to obtain the second processed data, specifically including:

[0032] Decoding the compressed 3DGS data corresponding to each of the spatial blocks, the decoding process comprising decoding the position vector using a Zstandard decompression algorithm, and decoding the processed color vector, the scale parameter, and the rotation parameter using a tabular asymmetric digital system entropy decoding algorithm to reconstruct 3DGS model structure data including the spatial position vector, the normal vector, the processed color vector, the scale parameter, and the rotation parameter;

[0033] Mapping each piece of the 3DGS model structure data to a Niagara particle unit, wherein the mapping fields obtained after mapping include particle position attributes, particle orientation vectors, particle color parameters, particle scaling parameters, and particle posture rotation parameters;

[0034] A compute shader is used in a graphics processor to perform parallel structural rearrangement on the mapping fields and generate a Niagara particle data buffer structure in a structured buffer format. The particle data buffer structure is then injected into a corresponding Niagara system instance through a Niagara system interface to complete particle attribute initialization and binding, thereby achieving real-time conversion of the processed data into a point cloud data structure compatible with the Niagara particle system.

[0035] Based on the above technical solution, preferably, the processing data in each of the spatial blocks is converted into a point cloud data structure compatible with the Niagara particle system, and a compute shader is used on a graphics processor to complete the data format conversion to obtain the second processed data, which specifically further includes:

[0036] Decoding the compressed 3DGS data corresponding to each of the spatial blocks, and uploading the decoded 3DGS data to the graphics processor in the form of a structured buffer, wherein the data unit of each structured buffer includes a spatial position vector, a normal vector, the processed color vector, the scale parameter, and the rotation parameter;

[0037] Establishing a parallel thread grid by scheduling a compute shader, so that each parallel thread grid processes one data unit, mapping the 3DGS data field into a particle position field, a particle orientation field, a particle color field, a particle scale field, and a particle pose field of a Niagara particle system in each parallel thread grid, performing normalization processing on the rotation parameters, and generating a unique index code;

[0038] The mapped particle attribute data is written into an output structured buffer in the format required by the Niagara particle system. After all the parallel thread grids are executed, the output structured buffer is bound to the Niagara particle system instance through the rendering interface function of the Niagara system as its input data source to complete the initialization and rendering scheduling of the particle life cycle, thereby generating the second processed data.

[0039] In a second aspect of the present application, a real-time processing device for a 3DGS model based on Unreal Engine is provided. The device is used to execute any of the above-described real-time processing methods for a 3DGS model based on Unreal Engine. The device includes an acquisition module, a processing module, and an output module, wherein:

[0040] The acquisition module is used to read the 3DGS model file and write the 3DGS data into the block-type memory pool structure;

[0041] The processing module is configured to perform normalization processing and compression processing on the 3DGS data in the block-type memory pool structure, and then write the data into a memory buffer to obtain first processed data;

[0042] The processing module is configured to perform coordinate system alignment processing on the first processed data according to the world coordinate system of the Unreal Engine, and to spatially divide the aligned processed data into blocks according to the point cloud density and the regional density difference coefficient;

[0043] The processing module is configured to convert the processed data in each of the spatial blocks into a point cloud data structure compatible with the Niagara particle system, and perform data format conversion using a compute shader on a graphics processor to obtain second processed data;

[0044] The output module is used to render the second processed data according to the camera trajectory prediction result based on the constructed multi-level resource pool management strategy and runtime memory monitoring mechanism, and generate the rendering result of the 3DGS model under the Unreal Engine.

[0045] Based on the above technical solution, preferably, the output module is used to calculate the loading priority of the second processed data, dynamically generate an Actor in the current scene according to the loading priority, attach a Niagara component, constitute a rendering object, and load it into the world scene within the rendering frustum;

[0046] The processing module is used to dynamically adjust the rendering resolution of the 3DGS model in different spatial blocks based on the world space depth information in the Unreal Engine during the rendering process, and set fading transition particles for the area where the rendering resolution changes to complete the smooth switching between resolution levels, thereby realizing real-time loading and dynamic updating of the 3DGS model in the Unreal Engine.

[0047] Based on the above technical solution, preferably, the processing module is used to perform normalization processing on the color vector in the 3DGS data, convert each channel value of the color vector into a floating point number and quantize it into a fixed-point integer of a preset precision to obtain a processed color vector;

[0048] The processing module is used to convert the covariance matrix into a scale parameter and a rotation parameter, wherein the scale parameter is converted using a logarithmic function and retains a preset accuracy, and the rotation parameter is quantized into an integer form by a spherical linear encoding function based on the relative rotation residual between the current rotation quaternion and the previous rotation quaternion;

[0049] The processing module is used to compress the processed color vector, the scale parameter and the rotation parameter using a multi-context entropy coding model constructed based on a tabular asymmetric digital system, and continuously write the compressed fixed-length data units into a memory buffer of a double buffer structure to complete the generation of the first processed data.

[0050] On the basis of the above technical solution, preferably, the processing module is used to calculate the difference coefficient between the point cloud density and the regional density based on the spatial distribution of the first processed data, and the regional density difference coefficient is the ratio of the point cloud density standard deviation to the point cloud density mean;

[0051] The processing module is configured to use a quadtree algorithm to perform spatial segmentation when the regional density difference coefficient is less than or equal to a preset threshold, and to use a KD tree algorithm to perform spatial segmentation when the regional density difference coefficient is greater than the preset threshold;

[0052] The processing module is configured to, during the spatial segmentation process, set an overlapping boundary region between each adjacent spatial segment, which has a predetermined length of the adjacent boundary, and generate corresponding spatial index information for each spatial segment, wherein the spatial index information includes a bounding box range, an offset address in a memory buffer, and a compressed data length;

[0053] The processing module is used to sequentially encode the spatial blocks based on the Morton encoding method.

[0054] On the basis of the above technical solution, preferably, the processing module is used to divide the spatial range of the entire 3DGS model into a number of regular spatial voxel units, and configure the side length of each of the spatial voxel units according to a preset resolution;

[0055] The processing module is configured to count the amount of 3DGS data contained in each of the spatial voxel units;

[0056] The processing module is configured to determine the point cloud density based on the number of points within a unit volume;

[0057] The processing module is used to calculate the difference coefficient based on the point cloud density and the area density, and is calculated by the following formula

[0058]

[0059] Among them, CV d is the coefficient of variation, N i is the data quantity, n is the number of the spatial voxel units, and ΔL is the side length of the spatial voxel unit.

[0060] On the basis of the above technical solution, preferably, the processing module is used to decode the compressed 3DGS data corresponding to each of the spatial blocks, and the decoding process includes decoding the position vector by a Zstandard decompression algorithm, and decoding the processed color vector, the scale parameter, and the rotation parameter by a tabular asymmetric digital system entropy decoding algorithm to reconstruct the 3DGS model structure data including the spatial position vector, the normal vector, the processed color vector, the scale parameter, and the rotation parameter;

[0061] The processing module is used to map each piece of the 3DGS model structure data into a Niagara particle unit, wherein the mapping fields obtained after mapping include particle position attributes, particle orientation vectors, particle color parameters, particle scaling parameters, and particle posture rotation parameters;

[0062] The processing module is configured to utilize a compute shader in a graphics processor to perform parallel structural rearrangement on the mapping fields and generate a Niagara particle data buffer structure in a structured buffer format, and inject the particle data buffer structure into a corresponding Niagara system instance through a Niagara system interface to complete particle attribute initialization and binding, thereby achieving real-time conversion of the processed data into a point cloud data structure compatible with the Niagara particle system.

[0063] Based on the above technical solution, preferably, the processing module is used to decode the compressed 3DGS data corresponding to each of the spatial blocks, and upload the decoded 3DGS data to the graphics processor in the form of a structured buffer, wherein the data unit of each structured buffer includes a spatial position vector, a normal vector, the processed color vector, the scale parameter, and the rotation parameter;

[0064] The processing module is configured to establish a parallel thread grid by scheduling a compute shader, so that each parallel thread grid processes one data unit, and map the 3DGS data field to a particle position field, a particle orientation field, a particle color field, a particle scale field, and a particle pose field of a Niagara particle system in each parallel thread grid, and perform normalization processing on the rotation parameters and generate a unique index code;

[0065] The processing module is configured to write the mapped particle attribute data into an output structured buffer in a format required by the Niagara particle system, and after all the parallel thread grids are executed, bind the output structured buffer to the Niagara particle system instance through the rendering interface function of the Niagara system as its input data source to complete the initialization and rendering scheduling of the particle life cycle, thereby generating the second processed data.

[0066] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs any of the methods described above.

[0067] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed, any one of the methods described above is executed.

[0068] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0069] 1. This application breaks the traditional static process limitations of Unreal Engine Editor loading and packaging by constructing a runtime-oriented 3DGS model processing mechanism, adopts asynchronous file reading and block-based memory pool structure to achieve high-concurrency writing of model data, significantly reduces memory overhead and transmission load through normalization and entropy coding compression, combines point cloud density-adaptive spatial blocking strategy with GPU compute shader to complete efficient conversion of Niagara particle system compatible formats, and dynamically controls model loading and unloading with multi-level resource pool and view frustum priority scheduling mechanism. Finally, through depth-aware resolution adjustment strategy, particle rendering continuity and performance optimization are achieved, thereby significantly improving the real-time processing capability and rendering efficiency of Unreal Engine in dynamic, high-density 3D modeling scenes.

[0070] 2. Loading priority is calculated based on the camera trajectory prediction results, and combined with multi-level resource pool management and frustum range scheduling mechanism, rendering objects are dynamically generated and the rendering resolution of the 3DGS model is adjusted. While ensuring the real-time responsiveness of the system, local loading and fading transition of high-density data are achieved, significantly improving the dynamic rendering efficiency and visual smoothness of 3D models in Unreal Engine.

[0071] 3. Normalize and quantize the color vector, convert the covariance matrix into scale parameters and rotation parameters, perform multi-context entropy coding compression, and use a double-buffer structure to continuously write to the memory buffer, effectively compressing the data volume, improving storage efficiency and throughput rate, and providing lightweight, high-fidelity first-processing data for subsequent graphics processing.

[0072] 4. An adaptive block partitioning algorithm based on the difference coefficient between point cloud density and regional density selects the optimal quadtree or KD tree structure for spatial partitioning, sets overlapping boundaries to prevent model boundary breaks, constructs spatial index information, and optimizes access order based on Morton coding to achieve efficient spatial organization and fast positioning scheduling of 3DGS data.

[0073] 5. Voxelization, unit volume point statistics, point cloud density and standard deviation calculations, and calculation of regional density variation coefficients based on precise mathematical formulas have established a quantifiable point cloud distribution evaluation metric. This provides an adaptive basis for the selection of spatial blocking algorithms, improves the matching degree between the blocking structure and data distribution, and thus enhances the rationality of subsequent rendering scheduling.

[0074] 6. Decode compressed 3DGS data back into structured data units and map them to Niagara particle system particle attribute fields. Highly concurrent format conversion is achieved using compute shaders in the GPU, ensuring compatibility between processed data and rendering system data structures, significantly improving GPU-side data conversion rates and particle rendering initialization efficiency.

[0075] 7. A parallel thread grid is established in the GPU to perform particle field mapping, rotation parameter normalization, and index code generation on the structured data buffer, and the results are output as a standard structured buffer of the Niagara system. This fully automates the conversion from compressed data to the particle system rendering source, ensuring real-time and consistent particle loading in high-throughput scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a flow chart of a real-time processing method for a 3DGS model based on Unreal Engine disclosed in an embodiment of the present application;

[0077] Figure 2 This is a module diagram of a real-time processing device for a 3DGS model based on Unreal Engine disclosed in an embodiment of the present application;

[0078] Figure 3 This is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.

[0079] Description of the accompanying drawings: 201, acquisition module; 202, processing module; 203, output module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0080] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0081] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0082] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0083] With the rapid development of computer vision and deep learning technologies, 3D scene reconstruction and understanding have become key technologies in cutting-edge fields such as autonomous driving, augmented reality, and virtual reality. However, in complex applications such as vehicle-road collaboration and digital twins, the traditional 3D modeling process that relies on manual loading and packaging of the Unreal Engine editor is inefficient, has redundant operations, and has poor real-time performance when processing heterogeneous and dynamically changing 3D data. This makes it difficult to adapt to system requirements for high-frequency updates and real-time interaction, seriously restricting the system's intelligence level and user experience.

[0084] This embodiment discloses a real-time processing method for 3DGS models based on Unreal Engine. Figure 1 , including the following steps S110-S150:

[0085] S110, reading the 3DGS model file and writing the 3DGS data into the block-type memory pool structure.

[0086] The Unreal Engine-based real-time 3DGS model processing method disclosed in the embodiments of the present application is applied to a server. The server includes, but is not limited to, electronic devices such as mobile phones, tablet computers, wearable devices, and personal computers (PCs). It can also be a background server running the Unreal Engine-based real-time 3DGS model processing method. The server can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0087] First, at runtime, the system calls the Unreal Engine-provided asynchronous file read interface FAsyncFileHandle to non-blockingly read a 3DGS model file. The 3DGS model file format is ply or splat, and contains raw data fields such as spatial position vectors, normal vectors, color vectors, scale parameters, and rotation parameters. During the data read process, the file content is parsed by data block and directly mapped into a memory buffer structure, ensuring concurrent data transfer and low latency. The parsed 3DGS data is then organized and written to a pre-built block-based memory pool structure. This block-based memory pool utilizes a block-based memory management strategy based on 4KB page alignment. Each memory block structure, FGSMemoryChunk, contains the data starting address, current write offset, total capacity, and an atomic lock flag to support simultaneous multi-threaded writes and avoid write conflicts. Each 3DGS data unit is aligned to 64 bytes using a fixed-length format. The structure contains a 12-byte position vector, a 12-byte normal vector, a 16-byte color vector (RGBA), a 12-byte scale parameter, and a 16-byte rotation parameter (represented as a quaternion). This format aligns to CPU cache lines for optimal access efficiency. During the write process, the system utilizes the AsyncTask mechanism to distribute write operations to an asynchronous thread pool. This is supplemented by a double-buffered structure and lock-free queues to achieve sequential writes in a highly concurrent environment. A CRC32 checksum is also used to verify the integrity of each memory block after writing. If a write anomaly is detected, the current memory block is immediately released and reloaded from the backup data, ensuring data security and consistency. The entire memory pool structure is dynamically scalable and supports a state rollback mechanism, providing high-speed cache support and stable storage for subsequent 3DGS data normalization, compression, and format conversion.

[0088] S120 , performing normalization processing and compression processing on the 3DGS data in the block-type memory pool structure, and then writing the data into a memory buffer to obtain first processed data.

[0089] In one possible implementation, the 3DGS data in the block-based memory pool structure is sequentially normalized and compressed, and then written into a memory buffer to obtain first processed data, specifically including: normalizing the color vector in the 3DGS data, converting each channel value of the color vector into a floating-point number and quantizing it into a fixed-point integer of a preset precision to obtain a processed color vector; converting the covariance matrix into a scale parameter and a rotation parameter, wherein the scale parameter is converted using a logarithmic function and retains a preset precision, and the rotation parameter is quantized into an integer form using a spherical linear coding function based on the relative rotation residual between the current rotation quaternion and the previous rotation quaternion; compressing the processed color vector, scale parameter, and rotation parameter using a multi-context entropy coding model constructed based on a tabular asymmetric digital system, and continuously writing the compressed fixed-length data units into a memory buffer of a double-buffered structure to complete the generation of the first processed data.

[0090] Specifically, the original color vector, including the red channel, green channel, blue channel and transparency channel, is extracted from each 3DGS data unit, and its value range is linearly mapped from the original integer range (such as 0 to 255) to the floating-point interval of [0 1] to meet the unified physical quantity scale standard. The mapped floating-point values ​​are then quantized according to the preset precision rules, and the floating-point value of each channel is multiplied by 10 4 The result is then rounded to a 16-bit fixed-point integer format, retaining four decimal places of accuracy to enhance the context modeling capabilities of entropy coding compression. The transparency channel in the color vector is extracted and constructed as an independent encoding model to enhance the expressiveness of different visualization levels. After normalization and quantization, a fully structured processed color vector is obtained.

[0091] First, the covariance matrix information is extracted from the 3DGS data and decomposed into a scale parameter representing the degree of three-dimensional discretization and a rotation quaternion describing the directionality using principal axis decomposition. The scale parameters are then converted to a logarithmic space using logarithmic functions. The conversion results retain a preset precision, such as retaining two decimal places, to reduce the impact of numerical dynamic range on coding efficiency. The rotation parameters are processed by calculating the relative rotation residual between the current rotation quaternion and the rotation quaternion in the previous data unit to obtain the relative attitude difference. After calculating the residual quaternion using unit quaternion multiplication, the spherical linear coding function is applied to convert the residual quaternion into an integer rotation parameter for subsequent quantization and embedding into a fixed data structure, completing the pre-compression processing of the scale and rotation parameters.

[0092] For the RGB channel portion of the color vector, the luminance histogram is assigned to multiple context groups. A tabular tANS entropy model is independently constructed for each context group, and encoding is performed using a state transition table based on symbolic probability statistics to achieve context-adaptive compression. A static tANS entropy model is established for the transparency channel as an independent variable to improve processing accuracy and compression ratio. For scale parameters, a scale context model is constructed based on their logarithmic quantized value range distribution, and a state table is constructed based on adjacent scale values, and entropy encoding is performed. For rotation parameters, a rotation context model is constructed using the quaternion quantization result corresponding to the relative rotation residual. This is then expanded according to the quaternion dimension components and entropy coding is performed based on the four-dimensional residual distribution. After encoding, each 3DGS data unit is organized into fixed-length structural units, each of which is 64 bytes long, ensuring structural consistency and streaming access efficiency.

[0093] A double-buffer structure is set up on the CPU side, consisting of two alternating memory buffers. A lock-free queue is used to schedule the compressed data unit write operation. The current buffer occupancy status is judged in real time in the write thread. When the primary buffer is full, it immediately switches to the backup buffer and triggers the data upload task. At the same time, the primary buffer is handed over to the background thread for the next step of asynchronous processing; each compressed data unit is written with 64-byte fixed-length alignment to ensure cache line alignment to improve the data transmission rate between CPU and GPU. After each write is completed, the data submission status is confirmed by setting the atomic flag, completing the generation of the entire first processing data and waiting for the subsequent processing process call.

[0094] S130 , performing coordinate alignment processing on the first processed data according to the world coordinate system of the Unreal Engine, and spatially dividing the aligned processed data into blocks according to the point cloud density and the regional density difference coefficient.

[0095] In one possible implementation, the first processed data is aligned with the world coordinate system of the Unreal Engine, and the aligned processed data is spatially blocked according to the point cloud density and the regional density difference coefficient, specifically including: calculating the difference coefficient between the point cloud density and the regional density based on the spatial distribution of the first processed data, where the regional density difference coefficient is the ratio of the point cloud density standard deviation to the point cloud density mean; when the regional density difference coefficient is less than or equal to a preset threshold, spatial blocking is performed using a quadtree algorithm, and when the regional density difference coefficient is greater than the preset threshold, spatial blocking is performed using a KD tree algorithm; in the spatial blocking process, an overlapping boundary area that accounts for a preset length of the adjacent boundaries is set between each adjacent spatial block, and corresponding spatial index information is generated for each spatial block, the spatial index information including the bounding box range, the offset address in the memory buffer, and the compressed data length; and sequentially encoding the spatial blocks based on the Morton encoding method.

[0096] Specifically, the spatial position vectors of all 3DGS data are obtained and their three-dimensional enclosing area is determined. That is, the minimum and maximum values ​​of all 3DGS data in the X, Y, and Z dimensions are calculated to form the minimum outer bounding cube of the three-dimensional space range. The spatial voxel unit side length ΔL is configured according to the preset resolution parameter, and the three-dimensional enclosing area is grid-divided using ΔL as the division scale to form a regular set of axis-aligned voxel units. Each voxel unit is numbered from v1 to v n , a total of n spatial voxel units.

[0097] Traverse all the 3DGS data in the first processed data, determine the spatial voxel unit to which it belongs according to its spatial position vector, and maintain an accumulative counter in each spatial voxel unit to record the number of 3DGS data contained in the current unit, recorded as N i According to the fixed volume ΔL of each spatial voxel unit 3 , the number of 3DGS data in the unit is divided by its volume to obtain the point cloud density of the voxel unit, which is recorded as:

[0098]

[0099] Among them, D i is the point cloud density. After completing the calculation of all voxel unit point cloud density, based on n D i value, and calculate the mean value μ of the point cloud density respectively d and standard deviation σ d The formula for calculating the mean value of point cloud density is as follows:

[0100]

[0101] The formula for calculating the standard deviation of point cloud density is as follows:

[0102]

[0103] Finally, the standard deviation of the point cloud density is compared with the mean of the point cloud density to calculate the regional density difference coefficient CV d , its complete expression is:

[0104]

[0105] Regional density variation coefficient CV d This is an important statistic used to quantify the spatial uniformity of 3DGS data. A larger value indicates greater fluctuations in point cloud density across spatial voxels and a more non-uniform spatial distribution. This provides an adaptive criterion for selecting a subsequent spatial partitioning algorithm. The entire computation can be accelerated through parallel GPU thread grids or multi-core CPU thread pools, adapting to the real-time processing requirements of large-scale 3DGS data.

[0106] When the regional density difference coefficient is less than or equal to a preset threshold (for example, 0.7), the 3DGS data is considered to be relatively evenly distributed in space and is suitable for spatial segmentation using the quadtree algorithm. In each iteration, the quadtree algorithm divides the current spatial block into four sub-regions along the XY plane, and continues the segmentation until the number of 3DGS data contained in each sub-region is lower than the set upper limit (for example, 200,000 points). When the regional density difference coefficient is greater than the preset threshold, it indicates that the 3DGS data exhibits non-uniform distribution characteristics. The KD tree algorithm is used to iteratively divide the space into two parts according to the maximum variance direction until the number of points in each sub-region is controlled within the target range, ensuring that the spatial segmentation results are adaptively matched with the point cloud density distribution.

[0107] An overlapping area is introduced on the shared boundary between each pair of adjacent spatial blocks, and its width is defined as a preset percentage (for example, 10%) of the length of the shared boundary. The overlapping area is used to buffer the boundary break problem in the particle rendering process. The overlapping data is processed together during the data loading and spatial indexing processes to ensure rendering continuity and geometric integrity. At the same time, corresponding spatial index information is generated for each spatial block. The spatial index information includes: the axis-aligned bounding box boundary coordinates of the block in three-dimensional space, the starting offset address of the block in the memory buffer, and the actual length of the compressed data corresponding to the block. All index information is organized into an index table structure that can be quickly retrieved.

[0108] For each spatial block that has completed boundary setting and index construction, the coordinates of the center point of its bounding box are calculated and normalized and discretized according to the block grid. The discretized three-dimensional coordinate components are converted into binary encoding and then interleaved to generate Morton code. Morton encoding sorts all spatial blocks in a Z-shaped spatial traversal order so that their order in the one-dimensional index maintains spatial locality. The encoding result serves as the unique sequence identifier of each spatial block in the index table, which is used to accelerate the efficiency of spatial queries such as frustum clipping, priority scheduling, and asynchronous loading.

[0109] S140 , converting the processed data in each spatial block into a point cloud data structure compatible with the Niagara particle system, and completing the data format conversion using a compute shader on a graphics processor to obtain second processed data.

[0110] In one possible implementation, the processed data in each spatial block is converted into a point cloud data structure compatible with the Niagara particle system, and the data format conversion is completed in the graphics processor using a compute shader to obtain the second processed data, specifically including: decoding the compressed 3DGS data corresponding to each spatial block, the decoding process including decoding the position vector using the Zstandard decompression algorithm, decoding the processed color vector, scale parameter and rotation parameter using the tabular asymmetric digital system entropy decoding algorithm, so as to reconstruct the 3DGS model structure data including the spatial position vector, normal vector, processed color vector, scale parameter and rotation parameter. Each piece of 3DGS model structure data is mapped to a Niagara particle unit. The mapping fields obtained after mapping include particle position attributes, particle orientation vector, particle color parameters, particle scaling parameters, and particle posture rotation parameters. In the graphics processor, a compute shader is used to rearrange the mapping fields in parallel and generate a Niagara particle data buffer structure in a structured buffer format. The particle data buffer structure is then injected into the corresponding Niagara system instance through the Niagara system interface to complete particle attribute initialization and binding, thereby realizing real-time conversion of processed data into a point cloud data structure compatible with the Niagara particle system.

[0111] Specifically, the compressed data to be decoded is first loaded into the input buffer of the graphics processor, and the Zstandard decompression algorithm is called to losslessly decode the position vector part to restore its original spatial position expression; then, an entropy decoding model based on a tabular asymmetric digital system is used to perform context-driven probability state reverse mapping on the processed color vector, scale parameter and rotation parameter, and restore them to the original quantized integer value according to the preset multi-context state table, and convert them into floating-point physical quantities; after decoding, all fields are uniformly combined into 3DGS model structure data, including spatial position vector, normal vector, processed color vector, scale parameter and rotation parameter, which are saved in the structured buffer on the GPU side as the input source for format mapping.

[0112] In each compute shader thread of the graphics processor, a piece of 3DGS model structure data is read, and each field is matched and assigned physical properties. The spatial position vector is mapped to the particle position attribute, the normal vector is mapped to the particle orientation vector, the processed color vector is mapped to the particle color parameter, the scale parameter is mapped to the particle scaling parameter, the rotation parameter is mapped to the particle posture rotation parameter, and the rotation parameter is normalized to the unit quaternion to improve the stability of particle rendering; each particle unit is independently bound to an index identifier for subsequent resource management. After completion, the particle unit is cached in the shader's local memory for batch output.

[0113] Based on GPU thread grid scheduling, each thread is assigned to process a particle unit, rearranges the particle fields in the local cache into the field order required by the Niagara particle system, and writes them into a structured output buffer. This buffer is organized as a structure array, and each particle unit has a fixed field format, including position vector, orientation vector, color vector, scale vector, and rotation quaternion, to ensure field alignment and transmission efficiency. After the structured buffer is filled, it is injected into the specified Niagara system instance through the rendering interface function provided by the Niagara system, and the initialization function is called to complete the writing and interactive binding of the particle lifecycle state, thereby achieving real-time conversion of the processed data into a point cloud data structure compatible with the Niagara particle system, resulting in the second processed data. This processing process fully utilizes GPU parallelism, achieving millisecond-level data conversion and high-frame-rate rendering initialization for large-scale data.

[0114] In one possible implementation, the processing data in each spatial block is converted into a point cloud data structure compatible with the Niagara particle system, and a compute shader is used in the graphics processor to complete the data format conversion to obtain the second processing data, which specifically includes: decoding the compressed 3DGS data corresponding to each spatial block, uploading the decoded 3DGS data to the graphics processor in the form of a structured buffer, and each data unit of the structured buffer includes a spatial position vector, a normal vector, a processing color vector, a scale parameter and a rotation parameter; establishing a parallel thread grid by scheduling the compute shader, so that each parallel thread grid processes one data unit, and in each parallel thread grid, In the row thread grid, the 3DGS data fields are mapped to the particle position field, particle orientation field, particle color field, particle scale field and particle posture field of the Niagara particle system, and the rotation parameters are normalized and a unique index code is generated; the mapped particle attribute data is written into the output structured buffer in the format required by the Niagara particle system, and after all parallel thread grids are executed, the output structured buffer is bound to the Niagara particle system instance through the rendering interface function of the Niagara system as its input data source to complete the initialization and rendering scheduling of the particle life cycle, thereby generating the second processed data.

[0115] Specifically, the compressed data in the spatial blocks are loaded into the input buffer of the graphics processor, and the Zstandard decompression algorithm is called to losslessly restore the position vector. At the same time, the entropy decoding model built based on the tabular asymmetric digital system is called to perform context reverse mapping on the processed color vector, scale parameter and rotation parameter, reconstruct the original quantized value and restore it to a floating-point physical expression. After decoding, all 3DGS data are organized in the form of a structured buffer. Each data unit contains the spatial position vector, normal vector, processed color vector, scale parameter and rotation parameter, and the high-speed upload of the structured buffer is completed through the DMA mechanism of the graphics processor.

[0116] On the graphics processor side, Dispatch Compute is used to call and start the compute shader kernel function. The thread dimension is configured so that each thread processes a 3DGS data unit in a structured buffer. The fields of the corresponding data unit are read in each parallel thread. The spatial position vector is mapped to the particle position field, the normal vector is mapped to the particle orientation field, the processed color vector is mapped to the particle color field, the scale parameter is mapped to the particle scaling field, and the rotation parameter is mapped to the particle posture field. Before executing the posture field assignment, the rotation parameter is normalized to the unit quaternion to ensure the stability of the particle system simulation. At the same time, the on-chip random number generator is called to generate a unique index code for each particle, which serves as the basic identifier for subsequent lifecycle management and resource tracking.

[0117] An independent write address is configured for each thread in the graphics processor, and atomic operations are used to control data writing synchronization. The results of the above field rearrangement are written to the data structure buffer defined by the Niagara particle system, maintaining field alignment and data type consistency. After all threads have completed writing, the Niagara rendering interface function is used to bind the output structured buffer to the specified Niagara particle system instance. The particle attributes are initialized and the life cycle is registered through the Niagara system scheduling function, and its activation state and update state control word are set. The coupling of the particle simulation module and the rendering module is completed. The output structure constitutes the second processed data, realizing the real-time conversion of the entire process from 3DGS data to a Niagara particle system compatible format.

[0118] S150 , based on the constructed multi-level resource pool management strategy and runtime memory monitoring mechanism, the second processed data is rendered according to the camera trajectory prediction result to generate a rendering result of the 3DGS model under the Unreal Engine.

[0119] In one possible implementation, based on the constructed multi-level resource pool management strategy and runtime memory monitoring mechanism, the second processed data is rendered according to the camera trajectory prediction result to generate a rendering result of the 3DGS model under the Unreal Engine, specifically including: calculating the loading priority of the second processed data, dynamically generating an Actor in the current scene according to the loading priority and attaching a Niagara component to form a rendering object and loading it into the world scene within the rendering frustum; during the rendering process, based on the world space depth information in the Unreal Engine, dynamically adjusting the rendering resolution of the 3DGS model in different spatial blocks, and setting fading transition particles for the area where the rendering resolution changes to complete smooth switching between resolution levels, thereby realizing real-time loading and dynamic updating of the 3DGS model under the Unreal Engine.

[0120] Specifically, the historical motion trajectory data of the camera Actor in the Unreal Engine is first collected at runtime. A state vector model is established based on the Kalman filter, which includes the camera position, velocity, and acceleration, and its motion trend for several frames in the future is predicted. Based on the predicted camera viewpoint position and direction, the distance parameters between the center of the three-dimensional bounding box corresponding to each spatial block and the current camera are calculated. At the same time, the projected area of ​​the bounding box in the camera's frustum is calculated to estimate its screen occupancy. The current camera motion coefficient is introduced as a dynamic factor, and a loading priority evaluation function is jointly constructed in the following form:

[0121]

[0122] All spatial blocks are sorted by priority, and high-priority items are selected for activation in the current rendering frame. The data of the corresponding spatial blocks is scheduled through the dynamic resource pool, automatically assigned Actors and bound to the generated Niagara particle components, instantiated as rendering objects in the Unreal Engine scene, and loaded into the world space covered by the current camera frustum to ensure that important areas are presented first.

[0123] During the rendering process, the specific implementation method of dynamically adjusting the rendering resolution of the 3DGS model based on the world space depth information provided by the Unreal Engine is as follows: the current camera's view frustum depth map is obtained in each rendering frame, the center of the bounding box of each spatial block is mapped to the view space, and its depth value Scene Depth is calculated. The resolution level is matched according to the depth value and the preset multi-level depth threshold table. The matching levels are as follows: L0: depth 0-50m, point cloud density is 100%, and the update frequency is 60Hz; L1: depth 50-150m, point cloud density is 75%, and the update frequency is 30Hz; L2: depth 150-500m, point cloud density is 50%, and the update frequency is 15Hz; L3: depth greater than 500m, point cloud density is 25%, and the update frequency is 5Hz.

[0124] When it is detected that a certain spatial block needs to switch the rendering resolution, a transition zone is constructed at the junction of the new and old levels. The transition zone range is set to 10 meters wide, and the smoothing function is used to calculate the blending weight w = smoothstep (r min ,r max ,d), where d is the current depth, r min ,r max The start and end depths of the transition zone are set; the particle mixing script is enabled in the Niagara particle system, and the particle transparency is gradually reduced in the old resolution particle group, while the particle transparency is gradually increased in the new level particle group. This ensures that the particle rendering results do not jump or break during the resolution conversion process, and achieves a gradual and smooth transition of rendering accuracy, improving the overall stability and visual continuity of real-time loading and dynamic updates of 3DGS models in Unreal Engine.

[0125] This embodiment also discloses a real-time processing device for 3DGS models based on Unreal Engine. Figure 2 , comprising an acquisition module 201, a processing module 202 and an output module 203, the device is used to execute any of the above-mentioned 3DGS model real-time processing methods based on the Unreal Engine, wherein:

[0126] The acquisition module 201 is used to read the 3DGS model file and write the 3DGS data into the block-type memory pool structure.

[0127] The processing module 202 is configured to perform normalization processing and compression processing on the 3DGS data in the block-type memory pool structure, and then write the processed data into a memory buffer to obtain first processed data.

[0128] The processing module 202 is configured to perform coordinate alignment processing on the first processed data according to the world coordinate system of the Unreal Engine, and to spatially divide the aligned processed data into blocks according to the point cloud density and the regional density difference coefficient.

[0129] The processing module 202 is used to convert the processed data in each spatial block into a point cloud data structure compatible with the Niagara particle system, and complete the data format conversion using a compute shader on a graphics processor to obtain second processed data.

[0130] The output module 203 is used to render the second processed data according to the camera trajectory prediction result based on the constructed multi-level resource pool management strategy and runtime memory monitoring mechanism, and generate a rendering result of the 3DGS model under the Unreal Engine.

[0131] In one possible implementation, the output module 203 is used to calculate the loading priority of the second processed data, dynamically generate an Actor in the current scene based on the loading priority, attach a Niagara component, constitute a rendering object, and load it into the world scene within the rendering frustum.

[0132] Processing module 202 is used to dynamically adjust the rendering resolution of the 3DGS model in different spatial blocks based on the world space depth information in the Unreal Engine during the rendering process, and set fade-out transition particles for the areas where the rendering resolution changes to complete smooth switching between resolution levels, thereby realizing real-time loading and dynamic updating of the 3DGS model in the Unreal Engine.

[0133] In a possible implementation, the processing module 202 is configured to perform normalization processing on the color vector in the 3DGS data, convert each channel value of the color vector into a floating point number and quantize it into a fixed-point integer with a preset precision to obtain a processed color vector.

[0134] Processing module 202 is used to convert the covariance matrix into scale parameters and rotation parameters, wherein the scale parameters are converted using a logarithmic function and retain a preset accuracy, and the rotation parameters are based on the relative rotation residual between the current rotation quaternion and the previous rotation quaternion, and are quantized into integer form using a spherical linear encoding function.

[0135] The processing module 202 is used to compress the color vector, scale parameter and rotation parameter using a multi-context entropy coding model constructed based on a tabular asymmetric digital system, and continuously write the compressed fixed-length data units into the memory buffer of the double buffer structure to complete the generation of the first processed data.

[0136] In a possible implementation, the processing module 202 is configured to calculate a difference coefficient between the point cloud density and the regional density based on the spatial distribution of the first processed data, where the regional density difference coefficient is a ratio of a standard deviation of the point cloud density to a mean of the point cloud density.

[0137] The processing module 202 is configured to use a quadtree algorithm to perform spatial partitioning when the regional density difference coefficient is less than or equal to a preset threshold, and to use a KD tree algorithm to perform spatial partitioning when the regional density difference coefficient is greater than the preset threshold.

[0138] The processing module 202 is used to set an overlapping boundary area that accounts for a preset length of the adjacent boundaries between each adjacent spatial block during the spatial block process, and generate corresponding spatial index information for each spatial block. The spatial index information includes the bounding box range, the offset address in the memory buffer and the compressed data length.

[0139] The processing module 202 is configured to sequentially encode the spatial blocks based on the Morton encoding method.

[0140] In a possible implementation, the processing module 202 is configured to divide the spatial range of the entire 3DGS model into a number of regular spatial voxel units, and configure the side length of each spatial voxel unit according to a preset resolution.

[0141] The processing module 202 is configured to count the amount of 3DGS data contained in each spatial voxel unit.

[0142] The processing module 202 is configured to determine the point cloud density according to the number of points within the unit volume.

[0143] The processing module 202 is used to calculate the difference coefficient based on the point cloud density and the area density, and is calculated by the following formula

[0144]

[0145] Among them, CV d is the coefficient of variation, N i is the number of data, n is the number of spatial voxel units, and ΔL is the side length of the spatial voxel unit.

[0146] In one possible implementation, the processing module 202 is used to decode the compressed 3DGS data corresponding to each spatial block. The decoding process includes decoding the position vector through the Zstandard decompression algorithm, and decoding the processing color vector, scale parameter and rotation parameter through the tabular asymmetric digital system entropy decoding algorithm to reconstruct the 3DGS model structure data including the spatial position vector, normal vector, processing color vector, scale parameter and rotation parameter.

[0147] The processing module 202 is used to map each piece of 3DGS model structure data into a Niagara particle unit. The mapping fields obtained after mapping include particle position attributes, particle orientation vectors, particle color parameters, particle scaling parameters, and particle posture rotation parameters.

[0148] Processing module 202 is configured to utilize a compute shader in the graphics processor to perform parallel structural rearrangement of the mapping fields and generate a Niagara particle data buffer structure in a structured buffer format. The processing module 202 then injects the particle data buffer structure into the corresponding Niagara system instance through the Niagara system interface to initialize and bind particle attributes, thereby enabling real-time conversion of processed data into a point cloud data structure compatible with the Niagara particle system.

[0149] In one possible implementation, the processing module 202 is configured to decode the compressed 3DGS data corresponding to each spatial block and upload the decoded 3DGS data to the graphics processor in the form of a structured buffer. The data unit of each structured buffer includes a spatial position vector, a normal vector, a processed color vector, a scale parameter, and a rotation parameter.

[0150] The processing module 202 is used to establish a parallel thread grid by scheduling the compute shader, so that each parallel thread grid processes one data unit, and in each parallel thread grid, the 3DGS data field is mapped to the particle position field, particle orientation field, particle color field, particle scale field and particle posture field of the Niagara particle system, and the rotation parameters are normalized and a unique index code is generated.

[0151] Processing module 202 is used to write the mapped particle attribute data into an output structured buffer in the format required by the Niagara particle system. After all parallel thread grids are executed, the output structured buffer is bound to the Niagara particle system instance through the Niagara system's rendering interface function as its input data source to complete the particle life cycle initialization and rendering scheduling, thereby generating second processed data.

[0152] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0153] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0154] The communication bus 302 is used to implement the connection and communication between these components.

[0155] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0156] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0157] The processor 301 may include one or more processing cores. The processor 301 utilizes various interfaces and circuits to connect various parts of the entire server. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and calling data stored in the memory 305, the processor 301 performs various server functions and processes data. Optionally, the processor 301 may be implemented using at least one hardware form selected from digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 301 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs. The GPU is responsible for rendering and drawing the content to be displayed on the display screen. The modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 301 and may be implemented separately on a single chip.

[0158] The memory 305 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also optionally be at least one storage device located away from the aforementioned processor 301. The memory 305, as a computer storage medium, may include an operating system, a network communication module, a user interface 303 module, and an application for a real-time processing method for a 3DGS model based on Unreal Engine.

[0159] exist Figure 3 In the electronic device shown, user interface 303 is primarily used to provide an input interface for the user and to obtain user input data. Processor 301 can be used to invoke an application stored in memory 305 that contains a method for real-time processing of 3DGS models based on Unreal Engine. When executed by one or more processors 301, the electronic device executes one or more of the methods described in the aforementioned embodiments.

[0160] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0161] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0163] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0164] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0165] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory 305 and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory 305 includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disk.

[0166] The present application also discloses a computer-readable storage medium storing instructions, which, when executed by one or more processors 301 , enable an electronic device to execute one or more methods in the above embodiments.

[0167] The above are merely exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A real-time processing method for 3DGS models based on Unreal Engine, characterized in that: The method comprises: Read 3DGS model file and write 3DGS data into block memory pool structure; After normalizing and compressing the 3DGS data in the block-type memory pool structure, the data is written into a memory buffer to obtain first processed data; Performing coordinate system alignment processing on the first processed data according to the world coordinate system of the Unreal Engine, and spatially dividing the aligned processed data into blocks according to the point cloud density and the regional density difference coefficient; Converting the processed data in each of the spatial blocks into a point cloud data structure compatible with the Niagara particle system, and completing the data format conversion using a compute shader on a graphics processor to obtain second processed data; Based on the constructed multi-level resource pool management strategy and runtime memory monitoring mechanism, the second processed data is rendered according to the camera trajectory prediction result to generate the rendering result of the 3DGS model under the Unreal Engine.

2. A real-time processing method for 3DGS models based on Unreal Engine according to claim 1, characterized in that: The multi-level resource pool management strategy and runtime memory monitoring mechanism based on the construction are used to render the second processed data according to the camera trajectory prediction result to generate the rendering result of the 3DGS model under the Unreal Engine, specifically including: Performing a loading priority calculation on the second processed data, dynamically generating an Actor in the current scene based on the loading priority and attaching a Niagara component to the Actor, forming a rendering object, and loading the object into the world scene within the rendering frustum; During the rendering process, based on the world space depth information in the Unreal Engine, the rendering resolution of the 3DGS model in different spatial blocks is dynamically adjusted, and fade-out transition particles are set for the area where the rendering resolution changes to complete the smooth switching between resolution levels, thereby realizing real-time loading and dynamic updating of the 3DGS model under the Unreal Engine.

3. The real-time processing method of a 3DGS model based on Unreal Engine according to claim 1, characterized in that: The 3DGS data in the block-based memory pool structure is sequentially normalized and compressed, and then written into a memory buffer to obtain first processed data, specifically including: Normalizing the color vector in the 3DGS data, converting each channel value of the color vector into a floating point number and quantizing it into a fixed-point integer of a preset precision to obtain a processed color vector; Converting the covariance matrix into a scale parameter and a rotation parameter, wherein the scale parameter is converted using a logarithmic function and retains a preset accuracy, and the rotation parameter is quantized into an integer form by a spherical linear encoding function based on the relative rotation residual between the current rotation quaternion and the previous rotation quaternion; The processed color vector, the scale parameter and the rotation parameter are respectively compressed using a multi-context entropy coding model constructed based on a tabular asymmetric digital system, and the compressed fixed-length data units are continuously written into a memory buffer of a double buffer structure to complete the generation of the first processed data.

4. The real-time processing method of a 3DGS model based on Unreal Engine according to claim 1, characterized in that: The step of aligning the first processed data according to the world coordinate system of the Unreal Engine and spatially dividing the aligned processed data according to the point cloud density and the regional density difference coefficient specifically includes: Calculating a difference coefficient between point cloud density and regional density based on the spatial distribution of the first processed data, wherein the regional density difference coefficient is a ratio of a standard deviation of the point cloud density to a mean of the point cloud density; When the regional density difference coefficient is less than or equal to the preset threshold, the quadtree algorithm is used for spatial block division; when the regional density difference coefficient is greater than the preset threshold, the KD tree algorithm is used for spatial block division; During the spatial block division process, an overlapping boundary region of a predetermined length of adjacent boundaries is set between each adjacent spatial block, and corresponding spatial index information is generated for each spatial block, the spatial index information including a bounding box range, an offset address in a memory buffer, and a compressed data length; The spatial blocks are sequentially encoded based on a Morton encoding method.

5. The real-time processing method of a 3DGS model based on Unreal Engine according to claim 4, characterized in that: The calculating the difference coefficient between the point cloud density and the regional density based on the spatial distribution of the first processed data specifically includes: Divide the spatial range of the entire 3DGS model into a number of regular spatial voxel units, and configure the side length of each of the spatial voxel units according to a preset resolution; For each of the spatial voxel units, counting the amount of 3DGS data contained therein; Determining the point cloud density according to the number of points within a unit volume; The difference coefficient is calculated based on the point cloud density and the area density, and is calculated by the following formula Among them, CV d is the coefficient of variation, N i is the data quantity, n is the number of the spatial voxel units, and ΔL is the side length of the spatial voxel unit.

6. The real-time processing method of a 3DGS model based on Unreal Engine according to claim 3, characterized in that: The processing data in each of the spatial blocks is converted into a point cloud data structure compatible with the Niagara particle system, and a compute shader is used on a graphics processor to complete the data format conversion to obtain the second processed data, specifically including: Decoding the compressed 3DGS data corresponding to each of the spatial blocks, the decoding process comprising decoding the position vector using a Zstandard decompression algorithm, and decoding the processed color vector, the scale parameter, and the rotation parameter using a tabular asymmetric digital system entropy decoding algorithm to reconstruct 3DGS model structure data including the spatial position vector, the normal vector, the processed color vector, the scale parameter, and the rotation parameter; Mapping each piece of the 3DGS model structure data to a Niagara particle unit, wherein the mapping fields obtained after mapping include particle position attributes, particle orientation vectors, particle color parameters, particle scaling parameters, and particle posture rotation parameters; A compute shader is used in a graphics processor to perform parallel structural rearrangement on the mapping fields and generate a Niagara particle data buffer structure in a structured buffer format. The particle data buffer structure is then injected into a corresponding Niagara system instance through a Niagara system interface to complete particle attribute initialization and binding, thereby achieving real-time conversion of the processed data into a point cloud data structure compatible with the Niagara particle system.

7. The real-time processing method of a 3DGS model based on Unreal Engine according to claim 3, characterized in that: The processing data in each of the spatial blocks is converted into a point cloud data structure compatible with the Niagara particle system, and a compute shader is used on a graphics processor to complete the data format conversion to obtain second processed data, which specifically includes: Decoding the compressed 3DGS data corresponding to each of the spatial blocks, and uploading the decoded 3DGS data to the graphics processor in the form of a structured buffer, wherein the data unit of each structured buffer includes a spatial position vector, a normal vector, the processed color vector, the scale parameter, and the rotation parameter; Establishing a parallel thread grid by scheduling a compute shader, so that each parallel thread grid processes one data unit, mapping the 3DGS data field into a particle position field, a particle orientation field, a particle color field, a particle scale field, and a particle pose field of a Niagara particle system in each parallel thread grid, performing normalization processing on the rotation parameters, and generating a unique index code; The mapped particle attribute data is written into an output structured buffer in the format required by the Niagara particle system. After all the parallel thread grids are executed, the output structured buffer is bound to the Niagara particle system instance through the rendering interface function of the Niagara system as its input data source to complete the initialization and rendering scheduling of the particle life cycle, thereby generating the second processed data.

8. A real-time processing device for 3DGS models based on Unreal Engine, characterized in that: The device is used to execute a real-time processing method for a 3DGS model based on an Unreal Engine according to any one of claims 1 to 7, and the device comprises an acquisition module (201), a processing module (202), and an output module (203), wherein: The acquisition module (201) is used to read the 3DGS model file and write the 3DGS data into the block-type memory pool structure; The processing module (202) is used to perform normalization processing and compression processing on the 3DGS data in the block-type memory pool structure, and then write the data into the memory buffer to obtain first processed data; The processing module (202) is used to perform coordinate system alignment processing on the first processed data according to the world coordinate system of the Unreal Engine, and to perform spatial segmentation on the aligned processed data according to the point cloud density and the regional density difference coefficient; The processing module (202) is used to convert the processed data in each of the spatial blocks into a point cloud data structure compatible with the Niagara particle system, and complete the data format conversion using a computation shader on a graphics processor to obtain second processed data; The output module (203) is used to render the second processed data according to the camera trajectory prediction result based on the constructed multi-level resource pool management strategy and runtime memory monitoring mechanism, and generate a rendering result of the 3DGS model under the Unreal Engine.

9. An electronic device, characterized in that: The electronic device comprises a processor (301), a communication bus (302), a user interface (303), a network interface (304) and a memory (305), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are both used to communicate with other devices, the communication bus (302) is used to realize connection and communication between components in the electronic device, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images