3D scene data generation methods, parsing methods, and rendering devices

Abstract

This application relates to the field of 3D scene rendering, specifically to a method for generating and parsing 3D scene data, a rendering device, and a computer storage medium. The method performs Level of Detail (LOD) hierarchical processing on the 3D scene data, dividing the foreground data into multiple LOD layers. Each LOD layer is further divided into multiple mesh nodes, each stored independently, allowing data to be loaded on demand. Simultaneously, by employing different compression methods for different data types, the GPU can directly parse and read the compressed data without decompression in memory, significantly reducing memory and video memory bandwidth usage and improving rendering efficiency.

Description

Technical Field

[0001] This application relates to the field of three-dimensional scene rendering, specifically to a three-dimensional scene data generation method, parsing method, rendering device, and computer storage medium. Background Technology

[0002] In recent years, 3D Gaussian Splashing (3DGS) technology has made significant progress in the field of 3D scene reconstruction and rendering, based on Neural Radiance Fields (NeRF). It achieves high-fidelity detail while maintaining low scene complexity, and also boasts high real-time rendering performance. Since its inception, 3DGS has fundamentally changed the paradigm of 3D scene reconstruction and rendering, and has achieved large-scale availability. Compared to traditional oblique photogrammetry mesh reconstruction and scene point clouds, 3DGS scenes have a significant advantage in detail rendering and possess excellent extensibility and scalability potential. It achieves high-fidelity scene reconstruction and 3D rendering with a more compact expression, gradually becoming a major technology in the field of 3D reconstruction.

[0003] The inventors of this application discovered during their research that the existing standard 3DGS data organization is in PLY format, which is unable to store and render large scenes. Summary of the Invention

[0004] In view of the above problems, embodiments of this application provide a method for generating and parsing three-dimensional scene data, a rendering device, and a computer storage medium to solve the above-mentioned technical problems existing in the prior art.

[0005] According to one aspect of the embodiments of this application, a method for generating three-dimensional scene data is proposed, applied to a three-dimensional scene data generation end, wherein the three-dimensional scene data generation end is adapted to a three-dimensional scene data parsing end, and the method includes:

[0006] Acquire 3D scene data;

[0007] The three-dimensional scene data is processed according to the preset number of LOD levels and the simplification algorithm to generate corresponding LOD layer data. The LOD layer data includes multiple first grid nodes, and the first grid node includes position data, scale data, color data and rotation data.

[0008] The position data Position is stored directly so that the position data generated by the 3D scene data parsing end remains unchanged;

[0009] The scaling ratio data Scale is quantized and compressed before being stored, so that the 3D scene data parsing end can directly process the quantized and compressed scaling ratio data Scale.

[0010] The color data Color is compressed using RGBA32-bit compression and then stored so that the 3D scene data parsing end can directly process the RGBA32-bit compressed color data Color.

[0011] The rotation state data (Rotation) is normalized and compressed before being stored, so that the 3D scene data parsing end can directly process the normalized and compressed rotation state data (Rotation).

[0012] Preferably, in some embodiments, the method further includes:

[0013] The 3D scene data is processed according to a preset LOD level and a simplification algorithm to generate collision data Collision. The collision data Collision includes multiple second mesh nodes, wherein the second mesh nodes and the first mesh nodes adopt the same partitioning rules.

[0014] Preferably, in some embodiments, the collision data includes: a head portion and a data portion;

[0015] The data portion includes multiple MESH data and BVH acceleration structures, wherein the MESH data uses the same partitioning rule as the first mesh node.

[0016] Preferably, in some embodiments, the MESH data includes vertex data and triangle data;

[0017] The method further includes:

[0018] A bounding box is constructed based on the vertex and triangle data of multiple MESH data, and the bounding box includes all geometric objects of the 3D scene;

[0019] Traverse all the geometric objects and divide them into geometric object groups according to a preset threshold;

[0020] Construct intermediate nodes and leaf nodes based on the geometric object group;

[0021] The BVH acceleration structure is generated based on the intermediate nodes and leaf nodes.

[0022] Preferably, in some embodiments, the method further includes:

[0023] The 3D scene data is processed according to the preset LOD level and simplification algorithm to generate environment data. The environment data includes position data, scale data, color data and rotation data.

[0024] The location data Position is stored directly;

[0025] The scaling ratio data Scale is quantized and compressed before being stored.

[0026] The color data is compressed into RGBA 32-bit and then stored.

[0027] The rotation state data Rotation is normalized, compressed, and then stored.

[0028] Preferably, in some embodiments, the method further includes:

[0029] The 3D scene data is processed according to the preset LOD level and a simplified algorithm to generate spherical harmonic data Shcoef;

[0030] The spherical harmonic data Shcoef is quantized and compressed, and the quantized and compressed spherical harmonic data Shcoef is stored together with the environmental data Environment.

[0031] According to another aspect of this application, a three-dimensional scene data parsing method is also proposed, applied to a three-dimensional scene data parsing terminal. The three-dimensional scene data parsing terminal includes an image processing unit and a non-volatile storage unit. The non-volatile storage unit stores three-dimensional scene data generated by the three-dimensional scene data generation method described in the above embodiments. The method includes:

[0032] The image processing unit directly loads the 3D scene data stored on the non-volatile storage unit into the video memory of the image processing unit;

[0033] The position data Position, the compressed scaling data Scale, the compressed color data Color, and the compressed rotation data Rotation are parsed.

[0034] Preferably, in some embodiments, the method further includes:

[0035] Obtain the quantization value corresponding to the compressed scaling ratio data Scale;

[0036] Convert the quantized value into an integer value;

[0037] The scaling ratio data Scale is determined by interpolation based on the integer value and the maximum and minimum values ​​corresponding to the preset scaling ratio data Scale.

[0038] According to a third aspect of the embodiments of this application, some embodiments also propose a three-dimensional scene rendering device, characterized in that it includes: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0039] The memory is used to store at least one program, which causes the processor to execute the three-dimensional scene data generation method described in the above embodiments; or, to execute the three-dimensional scene data parsing method described in the above embodiments.

[0040] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein at least one program is stored in the storage medium, and when the program is run on a three-dimensional scene rendering device, the three-dimensional scene rendering device performs the three-dimensional scene data generation method described in the above embodiments; or performs the three-dimensional scene data parsing method described in the above embodiments.

[0041] In summary, this application embodiment performs LOD (Level of Detail) hierarchical processing on 3D scene data, dividing the foreground data in the 3D scene data into multiple LOD layers. Each LOD layer is further divided into multiple mesh nodes, and each mesh node is stored independently, enabling data to be loaded on demand. Furthermore, the overlay of multiple LOD data layers allows for the display of models with different levels of detail. This method can display models of different granularities at different distances or on different devices, improving rendering efficiency and effectively enabling the display and management of large scenes. Furthermore, to improve the rendering effect of 3D scenes, this application embodiment also provides a data pass-through mechanism. This data pass-through mechanism first classifies the data of each mesh node and compresses the data using different compression methods according to different data categories. During 3D data parsing and rendering, the compressed data is directly transferred from the hard disk to the GPU unit's video memory, which reduces the amount of data. On the other hand, considering that the GPU instruction set usually contains instructions specifically for compressed data processing, these instructions can optimize the decoding process and reduce the consumption of GPU computing resources. Therefore, by using different compression methods for different data, this application embodiment enables the GPU to directly parse and read the compressed data without decompressing it in memory, greatly reducing the occupation of memory and video memory bandwidth and improving rendering efficiency.

[0042] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0043] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0044] Figure 1 This application provides a schematic diagram of the functional structure of a three-dimensional scene rendering device.

[0045] Figure 2 A flowchart of a three-dimensional scene data generation method provided in this application embodiment;

[0046] Figure 3 A schematic diagram of a multi-level LOD provided for an embodiment of this application;

[0047] Figure 4 This is a flowchart of the collision data processing provided in an embodiment of this application;

[0048] Figure 5 A flowchart of a three-dimensional scene data parsing method provided in this application embodiment;

[0049] Figure 6 This is a flowchart of a method for parsing scaling data provided in an embodiment of this application;

[0050] Figure 7 This is a structural schematic diagram of a three-dimensional scene rendering device provided in an embodiment of this application. Detailed Implementation

[0051] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein.

[0052] In the field of reconstructing three-dimensional scenes from two-dimensional images, 3D Gaussian Splash (hereinafter referred to as 3DGS) technology has made significant progress based on Neural Radiance Fields (NeRF). It achieves high-fidelity details while maintaining low scene complexity, and also has high real-time rendering performance. It has been widely applied in fields such as digital twins, cultural tourism, and simulation, and is a very promising three-dimensional scene expression and rendering technology.

[0053] 3DGS (3D Gaussian Splatting) technology is a method for 3D scene reconstruction and rendering. It reconstructs and renders high-fidelity 3D scenes using point cloud and image data. 3DGS uses Gaussian ellipsoids as the scene representation primitives, or Gaussian units, and describes the relationships between Gaussian ellipsoids using Gaussian functions, thereby achieving high-fidelity reconstruction and rendering of 3D scenes. In open, large-scale spaces, the total number of Gaussian units is enormous; real-world large scenes often contain tens of millions to billions of Gaussian units, which often cannot be rendered all at once and require layered, block-based rendering.

[0054] In existing technologies, the original standard 3DGS data is organized in PLY format, a widely used 3D model data storage format. PLY format supports storing various types of geometric data, such as vertex positions, normals, texture coordinates, and colors. It is characterized by its ease of use, flexibility, and cross-platform compatibility, and is supported by many 3D modeling software and rendering engines. Although PLY format has advantages such as high data precision and good compatibility, it also has some significant drawbacks. PLY files are typically large in size due to the lack of data compression, resulting in low efficiency when processing extremely large scenes. Furthermore, due to the large data volume, PLY format is also inefficient during rendering, consuming significant storage and transmission bandwidth.

[0055] To facilitate data storage and rendering of large scenes, the industry has designed several 3DGS data formats, such as SPZ, gltf, SOG, and Splat. However, these data formats focus on data compression in some cases and chunk management in others, making it impossible to manage ultra-large scenes and perform streaming data loading and rendering.

[0056] The inventors of this application discovered in their research that the aforementioned data formats lack effective data layering, making it impossible to manage and render large scenes. Furthermore, while the above solutions involve data compression, the compressed data must first be loaded from the hard drive into memory for decompression during data processing. Only after decompression can the graphics processing unit (GPU) load the decompressed data into video memory for processing. This results in the compressed data consuming a significant amount of memory after decompression, leading to substantial memory consumption. Simultaneously, data decompression consumes CPU resources, increasing data processing latency. Loading the decompressed data into the GPU also consumes significant system bandwidth, creating a bandwidth bottleneck and limiting rendering performance. Moreover, for devices with limited performance, frequent memory operations and decompression processes significantly impact device responsiveness and user experience, resulting in low rendering efficiency.

[0057] In view of this, embodiments of this application propose a method for generating and parsing 3D scene data, a device, and a readable computer storage medium. By performing LOD (Level of Detail) hierarchical processing on the 3D scene data, the foreground data in the 3D scene data is divided into multiple LOD layers. Each LOD layer is divided into multiple mesh nodes, and each mesh node is stored independently, enabling data to be loaded on demand. Furthermore, by overlaying multiple LOD data layers, models of different levels of detail can be displayed. This method allows for the display of models of different granularities at different distances or on different devices, improving rendering efficiency and effectively enabling the display and management of large scenes. Furthermore, to improve the rendering effect of 3D scenes, this application embodiment also provides a data pass-through mechanism. This data pass-through mechanism first classifies the data of each mesh node and compresses the data using different compression methods according to different data categories. During 3D data parsing and rendering, the compressed data is directly transferred from the hard disk to the GPU unit's video memory, which reduces the amount of data. On the other hand, considering that the GPU instruction set usually contains instructions specifically for compressed data processing, these instructions can optimize the decoding process and reduce the consumption of GPU computing resources. Therefore, by using different compression methods for different data, this application embodiment enables the GPU to directly parse and read the compressed data without decompressing it in memory, greatly reducing the occupation of memory and video memory bandwidth and improving rendering efficiency.

[0058] Figure 1 This illustration shows an application scenario of the 3D scene data generation and parsing methods proposed in this application. In this embodiment, the 3D scene data generation method is applied to a 3D scene data generation terminal, and the 3D scene data parsing method is applied to a 3D scene data parsing terminal. Both the 3D scene data generation terminal and the 3D scene data parsing terminal are virtual functional modules. In practical applications, these two functional modules can be simultaneously hosted on the same 3D rendering device. The 3D scene data generation terminal is mainly used for the acquisition and processing of 3D scene data. For example, after capturing a 3D scene and acquiring scene data through a camera or radar device, the data is processed. The acquired raw data is processed and refined to generate rendering data that can be called by the 3D scene data parsing terminal. This rendering data is usually stored on non-volatile memory, such as a hard drive or cloud server. When the rendering result needs to be presented, the 3D scene data parsing terminal calls the processed rendering data to present the 3D effect.

[0059] like Figure 1As shown, the 3D scene data generation end may include a data acquisition unit, a data processing unit, and a data storage unit. The data acquisition unit is mainly used for 3D scene scanning to obtain raw 3D scene data. The output processing unit is mainly used for processing the acquired raw 3D scene data to generate callable rendering data. The data storage unit is used to store the processed rendering data, which is usually a non-volatile memory such as a hard disk.

[0060] The 3D scene data parsing unit may include a data parsing unit and a display storage unit. The display storage unit is mainly used to read rendering data from non-volatile memory such as hard disks. The data parsing unit is mainly used to parse the rendering data and render the 3D scene. In practical applications, the data parsing unit and the display storage unit can be integrated into the image processing unit (GPU).

[0061] Specifically, the 3D scene data generation terminal executes the 3D scene data generation method proposed in the embodiments of this application, such as... Figure 2 As shown, it includes:

[0062] Step 110: Acquire 3D scene data

[0063] In practical applications, 3D scene data can be acquired using 3D data acquisition devices, including 3D scanning equipment, depth cameras, RGB-D cameras, and LiDAR (Light Detection and Ranging) systems. After preprocessing, filtering, registration, and fusion, the acquired data can generate high-quality 3D scene data. This 3D scene data includes point cloud data, image data, and mesh data.

[0064] Step 120: Process the 3D scene data according to the preset number of LOD levels and the simplification algorithm to generate corresponding LOD layer data. The LOD layer data includes multiple first grid nodes, and the first grid node includes position data, scale data, color data and rotation data.

[0065] In this embodiment, after acquiring the 3D scene data, the data is first processed according to a preset number of LOD levels and a simplified algorithm to perform LOD layering. First, the number of LOD layers and a distance threshold need to be determined. The number of LOD layers can be determined based on application requirements and performance specifications. Typically, at least two layers are required: a low-detail layer and a high-detail layer. The distance threshold sets a distance limit for each LOD layer. When the distance between the observer and the model exceeds this threshold, the system switches to the next coarser LOD layer.

[0066] When generating LOD layers, preset simplification algorithms (such as Quadric Error Metrics (QEM) algorithm, edge folding algorithm, etc.) can be used to simplify the original model and generate LOD models with different levels of detail. For example: LOD0 (highest detail): retains the highest level of detail of the original model; LOD1 (medium detail): simplifies the model and removes some details; LOD2 (low detail): further simplifies, retaining only the basic shape; LOD3 (lowest detail): may be just a simple geometry, such as a cube or sphere.

[0067] After generating different LOD layers, the models of different LOD layers are stored separately, and the distance threshold of each LOD layer is recorded. At the same time, an index is created for each LOD layer so that it can be quickly switched at runtime.

[0068] Figure 3 This application illustrates a structural diagram of each LOD layer proposed in its embodiments. Figure 3 The data structure includes nodes, indices, and units. A node is a grid partition for a specified LOD layer. Its size is related to the data distribution density and can be adaptive. It serves as the basic data unit for LOD layering. Each LOD layer contains multiple nodes, and each node contains several scene data points. Each scene data point consists of scene data with different attributes, including but not limited to position, scaling, color, and rotation attributes. An index is used to index the x and y values ​​of a partition, represented by a Uint32 value, with the x value in the lower 16 bits and the y value in the higher 16 bits. A unit refers to a group of nodes corresponding to the same index. Table 1 below shows the data organization method for LOD layering in 3D scene data, consisting of multiple files.

[0069] Table 1

[0070]

[0071] In Table 1, the meta.lcc file acts as a data description file, typically encoded in JSON format. It includes multiple attributes to describe the data information of the entire 3D scene, such as LOD level, index block size, number of Splats, and node size. During 3D scene data parsing, the 3D scene data parser first calls this meta.lcc file to determine various parameters for parsing, ensuring correct parsing of the 3D scene data.

[0072] The Index.bin file is an index for each scene block. The length of this file in a specific scene data is fixed (described by the relevant attributes in the meta.lcc file), but the length of different scene data may be inconsistent. Little-endian storage can be used when storing the data.

[0073] The data.bin file is binary data, which stores the main data in sequence, including but not limited to: position data, scale data, color data, and rotation data.

[0074] Table 2 shows the data structure of the first grid node:

[0075] Table 2

[0076]

[0077] In Table 2, Position: position data, each component is of type float; Color: color data, stored in RGBA format; Scale: scaling data; Rotation: rotation data, stored using a special compression method.

[0078] The above describes the data structure settings for the LOD layering. During subsequent data generation and parsing, the 3D scene rendering device will use this data structure as the basis for rendering operations.

[0079] After LOD layering, each mesh node is stored independently, allowing data to be loaded on demand. By stacking multiple layers of LOD data, models of different levels of detail can be displayed. This method can display models of different granularities at different distances or on different devices, improving rendering efficiency and enabling better display and management of large scenes.

[0080] Meanwhile, in order to achieve transparent transmission of grid node data and reduce the data volume of grid node data, in this embodiment of the application, when storing data through the data structure of the grid node, corresponding processing is first performed according to the data type. Specifically, as in steps 130 to 160.

[0081] Step 130: Directly store the position data Position so that the position data generated by the 3D scene data parsing terminal from the 3D scene data remains unchanged;

[0082] In the data structure organization of mesh nodes, Position data is used to describe the position information of 3D models or objects in 3D space. The applicant considers that Position data needs to accurately represent the model's position in space; any small error can lead to a shift in the model's position, affecting the realism and accuracy of the scene. Therefore, maintaining the accuracy of Position data is crucial. Furthermore, during rendering, Position data is used to calculate the vertex positions of the model. If Position data is compressed, the decompression process adds an extra computational burden, potentially reducing rendering performance. Directly using uncompressed Position data can reduce the computational load on the GPU and improve rendering efficiency. At the same time, the consistency of Position data is essential for the correct display of the scene. Compression and decompression processes may introduce errors, leading to inconsistencies in model positions and affecting the overall effect of the scene. Therefore, in this embodiment, Position data is not compressed in the data organization format of the mesh nodes but is stored directly to ensure that the Position data generated by the 3D scene data parser remains unchanged, thereby guaranteeing the accuracy and efficiency of scene rendering.

[0083] Step 140: Quantize and compress the scaling ratio data Scale and store it so that the 3D scene data parsing end can directly process the quantized and compressed scaling ratio data Scale.

[0084] Scale data primarily describes the scaling ratio of a 3D model or object along the X, Y, and Z axes. It's typically used to adjust the size of a 3D model or object. By changing the scaling ratio on the X, Y, and Z axes, the model can be enlarged or reduced to suit different scene requirements. In Level of Detail (LOD) layering, scale data can be used to control the scaling of models at different levels of detail. For example, at a greater view distance, when using a model with a lower LOD layer, adjusting the scale data can reduce the model's complexity, thereby improving rendering efficiency.

[0085] To reduce data volume, the data format for grid nodes proposed in this application performs quantization compression on the scale data to reduce storage space and transmission bandwidth requirements. Quantization compression is a data compression method that reduces data volume by reducing the number of bits required to represent the data. This method typically involves mapping continuous data values ​​to a set of discrete quantization levels, thereby reducing the storage space required for each data point.

[0086] Quantization compression is a lossy compression method that introduces quantization errors, leading to a loss of data precision. The magnitude of the quantization error is related to the number of quantization levels. However, the applicant considers that in 3D rendering scenarios, a certain degree of precision loss in scaling data (Scale) is tolerable. Furthermore, quantization compression can be implemented in hardware. Therefore, to achieve data pass-through and enable the graphics processing unit (GPU) to directly process the quantized and compressed data, this embodiment employs quantization compression to process the scaling data (Scale). This approach maintains the model scaling effect while reducing data volume, improving data processing efficiency, and supporting both data pass-through and direct processing by the GPU.

[0087] When stored, the quantized compressed scale data occupies 6 bytes in the data format of the grid node, representing the scaling ratio on the X, Y, and Z axes respectively.

[0088] Step 150: Compress the color data Color into RGBA32 bits and store it so that the 3D scene data parsing end can directly process the RGBA32-bit compressed color data Color;

[0089] The color data "Color" refers to the color information of each node, typically used to represent the color attributes of objects in a 3D scene. It provides color information for each node or primitive during rendering to achieve a realistic visual effect. In this embodiment, the color data "Color" is typically stored as a 32-bit unsigned integer (uint32) in RGBA (red, green, blue, transparency) format.

[0090] in:

[0091] R (Red): Red channel, 8 bits;

[0092] G (Green): Green channel, 8 bits;

[0093] B (Blue): Blue channel, 8 bits;

[0094] A (Alpha): Transparency channel, 8 bits;

[0095] Alpha refers to data used to describe the transparency of a 3D model or object. Alpha data is primarily used to adjust the transparency of a 3D model or object. By setting different alpha values, partial or complete transparency of the model can be achieved, thus simulating semi-transparent objects in the real world. During rendering, alpha data is used to control the model's blending mode. By adjusting the alpha value, the model can be blended with other objects in the scene, such as rendering semi-transparent materials like glass and water. Alpha data is usually stored in RGBA32 format along with other color channels (R, G, B). Compression can reduce the data volume and improve data processing efficiency while maintaining rendering quality.

[0096] In this embodiment, to reduce the amount of data, RGBA 32-bit compression is used for the color data. RGBA data is stored in a 32-bit compressed manner, which reduces data storage and transmission overhead while maintaining visual quality. RGBA 32-bit compression typically refers to encoding the color information of each pixel (including the four channels of red, green, blue, and transparency) into a 32-bit (4-byte) data format, with each pixel occupying 32 bits (4 bytes).

[0097] The RGBA 32-bit compression method enables high-fidelity 3D scene rendering and allows the compressed data to be directly accessed by the graphics processing unit (GPU), thus optimizing data storage and transmission efficiency.

[0098] Step 160: Normalize and compress the Rotation state data and store it so that the 3D scene data parsing end can directly process the normalized and compressed Rotation state data.

[0099] Rotation data refers to data used to describe the rotational state of a 3D model or object. Rotation data is primarily used to adjust the rotation angle and direction of a 3D model or object in 3D space. By setting different rotation parameters, the model can be rotated at any angle to adapt to different scene requirements. In animation and dynamic effects, rotation data can be used to control the rotational movement of the model. For example, it can be used to implement object rotation animations and character actions. Rotation data is typically stored as a Uint32 type in the mesh node data structure.

[0100] In this embodiment of the application, the rotation state data Rotation is subjected to special normalization compression. Normalization compression is a data compression method that reduces data storage space and transmission bandwidth by mapping data to a specific range (usually between 0 and 1). Moreover, the normalized compressed data can be directly called by the graphics processing unit (GPU) without the need for decompression in memory.

[0101] Normalized compression typically involves the following steps:

[0102] First, determine the data range, that is, determine the maximum and minimum values ​​of the original data. Then, perform a linear mapping, transforming the original data linearly to the range of 0 to 1. A common mapping formula is:

[0103] .

[0104] Finally, a quantization operation is performed to quantize the normalized floating-point data into integers, typically using 8-bit or 16-bit integers.

[0105] After the 3D scene data generation end generates and stores 3D scene data according to the above data structure, the 3D scene data parsing end fully utilizes the advantages of the mesh node data structure. It calls different levels of LOD (Level of Detail) layered data based on user needs. For example, for distant scenes, only higher-level LOD layered data is needed, as it contains only simple geometry and has a smaller amount of mesh node data. For close-up scenes, lower-level LOD layered data is needed, as it contains the highest detail of the original model, fully showcasing every detail of the 3D scene and containing a larger amount of data. Furthermore, through LOD layered calling and data block processing of the 3D scene data, only blocks near the viewpoint are loaded when the user's viewpoint moves, and the LOD level of the blocks is dynamically adjusted based on distance. This ensures real-time rendering of ultra-large scenes with limited resources while maintaining high rendering quality and performance.

[0106] Meanwhile, when the 3D scene data parsing end is parsing the data, different compression methods are used for different data of the mesh nodes at the 3D scene data generation end. These compression methods can all be read on the GPU through hardware. Therefore, in actual operation, the above-mentioned LOD layered data can be directly passed from the hard disk to the video memory of the image processing unit GPU. On the one hand, there is no need to decompress in memory. On the other hand, the image processing unit GPU can directly read the compressed position data, scale data, color data, and rotation data. While reducing storage space and transmission bandwidth, it can also ensure the timeliness of data processing, reduce the performance requirements of 3D rendering equipment, and improve rendering efficiency.

[0107] Furthermore, when processing the 3D scene data according to the preset LOD level and simplified algorithm, collision data (Collision) is also generated. Collision data is mainly used to detect whether objects in the scene collide, which is crucial for scenarios such as physics simulation, character movement, and object interaction. Through collision detection, dynamic responses between objects can be achieved, such as bouncing, sliding, and stopping.

[0108] In this embodiment, the collision data (Collision) and the LOD (Level of Detail) layering data used for rendering are stored separately. The collision data includes multiple second mesh nodes, wherein the second mesh nodes and the first mesh nodes adopt the same partitioning rule. That is, the collision data and the LOD layering data required for data rendering are consistent in spatial partitioning and are mutually aligned. The 3D scene is divided into multiple mesh nodes, and each mesh block contains not only the LOD layering data used for rendering but also the collision data used for collision detection. This design ensures the spatial consistency of the scene's visual representation and physical interaction.

[0109] In this embodiment, because the LOD layered data and collision data are spatially aligned, the same index and position information as the rendered data can be directly used during collision detection, eliminating the need for additional coordinate transformations or data lookups. This significantly improves the efficiency of collision detection. The alignment design ensures that the collision detection results are consistent with the visual representation, avoiding physical simulation errors or visual unnatural phenomena caused by data inconsistencies.

[0110] Table 3 shows the data structure of the Collision data file:

[0111] Table 3

[0112]

[0113] As shown in Table 3, the collision data includes a header and a data section. The header contains metadata information and several mesh headers. The data section contains several mesh data and BVH acceleration structures. Each mesh data contains vertex and triangle index data, and each mesh data uses the same partitioning rule as the first mesh node. Specifically, the vertex data in the mesh data contains x, y, and z values ​​for each vertex; each triangle data contains three index values ​​(starting from 0) for each face.

[0114] The BVH accelerated structure data is used for rapid collision testing, improving the efficiency and performance of collision detection. The generation of the BVH accelerated structure data is as follows: Figure 4 As shown, it includes:

[0115] Step 171: Construct a bounding box based on the vertex data and triangle data of the multiple MESH data, the bounding box including all geometric objects of the 3D scene;

[0116] The BVH acceleration structure includes intermediate nodes and leaf nodes.

[0117] Intermediate nodes: store bounding box information and location information for retrieval.

[0118] Leaf nodes: Store the starting index of the triangle, the number of triangles, and the bounding box information, used to retrieve triangle data.

[0119] When generating BVH accelerated structure data, the vertex data and triangle data of MESH data are first prepared, and the bounding box is constructed using the vertex data and triangle data.

[0120] Step 172: Traverse all the geometric objects and divide the geometric objects into geometric object groups according to a preset threshold;

[0121] This bounding box can completely contain all geometric objects in the scene. A binary tree can be used to recursively divide the geometric objects into smaller groups until each group contains a preset threshold number of geometric objects. This process is typically based on some partitioning strategy, such as uniform partitioning according to spatial location or partitioning according to the distribution density of the geometric objects.

[0122] Step 173: Construct intermediate nodes and leaf nodes based on the geometric object group;

[0123] For each group of geometric objects, construct either an intermediate node or a leaf node. An intermediate node contains the bounding box information of its child nodes and pointers to those child nodes. A leaf node contains the starting index of the triangle, the number of triangles, and their bounding box information.

[0124] Step 174: Generate the BVH acceleration structure based on the intermediate nodes and leaf nodes;

[0125] The constructed BVH tree is serialized into binary data for fast loading and use at runtime.

[0126] To further improve the efficiency of the BVH acceleration structure, the constructed BVH tree can be optimized. For example, the node partitioning strategy can be adjusted to minimize the tree depth, or the tree structure can be balanced so that each leaf node contains approximately the same number of geometric objects.

[0127] The above embodiments, on the one hand, ensure consistency between collision detection results and visual representation by aligning the mesh node partitioning rules in the Collision data with those in the LOD layered data, thus avoiding physical simulation errors or visual unnatural phenomena caused by data inconsistencies. On the other hand, by setting a BVH acceleration structure in the data portion of the Collision data, the rendering and collision detection of the 3D scene are accelerated.

[0128] Furthermore, in some embodiments, when processing the 3D scene data according to a preset LOD level and a simplification algorithm, environmental data is also generated. This environmental data (Environment) stores the background environment data of the 3D scene, including position data (Position), scaling data (Scale), color data (Color), and rotation data (Rotation). Similarly, to reduce the data volume, the position data (Position) is directly stored, and the environmental data (Environment) is also compressed accordingly. The compression method is similar to the compression method for the data in the `data` section. Specifically, this includes quantizing and compressing the scaling data (Scale) before storage, compressing the color data (Color) using RGBA 32-bit compression before storage, and normalizing and compressing the rotation data (Rotation) before storage. Optionally, the environmental data (Environment) may also include spherical harmonic data (Shcoef).

[0129] Furthermore, when processing the 3D scene data according to the preset LOD level and simplified algorithm, spherical harmonic data (Shcoef) is also generated. Shcoef is a method for representing and analyzing data that varies on a sphere. In the field of 3D graphics and rendering, spherical harmonic functions are commonly used to represent lighting and environment mapping. By decomposing lighting information into spherical harmonic coefficients, lighting data can be efficiently stored and transmitted, and lighting effects can be quickly calculated during rendering. Shcoef typically includes multiple spherical harmonic coefficients, each corresponding to a basis function in the spherical harmonic function.

[0130] In this application, the spherical harmonic data Shcoef is quantized and compressed, and the quantized and compressed spherical harmonic data Shcoef is stored together with the environmental data Environment.

[0131] It should be noted that the above embodiments only show the storage method and data structure of some 3D scene data, which may also include other data information, which will not be described in detail here. It can be compressed according to the ideas proposed in the above embodiments of this application, so as to reduce the amount of data through different compression methods while realizing LOD layered processing, realize data pass-through, and enable the graphics processing unit to directly process the compressed data without decompressing through memory.

[0132] Correspondingly, at the 3D scene data parsing end, it is necessary to parse the scene data generated by the 3D scene data generation end and render the 3D scene based on the parsed data. It should be noted that the 3D scene data parsing end includes an image processing unit and a non-volatile storage unit. The non-volatile storage unit stores the 3D scene data generated by the 3D scene data generation method. The scene data parsing end executes the following method: Figure 5 As shown:

[0133] Step 210: The image processing unit directly loads the 3D scene data stored on the non-volatile storage unit into the video memory of the image processing unit;

[0134] In this embodiment, a data pass-through method is adopted, where LOD layered data can be directly passed from the hard disk to the video memory. Since the GPU instruction set usually contains instructions specifically for compressed data processing, these instructions can optimize the decoding process and reduce GPU computational resource consumption. Therefore, in this embodiment, the LOD layered data is compressed according to type using a compression method supported by the GPU instruction set at the 3D scene data processing end. This eliminates the need to decompress the file in memory and allows the compressed LOD layered data to be directly passed through to the GPU's video memory for processing by the GPU, greatly improving rendering efficiency and being very friendly to performance-limited devices.

[0135] Step 220: Parse the position data Position, the compressed scaling data Scale, the compressed color data Color, and the compressed rotation data Rotation;

[0136] For different types of LOD layered data, the GPU processes them directly based on whether they are compressed and how they are compressed. This method can support streaming loading of ultra-large scenes and ensure high-quality rendering effects under various device and resource limitations.

[0137] In some embodiments, a processing method is also proposed for the scaling ratio data, such as... Figure 6 As shown, it includes:

[0138] Step 231: Obtain the quantization value corresponding to the compressed scaling ratio data Scale;

[0139] In 3D scene data processing, the scaling factor data (Scale) is quantized and compressed, meaning that the original continuous Scale values ​​are mapped to a smaller range of integers. In practical applications, these integers need to be restored to continuous Scale values ​​for accurate scaling operations.

[0140] Read the quantized values ​​of the scaling data (Scale) from the file; these values ​​are in integer form.

[0141] Step 232: Convert the quantized value into an integer value;

[0142] Convert the read byte data into a 32-bit integer.

[0143] Step 233: Determine the scaling ratio data Scale by interpolation based on the integer value and the maximum and minimum values ​​corresponding to the preset scaling ratio data Scale.

[0144] The actual scale value is calculated using an interpolation formula based on the minimum and maximum scale values ​​provided in the meta.lcc file. This interpolation formula is typically based on linear interpolation or other interpolation methods. The interpolation process ensures that even after data has been quantized and compressed, a scale value close to the original precision can still be recovered, thus guaranteeing the accuracy of rendering and operations.

[0145] In other embodiments, such as Figure 7 As shown in the embodiments of this application, a three-dimensional scene rendering device is also proposed. The three-dimensional scene rendering device is used to run the three-dimensional scene data generation method or the three-dimensional scene data parsing method proposed in the above embodiments. The three-dimensional scene rendering device may include: a processor 402, a memory 406, a communication interface 404, and a communication bus 408.

[0146] The processor 402, memory 406, and communication interface 404 communicate with each other via communication bus 408. The memory 406 stores at least one program 410, which causes the processor 402 to execute steps related to the three-dimensional scene data generation method or the three-dimensional scene data parsing method proposed in the embodiments of this application.

[0147] Specifically, program 410 may include program code, which includes computer-executable instructions.

[0148] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the UAV direction-determining device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0149] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0150] Specifically, program 410 can be called by processor 402 to cause the 3D scene rendering device to execute the 3D scene data generation method or 3D scene data parsing method proposed in the embodiments of this application, which will not be described in detail here.

[0151] This application also provides a computer-readable storage medium storing executable instructions. When the executable instructions are run on a 3D scene rendering device, the 3D scene rendering device performs the 3D scene data generation method or the 3D scene data parsing method provided in any of the above embodiments.

[0152] This application also provides a three-dimensional scene rendering program, which is used to execute the three-dimensional scene data generation method or the three-dimensional scene data parsing method provided in the above embodiments.

[0153] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of this application are not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0154] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0155] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of this application, various features of the embodiments of this application are sometimes grouped together into a single embodiment, figure, or description thereof.

[0156] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying abstract and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying abstract and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0157] It should be noted that the above embodiments are illustrative of this application and not restrictive, and those skilled in the art can devise alternative embodiments without departing from the scope. Unless otherwise specified, the steps in the above embodiments should not be construed as limiting the order of execution.

Claims

1. A method of generating three-dimensional scene data, characterized by, The method is applied to a three-dimensional scene data generation end which is adapted to a three-dimensional scene data analysis end, and comprises the following steps: Obtaining three-dimensional scene data; Processing the three-dimensional scene data according to a preset LOD level number and a simplification algorithm to generate corresponding LOD layered data, wherein the LOD layered data comprises a plurality of first mesh nodes, and each first mesh node comprises position data Position, scale data Scale, color data Color and rotation state data Rotation; Directly storing the position data Position so that the position data generated by the three-dimensional scene data analysis end remains unchanged; Storing the scale data Scale after data quantization compression so that the three-dimensional scene data analysis end directly processes the scale data Scale after data quantization compression; Storing the color data Color after RGBA32-bit compression so that the three-dimensional scene data analysis end directly processes the color data Color after RGBA32-bit compression; Storing the rotation state data Rotation after normalization compression so that the three-dimensional scene data analysis end directly processes the rotation state data Rotation after normalization compression.

2. The method of claim 1, wherein, The method further comprises the following steps: Processing the three-dimensional scene data according to a preset LOD level number and a simplification algorithm to generate collision data Collision, wherein the collision data Collision comprises a plurality of second mesh nodes, and the second mesh nodes and the first mesh nodes adopt the same division rule.

3. The method of claim 2, wherein, The collision data comprises a header part and a data part. The data part comprises a plurality of MESH data and BVH acceleration structure, wherein the MESH data adopts the same division rule as the first mesh nodes.

4. The method of claim 3, wherein, The MESH data comprises vertex data and triangular face data. The method further comprises the following steps: Constructing a bounding box according to the vertex data and the triangular face data of the plurality of MESH data, wherein the bounding box comprises all geometric objects of the three-dimensional scene; Dividing the geometric objects into geometric object groups according to a preset threshold value by traversing all the geometric objects; Constructing intermediate nodes and leaf nodes according to the geometric object groups; Generating the BVH acceleration structure according to the intermediate nodes and the leaf nodes.

5. The method of claim 1, wherein, The method further comprises the following steps: Processing the three-dimensional scene data according to a preset LOD level number and a simplification algorithm to generate environment data Environment, wherein the environment data Environment comprises position data Position, scale data Scale, color data Color and rotation state data Rotation; Directly storing the position data Position; Storing the scale data Scale after data quantization compression; Storing the color data Color after RGBA32-bit compression; The rotation state data Rotation is normalized and compressed and then stored.

6. The method of claim 5, wherein, The method further comprises: According to the preset LOD level number and simplification algorithm, the three-dimensional scene data is processed to generate spherical harmonic data Shcoef. The spherical harmonic data Shcoef is data-quantized and compressed, and the data-quantized and compressed spherical harmonic data Shcoef is stored together with the environment data Environment.

7. A method for parsing three-dimensional scene data, characterized in that, The three-dimensional scene data analysis end comprises an image processing unit and a non-volatile storage unit, and the non-volatile storage unit stores three-dimensional scene data generated by the three-dimensional scene data generation method of any one of claims 1-6, and the method comprises: The image processing unit directly loads the three-dimensional scene data stored in the non-volatile storage unit into the video memory of the image processing unit. The position data Position, the compressed scale data Scale, the compressed color data Color, and the compressed rotation state data Rotation are analyzed.

8. The method of claim 7, wherein, The method further comprises: The quantized value corresponding to the compressed scale data Scale is obtained. The quantized value is converted into an integer value. According to the integer value and the preset maximum value and minimum value corresponding to the scale data Scale, the scale data Scale is determined by an interpolation method.

9. A three-dimensional scene rendering device, characterized by It comprises: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one program, and the program causes the processor to execute the three-dimensional scene data generation method of any one of claims 1-6; or, execute the three-dimensional scene data analysis method of any one of claims 7-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, and the program runs on the three-dimensional scene rendering device, so that the three-dimensional scene rendering device executes the three-dimensional scene data generation method of any one of claims 1-6; or, executes the three-dimensional scene data analysis method of any one of claims 7-8.

Images