Spatially-serialized point cloud data processing method, system, and device

Abstract

The application provides a point cloud data processing method, system and device based on spatial serialization, the method comprising: sorting point cloud data by using a six-way standard sequence sorting method to obtain point cloud data sequences; dynamically and automatically selecting focal point blocks in the sequenced data according to a view cone and screen projection error, and dynamically adjusting the LOD level of each focal point block to obtain sequenced LOD data; after the sequenced LOD data is managed by VIBO block management, performing triangle optimization reconstruction; using a UV space remapping algorithm, performing patch and texture alignment processing on the point cloud surface after triangle optimization reconstruction, and performing multi-format texture packing to complete rendering output. The application reduces unnecessary calculation and memory access through dynamic adjustment of the LOD of the focal point block and VIBO block caching. Through dynamic LOD adjustment and optimized triangle reconstruction technology, combined with efficient texture packing and decompression, the rendering frame rate can be significantly improved while details are maintained.

Description

Point cloud data processing methods, systems, and devices based on spatial serialization Technical Field

[0001] This invention belongs to the field of point cloud data visualization and processing technology, specifically relating to a point cloud data processing method, system, and device based on spatial serialization. Background Technology

[0002] With the continuous development of WebGL technology, GPU-accelerated rendering has become an indispensable core technology in online visualization. Especially in applications involving large-scale point cloud and mesh hybrid scenes, WebGL and related GPU acceleration technologies provide powerful support for achieving efficient and real-time graphics rendering. However, with the increasing complexity of application scenarios and the rapid growth of data scale, traditional rendering methods have exposed the following three major bottlenecks, severely restricting the performance and user experience of large-scale point cloud and mesh hybrid scenes. 1. Traditional point cloud scanning methods often process point clouds in a random order. This method lacks spatial optimization, resulting in low efficiency when performing point cloud culling and rearrangement operations. Especially in large-scale point cloud data processing, the lack of spatial indexing and structured data storage leads to huge overhead for I / O operations and CPU processing, thus affecting the real-time performance and overall rendering performance. 2. Bottleneck of object-based vertex / index cache: In traditional graphics rendering, to improve rendering efficiency, vertex and index caches are usually stored in an object-based manner. While this approach can optimize to some extent, it still incurs high costs when frequent reassembly and batching are required. This approach cannot effectively handle multiple objects and complex relationships in large-scale scenes, leading to unstable frame rates during rendering and a limited user experience. 3. Bottleneck of a single texture compression format: Current rendering methods typically use a single texture compression format (such as JPEG, PNG, etc.), resulting in poor compatibility across different browsers, GPUs, or platforms. The texture decompression process may block the main thread, causing noticeable interactive delays during rendering. This bottleneck is particularly critical for high-quality rendering and large-scale scene visualization, becoming a key factor limiting system performance. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a point cloud data processing method, system, and device based on spatial serialization. By introducing key technologies such as spatially serialized point clouds, vertex / index block caching, right-handed triangle rearrangement, private data structures, multi-format texture packages and GPU decompression, and end-to-end local GPU computation, it effectively overcomes the bottlenecks in rendering large-scale point cloud and mesh hybrid scenes, thereby significantly improving graphics rendering efficiency, stability, and interactive performance.

[0004] In a first aspect, the present invention proposes a point cloud data processing method based on spatial serialization, comprising:

[0005] The point cloud data is sorted using a six-way standard sequence sorting method to obtain a point cloud data sequence;

[0006] Based on the error between the view frustum and the screen projection, the focus block in the serialized data is automatically selected dynamically, and the LOD level of each focus block is dynamically adjusted to obtain the serialized LOD data.

[0007] After the serialized LOD data is managed by VIBO blocks, it is reconstructed using triangle optimization.

[0008] The UV space remapping algorithm is used to align the facets and textures of the point cloud surface after triangle optimization and reconstruction, and then package the textures in multiple formats to complete the rendering output.

[0009] This invention reduces unnecessary computation and memory access, optimizing loading time by dynamically adjusting the LOD of focus blocks and using VIBO block caching. Furthermore, through dynamic LOD adjustment and optimized triangle reconstruction technology, combined with efficient texture packing and decompression, it significantly improves rendering frame rate while preserving detail. The use of UV space remapping technology further ensures accurate texture alignment, resulting in more realistic and detailed rendering results.

[0010] Preferably, the step of sorting the point cloud data using a six-way standard sequence sorting method to generate a circular sequence index table includes:

[0011] Divide the point cloud data into multiple buckets;

[0012] Sort the data in each bucket according to the selected sort key;

[0013] The sorted data in the buckets are sequentially written into a new array according to the six-way standard sequence sorting method to form a globally ordered point cloud data sequence.

[0014] By adopting a six-way standard sequence sorting method, the sorting of point cloud data not only improves data processing efficiency, but also optimizes memory access patterns and storage efficiency, providing a solid foundation for subsequent point cloud data processing and rendering optimization.

[0015] Furthermore, in order to reduce latency and improve the efficiency of parallel processing, the present invention also constructs a circular sequence index table; the circular sequence index table is used to perform circular traversal or local skip reading of point cloud data.

[0016] Preferably, the step of dynamically and automatically selecting focal blocks in the serialized data based on the error between the view frustum and the screen projection, and dynamically adjusting the LOD level of each focal block, includes:

[0017] Calculate the projection error of each region within the view frustum, and select the region with an error greater than a preset value as the focal block;

[0018] Based on the position of each focal block, projection error, and changes in the view frustum, determine the required LOD level for each focal block;

[0019] The LOD level is updated in real time based on the user's movement or changes in perspective.

[0020] Preferably, the serialized LOD data is managed in VIBO blocks, including:

[0021] The point cloud data is divided into blocks according to the fixed vertex limit of each focal block;

[0022] Each data block after being processed is stored continuously at different locations in the Vertex Buffer, and the position and offset of each block in the Vertex Buffer are recorded in the Index Buffer.

[0023] Preferably, the triangle optimization reconstruction includes:

[0024] Rearrange the triangle vertex indices in the serialized data according to the preset drawing order;

[0025] The normal vector of the triangle is calculated using the rearranged vertex order;

[0026] Determine whether the normal of each triangle points to a predetermined direction. If the normal direction is opposite to the reference direction, reverse the vertex order. If the normal direction is consistent with the reference direction, generate optimized triangle serialization data.

[0027] In this case, the winding direction of the triangle is the same as the normal direction.

[0028] Preferably, the step of using a UV space remapping algorithm to align the surface of the point cloud after triangle optimization and reconstruction with textures includes:

[0029] The point cloud surface is divided into multiple triangular patches, and the vertices of each triangular patch and their associated UV coordinates are determined.

[0030] Based on the vertex coordinates of the triangular facets, the texture mapping of each triangular facet is adjusted using a transformation matrix to ensure that the vertices of each triangular facet and their associated UV coordinates are correctly aligned.

[0031] Preferably, the step of packaging multi-format textures to complete rendering output further includes:

[0032] Encryption technology and metadata are used to encrypt the aligned point cloud data; the metadata includes data version information, block size information, and a compression table.

[0033] During the offline phase, the encrypted data is converted into different texture formats and packaged.

[0034] The packaged data supports real-time decompression on the GPU and WebGL rendering output.

[0035] Secondly, based on the same inventive concept, this invention also proposes a system for a point cloud data processing method based on spatial serialization, the system comprising:

[0036] The first processing module uses a six-way standard sequence sorting method to sort the point cloud data and obtain a point cloud data sequence.

[0037] The second processing module dynamically and automatically selects the focal block in the serialized data based on the error between the view frustum and the screen projection, and dynamically adjusts the LOD level of each focal block to obtain the serialized LOD data.

[0038] The optimization module performs triangle optimization reconstruction on the serialized LOD data after VIBO block management;

[0039] The third processing module uses the UV space remapping algorithm to perform patch and texture alignment processing on the point cloud surface after triangle optimization and reconstruction.

[0040] Additionally, the output module packages multi-format textures to complete the rendering output.

[0041] Thirdly, based on the same inventive concept, the present invention also proposes a point cloud data processing device based on spatial serialization, the point cloud data processing device comprising: a memory and at least one processor, wherein the memory stores instructions;

[0042] The at least one processor invokes the instructions in the memory to cause the spatially serialized point cloud data processing device to perform the spatially serialized point cloud data processing method as described in the first aspect.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] This invention proposes a point cloud data processing method, system, and device based on spatial serialization. It employs a six-directional standard sequence sorting method to sort point cloud data, generating a globally ordered data sequence and optimizing memory access patterns and storage efficiency. Based on view frustum and screen projection errors, it dynamically selects focal blocks and adjusts their Level of Detail (LOD) level. This allows for real-time LOD updates based on user movement or viewpoint changes, ensuring the system can efficiently process the areas to be rendered during the rendering process, avoiding excessive computation of useless data and memory consumption. Through VIBO block management and triangle optimization reconstruction technology, redundancy in vertex indices is reduced, making the rendering data more concise and efficient. Furthermore, the uniformity of triangle normal directions ensures the accuracy of the rendering results, further improving rendering quality. A UV spatial remapping algorithm is used to align the textures on the point cloud surface, ensuring the accuracy of texture mapping and avoiding potential texture misalignment issues during rendering. Furthermore, this invention effectively reduces unnecessary computation and memory access through dynamic LOD adjustment of focal blocks and VIBO block caching technology, optimizing loading time and significantly improving the rendering frame rate. Furthermore, encryption technology is used to encrypt the processed point cloud data, increasing data security. In addition, different texture formats are used to package the data, enabling real-time decompression on the GPU and support for WebGL rendering, improving system compatibility and scalability.

[0045] In summary, by optimizing the sorting, LOD adjustment, triangle reconstruction, texture alignment, and data packaging processes in point cloud data processing, this invention not only significantly improves the efficiency and quality of point cloud rendering but also enhances the system's security and compatibility, providing strong technical support for practical applications. Attached Figure Description

[0046] Figure 1 is a flowchart of the point cloud data processing method based on spatial serialization according to the present invention.

[0047] Figure 2 is a system of point cloud data processing method based on spatial serialization as shown in an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0049] Example 1: As shown in Figure 1, this invention proposes a point cloud data processing method based on spatial serialization, including:

[0050] Step 1: Sort the point cloud data using the six-way standard sequence sorting method to obtain the point cloud data sequence;

[0051] Preferably, the step of sorting the point cloud data using a six-way standard sequence sorting method to generate a circular sequence index table includes:

[0052] 1.1 Divide the point cloud data into multiple buckets;

[0053] First, the original point cloud dataset is obtained. This dataset contains a large number of 3D coordinate points, each containing its 3D position coordinates (x, y, z) in space. To improve sorting efficiency, the point cloud data is first divided into multiple buckets according to spatial regions. The number of points in each bucket is moderate, facilitating the sorting operation.

[0054] 1.2 Sort the data in each bucket according to the selected sort key;

[0055] Optionally, the data within the bucket can be sorted in ascending or descending order based on a pre-selected sort key (such as one of the x, y, or z coordinates). Common and efficient sorting algorithms such as quicksort and mergesort can be selected.

[0056] After sorting within each bucket, a six-directional standard sequence sorting method is used to globally sort the data in all buckets. Specifically, the six-directional standard sequence sorting method assigns a sorting order to each point according to the six directions in space, as shown in the table below (front, back, left, right, top, bottom). The sorted data in each bucket is sequentially written into a new array according to the rules of the six-directional standard sequence sorting method, forming a globally ordered point cloud data sequence.

[0057]

[0058] It should be noted that by employing the six-way standard sequence sorting method, the sorting of point cloud data not only improves data processing efficiency but also optimizes memory access patterns and storage efficiency, providing a solid foundation for subsequent point cloud data processing and rendering optimization. In practical applications, in addition to sorting, the sorted data can be dynamically adjusted according to different needs. For example, during rendering, different focal blocks can be dynamically selected and their LOD levels adjusted based on different regions of the view frustum. In this case, the sorted point cloud data can quickly locate the focal blocks through the generated circular sequence index table, improving rendering efficiency and quality.

[0059] Furthermore, after global sorting, a circular sequence index table is generated to further optimize data access and improve processing efficiency. This index table represents the order of the sorted point cloud data, enabling circular traversal or local skip operations to reduce latency and improve parallel processing efficiency. Specifically, an index table is created for the globally ordered point cloud data sequence, recording the position of each point in the array, and defining a circular sequence index value for each point, making data access more efficient.

[0060] Step 2: Dynamically and automatically select the focal blocks in the serialized data based on the error between the view frustum and the screen projection, and dynamically adjust the LOD level of each focal block to obtain the serialized LOD data;

[0061] In this embodiment, the dynamic focus block selection and LOD level adjustment method based on view frustum and screen projection error proposed in this invention aims to improve the display effect and computational efficiency during point cloud data rendering. By dynamically adjusting the level of detail (LOD) of each focus block, rendering quality and performance can be optimized according to changes in the user's viewing angle and screen projection error. The specific implementation steps are as follows:

[0062] 1) Calculate the projection error of each region within the view frustum:

[0063] First, acquire the point cloud data within the current view frustum. The view frustum is a spatial region determined by the camera's perspective; all point cloud data within this region needs to be calculated and appropriately rendered. For each focal block within the view frustum, calculate its projection error on the screen. The projection error represents the difference between the actual position of the point cloud data projected onto the screen and the expected position. The magnitude of the error reflects the accuracy of the point cloud data display at the current viewpoint.

[0064] The error value is usually determined by the difference between the actual screen projection position and the ideal position of the point cloud data, calculated using a three-dimensional transformation matrix.

[0065] Preferably, a preset value is set. When the projection error of a certain area is greater than the threshold, it is considered that the display accuracy of that area needs to be improved, and that area is selected as the focus block.

[0066] 2) Focus Block Selection: Based on the calculated projection error, areas with errors greater than a preset value are selected as focus blocks. These focus blocks are assigned a higher LOD level to ensure users can see more details, thereby improving the visual experience. The selection of focus blocks depends not only on the error but also on factors such as the user's current position, viewing angle, and screen resolution.

[0067] 3) Determine the required LOD level for each focal block: For each selected focal block, dynamically determine its required LOD level based on its position, projection error, and changes in the view frustum. Specifically, if the focal block is at the center of the view frustum and has a large error, a higher LOD level is required; conversely, a lower LOD level can be used.

[0068] It should be noted that the closer the focal block is to the center of the view frustum, the higher the required LOD level. Furthermore, if the projection error is large, more detail is needed to compensate for the error, thus improving the LOD level. As the user's viewing angle changes, the position and extent of the view frustum also change, which in turn affects the selection of the focal block and the adjustment of the LOD level.

[0069] 4) Real-time LOD level updates: The LOD level of each focus block is adjusted in real time based on the user's movement or changes in perspective. For example, when the user approaches a focus block, the level of detail displayed on the screen increases to ensure that the rendering effect of that focus block is optimal; while when the user moves away from the focus block, the LOD level is automatically reduced to optimize computational efficiency and reduce resource consumption.

[0070] For example, user movement: when a user moves in a certain direction, the position of the focus block changes relatively, and the system needs to recalculate the projection error based on the new perspective information and update the corresponding LOD level.

[0071] Viewpoint changes: If the user rotates the viewpoint, the relative position of the focal block and the projection error will also change. The system should detect viewpoint changes in real time and dynamically adjust the LOD level to ensure the smoothness and accuracy of point cloud rendering.

[0072] 5) Finally, generate the serialized LOD data:

[0073] After completing the focus block selection and LOD level adjustment, the system serializes the updated point cloud data in LOD format and stores it in a data format that can be quickly accessed by the subsequent rendering engine. This data structure contains point cloud data at different LOD levels and is optimized according to the view frustum to ensure an efficient rendering and loading process.

[0074] In practical applications, especially in scenarios such as virtual reality (VR) or augmented reality (AR), the user's perspective changes very frequently, making real-time dynamic adjustment at the LOD level particularly important. The method described in this embodiment ensures that regardless of how the user moves or rotates the viewpoint, the rendered point cloud data can automatically adjust its level of detail as needed, thus maintaining a smooth visual effect while avoiding performance bottlenecks caused by excessive detail.

[0075] Step 3: After the serialized LOD data is managed by VIBO blocks, triangle optimization reconstruction is performed;

[0076] This embodiment proposes a scheme that combines VIBO (Vertex Index Buffer Object) block management in the process of triangle optimization and reconstruction of LOD data. By processing point cloud data in blocks, sorting vertices and optimizing normal directions, the performance and accuracy of point cloud rendering are improved. Especially in highly dynamic and complex scenes, it can provide more stable and smooth visual effects.

[0077] Before performing triangle optimization and reconstruction, the serialized LOD data is first managed using VIBO blocks to facilitate more efficient storage and access. The specific process is as follows:

[0078] The point cloud data is divided into blocks according to a fixed vertex limit for each focal block. It should be noted that the number of vertices in each focal block cannot exceed the preset maximum value to avoid memory overflow or access delay caused by excessively large blocks.

[0079] Each data block, after being processed into chunks, such as vertex coordinates, texture coordinates, and normals, is stored contiguously at different locations in the VertexBuffer. The position and offset of each chunk within the VertexBuffer are recorded in the Index Buffer. Index information is created for each data block in the Index Buffer, recording its starting position and offset within the Vertex Buffer. The Index Buffer allows for precise positioning of each data block, enabling efficient loading of the required vertex data during rendering.

[0080] Preferably, the triangle optimization reconstruction includes:

[0081] The triangle vertex indices are rearranged according to a preset drawing order to ensure that the vertices are arranged in a certain order to conform to the rendering specifications in computer graphics; the preset drawing order is usually counterclockwise or clockwise.

[0082] Based on the rearranged vertex order, calculate the normal vector for each triangle. The normal vector is obtained by the cross product of the vertices, and it describes the outward direction of the triangle's face.

[0083] The algorithm determines whether the normal of each triangle points in a predetermined direction. If the normal direction is opposite to the reference direction, the vertex order is reversed. By reversing the vertex order, the normal direction is corrected, ensuring the correct display of the triangles during rendering. The operation of reversing the vertex order can be achieved by swapping the order of the vertices. If the normal direction is consistent with the reference direction, optimized triangle serialization data is generated.

[0084] In the optimized triangle data, ensure that the winding direction (vertices' order) of each triangle is consistent with the normal direction. Typically, counter-clockwise vertex order corresponds to the outer normal, while clockwise vertex order corresponds to the inner normal. By unifying the winding direction, rendering issues caused by inconsistencies between the normal and vertex order can be avoided.

[0085] By following the steps above, we can ensure that each triangle in the LOD data is displayed correctly during rendering and optimize performance. The VIBO block management method effectively organizes data blocks and improves memory access efficiency, while the triangle optimization and reconstruction process ensures the consistency of vertex order and normal direction, thereby avoiding potential visual errors and wasted computational resources during rendering.

[0086] Step 4: Using the UV space remapping algorithm, the surface of the point cloud after triangle optimization and reconstruction is aligned with the texture, and multi-format textures are packaged to complete the rendering output.

[0087] This embodiment proposes a point cloud surface texture alignment method based on the UV spatial remapping algorithm, and combines it with multi-format texture packaging technology to complete the rendering output of point cloud data. This scheme ensures data security, storage efficiency, and real-time rendering effects by optimizing and aligning the textures on the point cloud surface, and by employing encryption and data packaging technologies.

[0088] First, the point cloud surface is segmented into multiple triangular patches to ensure that the geometry and texture mapping of each patch can be processed independently. This process is as follows: Based on the topological structure of the point cloud data, a triangulation algorithm is applied to divide the point cloud surface into multiple interconnected triangular patches. Each patch consists of three vertices, and each vertex of the patch has an associated UV coordinate (texture coordinate). In each segmented triangular patch, its three vertices are determined, and a corresponding UV coordinate is assigned to each vertex. UV coordinates are a two-dimensional coordinate system used to map textures to the triangular face.

[0089] To ensure that the texture of each triangular facet is correctly aligned to the surface, texture mapping needs to be adjusted based on the vertex positions and transformation matrices of the triangles. Preferably, a corresponding transformation matrix is ​​generated based on the geometric transformations of the point cloud data (such as rotation, scaling, and translation). The transformation matrix is ​​used to adjust the texture mapping of each triangular facet so that it correctly conforms to the shape of the point cloud surface. Combined with the transformation matrix, the UV coordinates of each triangular facet are adjusted so that they correspond to the vertex coordinates of the facet in UV space. This step ensures that the texture mapping of each triangular facet is perfectly aligned with its geometry. After texture alignment, texture packaging and data encryption are performed to ensure efficient rendering on different hardware platforms.

[0090] To ensure data security and provide necessary information for the subsequent rendering process, this embodiment employs encryption technology and metadata processing methods.

[0091] Specifically, the point cloud data after UV spatial remapping is encrypted to ensure that the data is not tampered with during transmission and storage. For example, 128-bit identifiers and AES-GCM encryption technology are used to encrypt the data. The encryption process combines elements such as symmetric keys, initialization vectors (Nonce), additional verification data (AD), and plaintext data (P) to ensure the confidentiality, integrity, and security of the data.

[0092] Simultaneously, metadata is embedded in the encrypted data, primarily including: Data version information: used to identify the version of the dataset for subsequent updates and management; Block size information: records the size of each data block to ensure correct data positioning during decryption and rendering. Compression table: records the data compression and decompression methods to facilitate efficient data access during real-time decompression and rendering on the GPU. The specific data structure is defined as follows:

[0093]

[0094] Preferably, in this embodiment, the encryption steps are as follows:

[0095] Initialization phase:

[0096] K: Generates a symmetric key (128, 192, or 256 bits). This key is used to encrypt and decrypt data and is the core of the AES-GCM encryption process.

[0097] IV: Generates a unique initialization vector (Nonce), 12 bytes is recommended. The uniqueness of the IV ensures that each encryption produces a different ciphertext, even if the same plaintext is encrypted.

[0098] Encryption process:

[0099] P (plaintext): The original data that needs to be encrypted.

[0100] AD (Additional Authentication Data): Additional information beyond the plaintext data (such as message headers, version numbers, etc.). Although this data is transmitted in plaintext, it participates in encryption authentication to ensure integrity.

[0101] Generate encrypted text and authentication tags:

[0102] C (ciphertext): The plaintext P is encrypted using the AES-GCM algorithm, and the ciphertext C is output.

[0103] Tag (Authentication Tag): An authentication tag generated by the AES-GCM algorithm, used to verify the integrity and authenticity of data.

[0104] Hash value:

[0105] H: The hash value of the data calculated using SHA-256. This value is used to provide additional metadata integrity verification, ensuring that the data has not been tampered with.

[0106] When encrypting data using the AES-GCM algorithm, additional metadata (such as version, block size, compression table, etc.) ensures the integrity of the data during storage and transmission.

[0107] Preferably, after data security protection is completed, in order to be compatible with different platform devices, the encrypted data is converted into different texture formats and packaged during the offline stage;

[0108] For example, the available texture formats are:

[0109] JPEG: Suitable for scenarios that require a high compression ratio but allow for some loss of image quality; typically used for images.

[0110] PNG: Lossless compression, suitable for textures where image quality needs to be maintained.

[0111] WebP: A more modern format that offers better compression ratios than JPEG, while also supporting transparency.

[0112] Basis: A texture format specifically optimized for mobile and web platforms, capable of efficient decoding on multiple platforms and supporting efficient multi-level progressive loading.

[0113] During client loading, the appropriate format is selected based on the platform's graphics card support and system characteristics. Preferably, the client automatically detects supported extension technologies (such as EXT_texture_compression_bptc, ​​s3tc, etc2) to determine the texture formats supported by the current device. The client intelligently selects the optimal texture format based on the following factors:

[0114] Formula 1: Used to calculate the number of bytes stored in video memory for a certain texture, i.e., video memory usage:

[0115] ;

[0116] W and H are the width and height (in pixels) of the texture or video frame.

[0117] Formula 2: Used to select the optimal texture format, taking into account memory usage, decoding / transcoding time, and trade-off factors:

[0118] ;

[0119] The video memory usage f is the video memory usage of the current format f calculated according to Formula 1.

[0120] Decoding time f is the time (in milliseconds) for decoding or transcoding a texture, and it can measure the overhead of loading and rendering.

[0121] λ is a tradeoff coefficient used to balance the impact of video memory usage and decoding time, and it is adjusted according to actual needs.

[0122] Through calculation, the client will select the best texture format with the lowest video memory cost and the shortest decoding time, thereby ensuring efficient loading and rendering.

[0123] In traditional texture loading processes, compressed texture formats (such as Basis) typically require CPU decompression before being transferred to the GPU. This not only increases the CPU load but also leads to significant data transfer between the CPU and GPU. To avoid this problem, this embodiment proposes a GPU-side real-time decompression technique. By directly handling the texture decompression process with the GPU, texture data can be decoded in real-time on the GPU into a format more suitable for the current hardware (such as ASTC or ETC) and directly written into the GPU texture object. The specific implementation is as follows: In the front-end or offline stage, the texture is compressed into Basis format using a compression tool (such as Basis Universal); in the application, the texture decompression process is implemented using GLSL shaders, i.e., loading the Basis format texture data into GPU memory. A decoding shader written in GLSL decodes the compressed texture data into ASTC or ETC format in real-time. The decoded texture is then directly written into the GPU texture object, ready for rendering.

[0124] To achieve efficient and stable WebGL rendering output, this embodiment will optimize PBR materials and rasterization technology through the following specific process to ensure that the rendering time of each frame is less than 20 milliseconds and to provide high-quality visual effects.

[0125] Before rendering begins, initialization is performed, including loading rendering resources and setting the WebGL context. Triangle data is converted into pixels through rasterization, and the color, depth, and other information of each pixel are processed. Using a PBR material model, including parameters such as diffuse, specular, roughness, metallicity, and ambient occlusion, the final color value of each pixel is calculated. In the fragment shader, the lighting effects of each pixel are calculated based on scene light information and object material properties. PBR rendering calculations include ambient light, diffuse light, and reflected light to ensure a more realistic visual effect. In PBR, BRDF is used to describe the reflection characteristics of light. High-quality reflection effects are generated by calculating BRDF in real time. Rendering is kept smooth through real-time monitoring, LOD (Level of Detail), and deferred rendering techniques, and resource allocation is dynamically adjusted according to the load to ensure rendering efficiency. Rendering results are output, visual effects are optimized, and performance is continuously evaluated and adjusted.

[0126] In summary, this invention reduces unnecessary computation and memory access, optimizing loading time through dynamic LOD adjustment of focus blocks and VIBO block caching. Furthermore, the dynamic LOD adjustment and optimized triangle reconstruction technology, combined with efficient texture packing and decompression, significantly improves rendering frame rate while preserving detail. The further adoption of UV space remapping technology ensures accurate texture alignment, resulting in more realistic and detailed rendering results.

[0127] Example 2, as shown in Figure 2, the present invention also proposes a system for point cloud data processing based on spatial serialization, the system comprising:

[0128] The first processing module uses a six-way standard sequence sorting method to sort the point cloud data and obtain a point cloud data sequence.

[0129] The second processing module dynamically and automatically selects the focal block in the serialized data based on the error between the view frustum and the screen projection, and dynamically adjusts the LOD level of each focal block to obtain the serialized LOD data.

[0130] The optimization module performs triangle optimization reconstruction on the serialized LOD data after VIBO block management;

[0131] The third processing module uses the UV space remapping algorithm to perform patch and texture alignment processing on the point cloud surface after triangle optimization and reconstruction.

[0132] Additionally, the output module packages multi-format textures to complete the rendering output.

[0133] The system using the point cloud data processing method of the present invention can efficiently render complex point cloud scenes on different hardware platforms, improving rendering performance while ensuring the quality of visual effects.

[0134] In embodiment three, the present invention also proposes a point cloud data processing device based on spatial serialization, the point cloud data processing device comprising: a memory and at least one processor, wherein the memory stores instructions;

[0135] The at least one processor invokes the instructions in the memory to cause the spatially serialized point cloud data processing device to perform the spatially serialized point cloud data processing method as described in the first aspect.

[0136] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0137] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0138] Although the description of the invention has been given in conjunction with the specific embodiments described above, it will be apparent to those skilled in the art that many substitutions, modifications, and variations can be made based on the foregoing. Therefore, all such substitutions, modifications, and variations are included within the spirit and scope of the appended claims.

Claims

1. A point cloud data processing method based on spatial serialization, characterized in that, include: The point cloud data is sorted using a six-way standard sequence sorting method to obtain a point cloud data sequence. Based on the error between the view frustum and the screen projection, the focal blocks in the serialized data are dynamically and automatically selected, and the LOD level of each focal block is dynamically adjusted to obtain the serialized LOD data. After the serialized LOD data is managed by VIBO blocks, triangle optimization and reconstruction are performed. A UV space remapping algorithm is used to align the facets and textures of the reconstructed point cloud surface, and multi-format textures are packaged for rendering output. The step of sorting the point cloud data using a six-way standard sequence sorting method to obtain a point cloud data sequence includes: dividing the point cloud data into multiple buckets; sorting the data within each bucket according to the selected sorting key; and writing the sorted data within each bucket into a new array sequentially according to the six-way standard sequence sorting method to form a globally ordered point cloud data sequence. The VIBO block management of the serialized LOD data includes: dividing the point cloud data into blocks according to the fixed vertex upper limit of each focal block; storing each block of data continuously at different positions in the Vertex Buffer, and recording the position and offset of each block in the Vertex Buffer in the Index Buffer.

2. The point cloud data processing method based on spatial serialization according to claim 1, characterized in that, Also includes: Construct a circular sequence index table; the circular sequence index table is used to perform circular traversal or local skip reading of point cloud data.

3. The point cloud data processing method based on spatial serialization according to claim 2, characterized in that, The step of dynamically and automatically selecting focal blocks in the serialized data based on the projection error between the view frustum and the screen, and dynamically adjusting the LOD level of each focal block, includes: calculating the projection error of each region within the view frustum, selecting regions with errors greater than a preset value as focal blocks; determining the required LOD level for each focal block based on its position, projection error, and changes in the view frustum; wherein the LOD level is updated in real time according to the user's movement or changes in viewing angle.

4. The point cloud data processing method based on spatial serialization according to claim 3, characterized in that, The triangle optimization and reconstruction includes: rearranging the vertex indices of the triangles in the serialized data according to a preset drawing order; using the rearranged vertex order to calculate the normal vector of the triangle; determining whether the normal of each triangle points to a predetermined direction; if the normal direction is opposite to the reference direction, then reversing the vertex order; if the normal direction is consistent with the reference direction, then generating optimized triangle serialized data; wherein, the winding direction of the triangle is consistent with the normal direction.

5. The point cloud data processing method based on spatial serialization according to claim 4, characterized in that, The UV space remapping algorithm is used to perform patch and texture alignment processing on the point cloud surface after triangle optimization and reconstruction. This includes: dividing the point cloud surface into multiple triangular patches, determining the vertices of each triangular patch and its associated UV coordinates; and adjusting the texture mapping of each triangular patch based on the vertex coordinates of the triangular patches and the transformation matrix to ensure that the vertices of each triangular patch and its associated UV coordinates are correctly aligned.

6. The point cloud data processing method based on spatial serialization according to claim 5, characterized in that, The process of packaging multi-format textures to complete rendering output also includes: encrypting the aligned point cloud data using encryption technology and metadata; wherein, the metadata includes data version information, block size information, and a compression table; in the offline stage, the encrypted data is converted into different texture formats for data packaging; wherein, the packaged data supports real-time decompression on the GPU and WebGL rendering output.

7. A system for processing point cloud data based on spatial serialization according to any one of claims 1-6, characterized in that, The system includes: a first processing module, which sorts point cloud data using a six-way standard sequence sorting method to obtain a point cloud data sequence; a second processing module, which dynamically and automatically selects focal blocks in the serialized data based on the view frustum and screen projection error, and dynamically adjusts the LOD level of each focal block to obtain serialized LOD data; an optimization module, which performs triangle optimization and reconstruction on the serialized LOD data after VIBO block management; a third processing module, which uses a UV spatial remapping algorithm to perform patch and texture alignment processing on the point cloud surface after triangle optimization and reconstruction; and an output module, which processes the data from the third processing module. The block output results are packaged into multi-format textures to complete the rendering output; wherein, the point cloud data is sorted using a six-way standard sequence sorting method to obtain a point cloud data sequence, including: dividing the point cloud data into multiple buckets; sorting the data in each bucket according to the selected sorting key; writing the sorted data in the buckets into a new array sequentially according to the six-way standard sequence sorting method to form a globally ordered point cloud data sequence; the serialized LOD data is managed by VIBO block segmentation, including: dividing the point cloud data into blocks according to the fixed vertex upper limit of each focal block; storing each block of data continuously in different positions of the Vertex Buffer, and recording the position and offset of each block in the Vertex Buffer in the Index Buffer.

8. A point cloud data processing device based on spatial serialization, characterized in that, The point cloud data processing device includes: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the spatially serialized point cloud data processing device to execute the spatially serialized point cloud data processing method as described in any one of claims 1-6.

Images