A point cloud rendering method based on multithreading and LOD

Abstract

The present application discloses a point cloud rendering method based on multithreading and LOD, which relates to the field of video rendering, including: dividing point cloud data into n data blocks; calculating the starting position of each data block in the point cloud file and creating n threads; extracting m LOD point cloud data of different detail levels from the octree of each data block; calculating the ratio of the current point cloud size to the rendering viewport and the distance of the viewpoint from the camera, and calculating the detail level of the current LOD point cloud to be rendered according to a preset viewport ratio threshold and a viewing distance threshold; selecting point cloud data of corresponding detail level from the m LOD point cloud data; for each interactive operation event, performing parallel coordinate transformation and illumination calculation on the current LOD point cloud data render_data to be rendered in different threads; submitting the calculation results to the renderer for rendering; in view of the low rendering efficiency in the prior art, the present application performs efficient rendering of large-scale point cloud data.

Description

Technical Field

[0001] The present application relates to the field of video rendering, and in particular to a point cloud rendering method based on multithreading and LOD. Background Art

[0002] With the rapid development of 3D scanning and sensor technologies, the acquisition of large-scale point cloud data has become increasingly convenient. This massive amount of point cloud data has been widely used in fields such as virtual reality and autonomous driving, providing a critical data foundation for technological advancement and application innovation in these industries. However, in practical applications, the efficient processing and real-time rendering of large-scale point cloud data still face many technical challenges.

[0003] The loading and rendering of point cloud data often presents significant performance bottlenecks and time-consuming issues. This is especially true during the rendering phase, where each point undergoes complex operations such as coordinate transformation, lighting calculation, and projection to be correctly displayed on the screen. This process is not only computationally intensive but can also easily lead to congestion in the graphics rendering pipeline, severely impacting rendering efficiency and user experience.

[0004] Real-time applications, such as autonomous vehicles' real-time environmental perception and instant scene reconstruction in virtual reality, place extremely high demands on point cloud rendering speed and frame rate stability. However, even with high-performance GPU acceleration, real-time rendering of large-scale point cloud data often becomes a technical bottleneck that restricts the development of related applications.

[0005] While some existing solutions address point cloud rendering efficiency, such as using spatial indexing structures like octrees to organize and manage point cloud data to reduce the number of passes during rendering, or utilizing techniques like frustum culling and backface culling to remove invisible point cloud data and reduce the rendering burden, these methods still suffer from low computational efficiency and high memory usage when processing large-scale point cloud data, making them difficult to meet the performance requirements of real-time interactive applications. Summary of the Invention

[0006] In response to the problem of low rendering efficiency in the existing technology, this application provides a point cloud rendering method based on multi-threading and LOD. Under the multi-threaded parallel framework, the LOD detail level and spatial division granularity are adaptively adjusted to efficiently render large-scale point cloud data.

[0007] The purpose of this application is achieved through the following technical solutions.

[0008] This application provides a point cloud rendering method based on multithreading and LOD, including: , divide the point cloud data into n data blocks, the number of bytes of each data block is ; Calculate the starting position of each data block in the point cloud file, create n threads, and each thread reads a data block in parallel; use the octree algorithm to filter the n data blocks at different scales, and extract m LOD point cloud data of different detail levels from the octree of each data block; the detail level is determined by the spatial division granularity of the octree. The smaller the granularity, the higher the detail level and the denser the point cloud data; calculate the ratio of the current point cloud size to the rendering viewport , and the distance from the viewpoint to the camera , based on a preset viewport ratio threshold and sight distance threshold , calculate the level of detail of the LOD point cloud to be rendered ; According to the LOD point cloud detail level to be rendered , select the point cloud data corresponding to the level of detail from the m LOD point cloud data as the current LOD point cloud data to be rendered ; Get the user's interactive operation events, and for each interactive operation event, perform parallel coordinate transformation and lighting calculation on the LOD point cloud data render_data to be rendered in different threads; after all calculation threads are completed, submit the calculation results to the renderer, and the renderer renders the submitted LOD point cloud data; among them, interactive operation events include rotation, scaling and translation.

[0009] LOD stands for Level of Detail, which represents multiple levels of detail. LOD point cloud data refers to a collection of point cloud data with different levels of detail generated by simplifying and sampling the original point cloud. LOD point cloud data is typically organized in a data structure such as an octree, with nodes at different levels representing point cloud representations of varying granularity. During rendering, the appropriate LOD level is dynamically selected for display based on factors such as the viewpoint position and viewport size to improve rendering efficiency. The level of detail represents the degree of simplification of the LOD point cloud data and is a measure of the fineness of the point cloud. In a spatial partitioning structure such as an octree, the depth of the tree corresponds to different levels of detail. The leaf nodes of the tree represent the original, most complete point cloud data with the highest level of detail; the root of the tree represents the coarsest point cloud representation with the lowest level of detail. During the rendering process, selecting the appropriate level of detail based on factors such as the viewpoint position can reduce computational overhead and improve rendering performance while ensuring visual quality. Indicates the proportion of the currently rendered point cloud within the screen viewport. It is generally calculated by dividing the screen pixel area of ​​the current point cloud by the total pixel area of ​​the viewport. This ratio reflects the relative size of the point cloud rendering object within the field of view and is a key parameter affecting rendering performance and quality. A larger ratio indicates a more prominent point cloud within the field of view, and a higher level of detail should be used; a smaller ratio indicates a less important point cloud rendering object within the field of view, and a lower level of detail should be used. Indicates the distance from the current viewpoint (observer position) to the camera. In 3D rendering, the camera represents the point of observation. The distance between the viewpoint and the camera directly affects the size and clarity of the object on the screen. The farther the distance, the smaller the pixel area of ​​the object on the screen, and the lower the level of detail required for rendering; the closer the distance, the larger the pixel area of ​​the object on the screen, and the higher the level of detail required for rendering. Therefore, It is another key parameter that determines the choice of LOD level.

[0010] Furthermore, the LOD point cloud level of detail to be rendered is calculated The formula is:

[0011] ;in, Indicates the distance from the viewpoint to the camera; Represents a preset reference distance, used to control the rate of change of the LOD detail level; Indicates the ratio of the current point cloud size to the rendering viewport; Represents a parameter; C represents a preset constant used to adjust the magnitude of the LOD detail level change.

[0012] in, Is a preset distance value used to control the rate of change of the LOD level of detail. When the viewpoint distance is equal to this value, the LOD level of detail will change by one unit. Parameters Used to control the relationship between LOD level of detail and viewport scale. The value range of is usually between 0 and 1. =0, the formula degenerates to only consider the line of sight factor; when = 1, the formula degenerates to only considering the viewport scale factor. Generally, l can be set to 0.5, indicating that view distance and viewport scale each have a 50% impact on the LOD level of detail. However, if the point cloud density in the scene is uneven, or if you want to emphasize details near or far, you can adjust the value of l appropriately. For example, if you want to maintain high detail in nearby point clouds while simplifying those far away, you can increase l to a larger value, such as 0.6 to 0.8.

[0013] Preset constants Used to adjust the range of LOD detail changes, that is, to control the difference in details between different LOD levels. The larger it is, the greater the difference in details between adjacent LOD levels; The smaller it is, the smaller the difference in detail between adjacent LOD levels. The value range of needs to be determined according to the specific point cloud data and the desired visual effect. Generally speaking, It can be set to an integer between 1 and 5. If the density of the point cloud changes slowly, or if you want a smooth transition in the rendering effect, you can set Set it to a smaller value, such as 1 to 2; if the density of the point cloud changes dramatically, or you want to emphasize the differences between different LOD levels, you can set Set it to something larger, like 3 to 5.

[0014] Furthermore, according to the total number of bytes of the point cloud file , divide the point cloud data into n data blocks, the number of bytes of each data block is , including: Get the total number of bytes of the point cloud file ; According to the preset data block size , calculate the number of data blocks n, the calculation formula is: ; Where ceil represents the upward rounding function; according to the number of data blocks n, the point cloud data is divided into n data blocks, and the number of bytes of each data block does not exceed , the number of bytes of point cloud data in the kth data block is : ; Where k represents the number of the data block, the value range is [1, n], and min represents the minimum value function.

[0015] Furthermore, the starting position of each data block in the point cloud file is calculated, n threads are created, and each thread reads a data block in parallel, including: for the kth data block, its starting position in the point cloud file is calculated , the calculation formula is: ; Where k represents the number of the data block, and its value range is [1, n]. Create n threads, each thread corresponds to a data block, and the kth thread is responsible for reading the point cloud data of the kth data block. In the kth thread, according to the starting position of the kth data block and data block size , use the file random access method to read the point cloud data of the data block from the point cloud file, and the number of bytes of data read is ;Store the point cloud data read by the kth thread in a pre-allocated shared memory block In , the starting address of the shared memory block is: ;in, Indicates the starting address of the kth data block in the shared memory block; all threads perform the read operation in parallel, and after all threads are completed, the shared memory block The complete point cloud data is stored in for use in subsequent steps.

[0016] Furthermore, the n data blocks are filtered at different scales using the octree algorithm, and m LOD point cloud data of different detail levels are extracted from the octree of each data block, including: For each data block in the octree, the octree algorithm is used to perform spatial division and generate the corresponding octree , the root node of the octree represents the bounding box of the entire data block, and the child nodes represent the bounding boxes of the subspaces after recursive partitioning; set m different space partitioning granularities ,in, Indicates the LOD level, the value range is [1, m]; for each data block octree , perform m filtering to obtain n×m LOD point cloud data , where k represents the number of the data block and l represents the LOD level; for each LOD level l, merge the LOD point cloud data corresponding to all data blocks to obtain the LOD point cloud data of the lth layer : ; Among them, merge represents a function that merges multiple point cloud data sets into a complete point cloud data set. The merged point cloud data set contains the point cloud data in all input sets and does not contain duplicate point cloud data.

[0017] Furthermore, set m different space division granularities , including: determining the spatial division granularity of the first layer LOD The scaling factor of the spatial division granularity between two adjacent LOD levels ; Preferably, by analyzing the density change of the point cloud data between two adjacent LOD levels, adaptively adjust The value of makes the spatial division granularity scaling between adjacent LOD levels more reasonable, ensuring smooth transition and gradual loss of details between different LOD levels. Specifically, calculate the change in point cloud data density between two adjacent LOD levels. Layer LOD and Layer LOD, you can select a fixed-size cube as the sampling area, count the number of points in the sampling area of ​​the two LOD levels respectively, and then calculate the ratio of the number of points to get the density change factor . ;in, Indicates the The number of points of the layer LOD within the sampling area, Indicates the The number of points in the sampling area of ​​the layer LOD. According to the adaptively adjusted spatial partitioning granularity scaling factor , update the The spatial division granularity of layer LOD . ; Through adaptive adjustment The value of can make the spatial division granularity scaling between adjacent LOD levels more reasonable, ensuring smooth transition and gradual loss of details between different LOD levels.

[0018] according to and , calculate the The spatial division granularity of layer LOD , the calculation formula is: , where pow represents a power function; preferably, , The weight factor representing the spatial granularity of the first level of detail (LOD) can be dynamically adjusted based on factors such as the importance of different LOD levels and the distribution characteristics of the point cloud data. By introducing the weight factor, the spatial granularity of different LOD levels can be more flexibly controlled, improving the quality and efficiency of LOD point cloud data generation.

[0019] Furthermore, for each data block's octree , perform m filtering, including: starting from the root node of the tree, recursively traversing each node of the octree; for the currently visited node node, calculate the spatial density of the point cloud data contained in the corresponding node , the calculation formula is: ;in, Indicates the number of point cloud data contained in the node node, Represents the volume of the spatial bounding box corresponding to the node node; calculates the normal vector variance of the local point cloud data of the node node , as the geometric complexity measure of the corresponding node, is calculated as: ;in, Represents the normal vector of the i-th point in node node, Represents the average value of the normal vectors of all points in the node node; according to the spatial density of the node node , normal vector variance Hedi Space partition granularity , calculate the importance of node details , the calculation formula is: ;Will With preset threshold To compare: If Less than or equal to , then add the point cloud data of node node to the LOD point cloud data of layer l and stop traversing the child nodes of the corresponding node; if Greater than , then continue to recursively traverse the child nodes of node node.

[0020] Furthermore, the user's interactive operation events are obtained, and for each interactive operation event, the LOD point cloud data to be rendered is processed in different threads. Perform parallel coordinate transformation and lighting calculations, including: obtaining user interaction events; for each interaction event, calculating the corresponding coordinate transformation matrix based on the event type and parameters and lighting parameters ; For each interactive operation event, create threads, Is a positive integer, each thread has a LOD point cloud data to be rendered A continuous subset of Perform coordinate transformation and lighting calculations; in each thread, according to the coordinate transformation matrix and lighting parameters , the subset responsible for the thread Perform coordinate transformation and lighting calculation on each point in the to obtain the transformed point cloud subset ; All calculation threads are completed, and the transformed point cloud subsets calculated by each thread are Merge into a complete point cloud data set: ;in, Represents the transformed point cloud subset calculated by the i-th thread, and the value range of t is The merge function is used to merge multiple point cloud subsets into a complete point cloud data set according to the original point index order; the merged point cloud data Submit to the renderer; the renderer processes the submitted point cloud data Rendering is performed to generate the final point cloud visualization image.

[0021] Furthermore, each thread is divided into continuous subsets , through the following steps: The point cloud data in the image is divided into continuous subsets , the value range of t is ; Each subset The number of point cloud data included for: ;in, express The total number of point cloud data, ceil represents the rounding function; the tth thread is responsible for processing the tth subset .

[0022] Furthermore, in each thread, according to the coordinate transformation matrix and lighting parameters , the subset responsible for the thread Perform coordinate transformation and lighting calculation on each point in the to obtain the transformed point cloud subset , including: for the subset For each point in point, multiply the coordinates of point point by the coordinate transformation matrix , get the transformed coordinates ; According to the normal vector of point point , lighting parameters And the transformed coordinates , calculate the light intensity of point point , the lighting calculation adopts the Phong lighting model; the transformed coordinates and light intensity As the new coordinates and color values ​​of point point, add it to the transformed point cloud subset middle.

[0023] Compared with the existing technology, the advantages of this application are:

[0024] By dividing large-scale point cloud data into multiple data blocks and utilizing multi-threaded parallel reading, the method of this application can significantly improve the efficiency of loading and transmitting point cloud data. Compared with traditional serial reading methods, multi-threaded parallel reading can fully utilize the computer's multi-core processing power to accelerate the loading process of point cloud data. At the same time, by dividing point cloud data into multiple data blocks, the size of each data block can be reduced, thereby reducing data reading latency and improving the concurrency of data transmission.

[0025] The method of this application generates point cloud data at multiple LOD levels of detail by using an octree algorithm to spatially partition and filter each data block at different scales. This multi-level LOD representation adaptively selects the current LOD level of detail to be rendered based on parameters such as viewpoint position and viewport size, dynamically filtering visible point cloud data. Compared to directly rendering the raw point cloud data, this method can significantly reduce the rendering overhead of redundant data, lower the data redundancy of point cloud rendering, and improve rendering efficiency.

[0026] During point cloud rendering, user interactions (such as translation, rotation, and scaling) require real-time coordinate transformation and lighting calculations for the point cloud data, placing high demands on rendering performance. The method in this application utilizes multithreading to perform parallel coordinate transformations and lighting calculations on visible point cloud data, distributing computational tasks across multiple threads for concurrent execution. This significantly improves the speed of real-time interactive response during point cloud rendering. Compared to single-threaded computing, multithreaded parallel computing can fully utilize the multi-core processing power of modern computers, accelerating the transformation and lighting calculations of point cloud data, thereby achieving a smoother, more real-time user interaction experience.

[0027] This application adopts a data screening strategy based on importance, giving priority to rendering point cloud data that has a greater impact on visual effects. By calculating indicators such as the spatial density and geometric complexity of point cloud data, its contribution to rendering quality is evaluated, and screening is performed based on the importance threshold. This strategy can minimize the rendering of point cloud data that has less impact on visual effects while ensuring rendering quality, thereby reducing computing overhead and improving rendering performance. Compared with ordinary equal-interval sampling methods, data screening based on importance can more efficiently and specifically select key point cloud data to optimize rendering performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present application will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbers represent the same structures, wherein:

[0029] Figure 1 is an exemplary flowchart of a point cloud rendering method based on multithreading and LOD according to some embodiments of the present application;

[0030] Figure 2 This is an exemplary flowchart of rendering interaction according to some embodiments of the present application. DETAILED DESCRIPTION

[0031] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0032] Figure 1 This is an exemplary flow chart of a point cloud rendering method based on multithreading and LOD according to some embodiments of the present application, including: , divide the point cloud data into n data blocks, the number of bytes of each data block is ; Calculate the starting position of each data block in the point cloud file, create n threads, and each thread reads a data block in parallel; use the octree algorithm to filter the n data blocks at different scales, and extract m LOD point cloud data of different detail levels from the octree of each data block; the detail level is determined by the spatial division granularity of the octree. The smaller the granularity, the higher the detail level and the denser the point cloud data; calculate the ratio of the current point cloud size to the rendering viewport , and the distance from the viewpoint to the camera , based on a preset viewport ratio threshold and sight distance threshold , calculate the level of detail of the LOD point cloud to be rendered ; According to the LOD point cloud detail level to be rendered , select the point cloud data corresponding to the level of detail from the m LOD point cloud data as the current LOD point cloud data to be rendered ; Get the user's interactive operation events, and for each interactive operation event, perform parallel coordinate transformation and lighting calculation on the LOD point cloud data render_data to be rendered in different threads; after all calculation threads are completed, submit the calculation results to the renderer, and the renderer renders the submitted LOD point cloud data; among them, interactive operation events include rotation, scaling and translation.

[0033] Get the total number of bytes in the point cloud file :If the point cloud file is stored in the file system, you can use the file operation library or system API to get the file's byte size, such as using the std::filesystem::file_size function in C++. If the point cloud file is stored in a database, you can get the file's byte size by querying the database metadata or using the functions provided by the database. , calculate the number of data blocks n: use the formula: , where ceil represents a ceiling function. In the code, you can use the ceil function provided by the math library or a custom ceiling function. Based on the number of data blocks n, divide the point cloud data into n data blocks: Create an array or vector blocks of length n to store the information of each data block. Traverse the data block number k, from 1 to n: Calculate the starting byte position of the kth data block: . Calculate the number of bytes in the kth data block: , where min represents the minimum function. The starting byte position and number of bytes of the kth data block are stored in blocks[k-1].

[0034] Calculate the starting position of each data block in the point cloud file, create n threads, and each thread reads a data block in parallel; including: Calculate the starting position of each data block in the point cloud file: Create an array or vector of length n , used to store the starting position of each data block. Traverse the data block number k, from 1 to n: calculate the starting position of the kth data block in the point cloud file , using the formula: .Will Stored in Create n threads to read the data block in parallel: Create a thread array or vector threads of length n. Traverse the data block number k, from 1 to n: Create the kth thread and store it in middle.

[0035] In the execution function of the kth thread: according to Get the starting position of the kth data block File random access reads data blocks: According to the calculated number of bytes of the kth data block , determine the amount of data to be read. Use the file random access method to move the file pointer to the starting position of the kth data block. Starting from the current position of the file pointer, read continuously Bytes of data. The read operation uses the low-level file I / O functions provided by the system, such as read or fread, to read in bytes. The read data is stored in a temporary memory buffer. The read data is stored in the shared memory block: Calculate the starting address of the kth data block in the shared memory block , the formula is: Use memory copy functions, such as memcpy, to copy the data in the temporary memory buffer to the corresponding location of the shared memory block. The starting address of the copy is , the number of bytes copied is The copy operation is performed in memory and is faster.

[0036] The thread exits after execution is completed: After the current thread completes the reading and copying of the data block, the thread execution is completed. When the thread exits, resources such as temporary memory buffers are released. The way the thread exits depends on the parallel programming framework or library used, such as or Wait for all threads to finish executing: The main thread or control thread traverses the thread array or vector threads and checks the execution status of each thread one by one. Use thread synchronization primitives such as or , wait for each thread to finish executing. When all threads have finished executing, the reading and copying operations of all data blocks have been completed. Complete point cloud data in shared memory block: After all threads have finished executing, the shared memory block The complete point cloud data is stored in the shared memory block. The starting address of the shared memory block is , the total number of bytes is Subsequent data processing steps can directly access the shared memory block without having to read data from the file. The use of shared memory blocks can improve data access efficiency and reduce the overhead of file I / O operations.

[0037] For n data blocks, the octree algorithm is used to filter at different scales, and m LOD point cloud data with different levels of detail are extracted from the octree of each data block; the level of detail is determined by the spatial division granularity of the octree. The smaller the granularity, the higher the level of detail and the denser the point cloud data. First, traverse each data block in the shared memory block: according to the shared memory block The starting address and size of each data block , calculate the number of data blocks n in the shared memory block. Use a loop or iterator to access each data block in the shared memory block one by one. For the kth data block, its starting address in the shared memory block is: According to the starting address and size of the data block, the point cloud data of the data block is read and stored in a temporary memory buffer.

[0038] Create an octree object: For the kth data block, create an empty octree object , used to store the spatial partitioning results of the data block. Octree objects can be implemented using classes or structures, and contain attributes such as root node pointer and number of nodes. Initialized to NULL, indicating an empty octree. Calculate the bounding box of the data block: traverse all point cloud data in the kth data block and find the minimum and maximum values ​​in the three dimensions of x, y, and z. Let the coordinates of the point cloud data be (x, y, z), then the minimum point of the bounding box is and maximum point The coordinates of are calculated as follows: ; ;in, 、 、 Respectively represent the minimum values ​​of the x, y, and z coordinates of all points in the data block, 、 、 Respectively represent the maximum values ​​of the x, y, and z coordinates of all points in the data block.

[0039] The bounding box can be represented by a structure To represent the coordinates of the minimum and maximum points. Start the recursive space division process: the point cloud data of the data block and the bounding box of the root node Passed to the octree construction function .exist In the function, a new node object node is created to represent the octree node currently being processed. The bounding box of the root node is assigned to the bounding box of the current node node.bounding_box. Recursively call Function to start the space partitioning process.

[0040] Recursively partition the octree space: In the function, whether the space needs to be further divided is determined according to the following conditions: If the bounding box size of the current node is less than or equal to the preset space division threshold , then mark the current node as a leaf node and stop dividing. If the number of point cloud data contained in the current node is less than or equal to the preset point count threshold , then mark the current node as a leaf node and stop dividing. If further division of the space is required, perform the following steps: divide the bounding box node.bounding_box of the current node equally in the x, y, and z dimensions to obtain eight sub-bounding boxes For each sub-bounding box , divide the point cloud data contained in the current node into corresponding child nodes according to its coordinate range. For each child node, recursively call Function, pass in the corresponding child bounding box and point cloud data, and continue to divide the space. The recursive process continues until the condition of the leaf node is reached (bounding box size or point number threshold). The bounding box and point cloud data of the node are calculated during the generation of the octree: in the process of recursive division of space, the corresponding bounding box node.bounding_box is calculated for each node. The bounding box is calculated by finding the minimum and maximum coordinate values ​​of the point cloud data contained in the node. When dividing the point cloud data into child nodes, the bounding box of the child node is updated at the same time to ensure that the bounding box can correctly contain all the point cloud data. For leaf nodes, the point cloud data they contain is stored in the node's attribute node.points for subsequent access and processing.

[0041] An octree algorithm is used to spatially partition each data block. First, an empty octree object is created, and the bounding box of the data block is calculated as the bounding box of the root node. The data block's point cloud data and the bounding box of the root node are then passed to the octree's constructor, beginning the recursive spatial partitioning process. During the recursive process, the size of the current node's bounding box and the point count threshold determine whether further spatial partitioning is necessary. If partitioning is necessary, the current node's bounding box is divided into eight equal sub-bounding boxes, and the point cloud data is allocated to the corresponding sub-nodes. The recursive process continues until a leaf node is reached. During octree generation, the bounding box for each node is calculated, and the point cloud data for the leaf node is stored in the node's attributes. Through the spatial partitioning of the octree, point cloud data can be organized into a hierarchical structure, facilitating subsequent LOD point cloud data extraction and rendering. Octree node bounding boxes and point cloud data storage can be implemented using structures or classes for efficient data access and processing.

[0042] Set different spatial division granularities: Determine the spatial division granularity of the first layer LOD , for example, set to 0.1, indicating that the spatial division granularity of the first LOD layer is one tenth of the data block side length. Determine the scaling factor of the spatial division granularity between two adjacent LOD levels For example, if it is set to 0.5, it means that the space division granularity is reduced to half of the original one for each LOD level increase. and , calculate the spatial division granularity of the lth layer LOD , implemented using the power function: ; where l represents the LOD level, ranging from 1 to m, and m is the maximum number of LOD levels. For example, when is 0.1, When it is 0.5, the spatial division granularity of the second layer LOD 0.05, the spatial division granularity of the third layer LOD The value is 0.025, and so on. Different spatial division granularity corresponds to different LOD levels. The smaller the granularity, the higher the LOD level and the denser the point cloud data.

[0043] Filter the octree of each data block to obtain point cloud data at different LOD levels: For the octree of each data block , perform m filtering to obtain n × m LOD point cloud data , where k represents the number of the data block and l represents the LOD level. Starting from the root node of the tree, recursively traverse each node of the octree. For the currently visited node, calculate the spatial density of the point cloud data contained in the corresponding node and the normal vector variance of the local point cloud data . Spatial density The calculation formula is: ,in Indicates the number of point cloud data contained in the node, and node_volume indicates the volume of the node. Normal vector variance The calculation formula is: ,in Represents the normal vector of each point in the node, Represents the average of the normal vectors of all points in the node.

[0044] Calculate the importance of the details of the node based on the node's spatial density, geometric complexity, and the lth spatial partition granularity. The importance of details The calculation formula is: ,in For the The spatial division granularity of the layer LOD. With preset threshold Compare and decide whether to add the point cloud data of node node to the LOD point cloud data of layer l If Less than or equal to , then add the point cloud data of node node to , and stop traversing the child nodes of the corresponding node. Greater than , then continue to recursively traverse the child nodes of node node. Merge the LOD point cloud data corresponding to all data blocks: For each LOD level l, merge the LOD point cloud data corresponding to all data blocks to obtain the LOD point cloud data of the lth layer The merge operation is implemented by the merge function, which merges multiple point cloud datasets into a complete point cloud dataset. The merged point cloud dataset contains the point cloud data from all input datasets and does not contain duplicate point cloud data.

[0045] The merging operation can be performed in a parallel manner, such as using parallel computing frameworks such as OpenMP or CUDA, to improve the efficiency of data processing. According to the set spatial division granularity and LOD level, the octree of each data block is filtered to obtain point cloud data of different LOD levels. First, the spatial division granularity of the first layer LOD and the scaling factor between adjacent LOD levels are determined, and the spatial division granularity of different LOD levels is obtained by power function calculation. Then, the octree of each data block is recursively traversed to calculate the spatial density, normal vector variance and detail importance of the node, and the threshold is used to determine whether the point cloud data of the node is added to the point cloud data of the corresponding LOD level. Finally, for each LOD level, the LOD point cloud data corresponding to all data blocks are merged to obtain complete LOD point cloud data.

[0046] For each LOD level l, merge the LOD point cloud data corresponding to all data blocks to obtain the l-th layer LOD point cloud data to calculate the current LOD point cloud detail level to be rendered The formula is: ;in, Indicates the distance from the viewpoint to the camera; Represents a preset reference distance, used to control the rate of change of the LOD detail level; Indicates the ratio of the current point cloud size to the rendering viewport; Represents a parameter; C represents a preset constant used to adjust the range of change in the LOD detail level; during the rendering process, the distance view_distance from the viewpoint to the camera is calculated in real time, which can be calculated from the position of the camera and the position of the viewpoint.

[0047] Set the preset reference distance , used to control the rate of change of the LOD level of detail. It can be adjusted according to specific application scenarios and performance requirements. Calculate the ratio of the current point cloud size to the rendering viewport , which can be calculated from the bounding box of the point cloud and the size of the viewport. Set the parameter l to control the rate of change of the LOD level of detail. The value range of l can be adjusted according to the specific application scenario and performance requirements, and is usually between 0 and 1. Set the preset constant C to adjust the amplitude of the change of the LOD level of detail. The value of C can be adjusted according to the specific application scenario and performance requirements, and is usually between 0 and 1. log2 represents a logarithmic function with base 2, which is used to map the viewpoint distance to the range of change of the LOD level of detail. Indicates the ratio of the viewpoint distance to the reference distance. A larger value indicates a higher LOD level of detail. l is used to control the rate at which the LOD level of detail changes. A larger value indicates a faster change in the LOD level of detail. Indicates the effect of the current point cloud size on the LOD level of detail. The larger the value, the higher the LOD level of detail.

[0048] Figure 2 This is an exemplary flowchart of the rendering interaction shown in some embodiments of the present application, which calculates the ratio of the current point cloud size to the rendering viewport. : Get the width of the current rendering viewport and height , in pixels. Calculate the bounding box of the current point cloud and get its projection size on the X and Y axes and , expressed in the same units. Calculate the proportion of the point cloud on the X-axis and Y-axis and : ; ;Pick and The maximum value is the ratio of the current point cloud size to the rendering viewport : ; The value range is 0 to 1, indicating the relative size of the point cloud in the viewport.

[0049] Calculate the distance from the viewpoint to the camera : Get the current camera position , expressed in the form of three-dimensional coordinates (x, y, z). Get the position of the current viewpoint , expressed in the form of three-dimensional coordinates (x, y, z). Calculate the Euclidean distance between the camera position and the viewpoint position as the distance of the viewpoint from the camera : ; The units are consistent with the camera position and viewpoint position, usually expressed in meters or world coordinate units. and sight distance threshold , calculate the level of detail of the LOD point cloud to be rendered : Set the preset viewport ratio threshold , which represents the minimum acceptable size of the point cloud in the viewport. Less than , it means that the point cloud is too small in the viewport and the LOD level of detail needs to be reduced.

[0050] Set a preset viewing distance threshold , represents the maximum acceptable distance between the viewpoint and the camera. Greater than When , it means that the viewpoint is too far from the camera and the LOD level of detail needs to be lowered. Calculate the LOD point cloud level of detail to be rendered :if Less than ,or Greater than , then reduce the LOD level of detail: ; Otherwise, keep the current LOD level of detail: ; The value range is 1 to m, which indicates the level of detail of the LOD point cloud data to be rendered, and m is the maximum level of detail of the LOD point cloud data.

[0051] A point cloud container is a data structure used to store and manage point cloud data. Its primary function is to organize and maintain point cloud data at different levels of detail (LODs) for fast retrieval and access during rendering. In LOD-based point cloud rendering methods, the original point cloud data is typically divided into multiple data blocks, simplified, and sampled to generate LOD point cloud data at different levels of detail (LODs). This LOD point cloud data must be efficiently stored and managed so that the appropriate LOD can be dynamically selected during rendering based on the viewpoint position and viewport size. Point cloud containers typically use a tree-like hierarchical structure, such as an octree or kd-tree, to organize point cloud data at different LODs. Each tree node represents a spatial region and contains a subset of the point cloud within that region. Parent nodes represent lower LODs, while child nodes represent higher LODs. By traversing the tree structure, one can quickly locate the LOD level that suits the current rendering parameters and retrieve the corresponding point cloud data.

[0052] According to the current LOD point cloud level of detail to be rendered , select the point cloud data corresponding to the level of detail from the m LOD point cloud data as the current LOD point cloud data to be rendered :according to The value of m LOD point cloud data arrive Select the point cloud data corresponding to the level of detail: . Select the point cloud data The point cloud is passed to the rendering pipeline for rendering and display. During the rendering process, the ratio of the current point cloud size to the rendering viewport and the distance from the viewpoint to the camera are calculated in real time, and the LOD point cloud level of detail to be rendered is calculated based on the preset threshold. First, the size of the rendering viewport and the bounding box of the point cloud are obtained, and the relative size of the point cloud in the viewport is calculated. Then, calculate the Euclidean distance view_distance between the camera position and the viewpoint position. and Compare the result with the preset threshold to calculate the level of detail of the LOD point cloud to be rendered Finally, select the point cloud data corresponding to the level of detail from the m LOD point cloud data as the point cloud data to be rendered. , and passed to the rendering pipeline for rendering and display.

[0053] Get the user's interactive operation events, and for each interactive operation event, process the LOD point cloud data to be rendered in different threads Perform parallel coordinate transformation and lighting calculations; after all calculation threads are completed, submit the calculation results to the renderer, which renders the submitted LOD point cloud data; among them, interactive operation events include rotation, scaling and translation. First, obtain the user's interactive operation events: listen for the user's mouse, keyboard or touch events, such as mouse press, move, release, keyboard key, touch screen, etc. For each event, obtain the event type (such as rotation, scaling, translation) and related parameters (such as rotation angle, scaling factor, translation vector). and parameters Encapsulated as an interactive operation event , add to the event queue Waiting for processing.

[0054] For each interactive operation event, calculate the corresponding coordinate transformation matrix based on the event type and parameters and lighting parameters : From the event queue Extract an interactive operation event . According to the event type and parameters , calculate the corresponding coordinate transformation matrix : For rotation events: Get the rotation axis and rotation angle (in radians). and rotation angle , construct the rotation matrix : is the unit vector (x, y, z), is the radian value; ; ; ; For zoom events: Get the zoom factor 、 and , respectively, represent the scaling ratio along the X-axis, Y-axis, and Z-axis. According to the scaling factor 、 and , construct the scaling matrix : For translation events: Get the translation vector , represents the translation along the X-axis, Y-axis and Z-axis. According to the translation vector , construct the translation matrix : translation vector for ; . The rotation matrix , scaling matrix and translation matrices Multiply to get the final coordinate transformation matrix : .

[0055] According to event type and parameters , calculate the corresponding lighting parameters : For rotation events: According to the rotation matrix , update the light source direction . Assume that the initial light source direction is , the updated light source direction is: ; Updated light source direction is the rotation matrix Multiplied by the initial light direction, ignoring the fourth component.

[0056] For zoom events: according to the zoom factor , update the light intensity Assuming the initial light intensity is intensity, the updated light intensity is: ; Updated light intensity The product of the initial light intensity and the scaling factor. For translation events: lighting parameters Unchanged. Get the coordinate transformation matrix corresponding to each interactive operation event and lighting parameters . Coordinate transformation matrix It will be used to transform the coordinates of the point cloud data, converting the point cloud data from the original coordinate system to the transformed coordinate system. It will be used to update the light source direction and intensity in the lighting calculation, affecting the lighting effect of the point cloud data.

[0057] For each interaction event, create threads, each thread processes the LOD point cloud data to be rendered. A continuous subset of Performing coordinate transformation and lighting calculations: Determining the number of threads , can be adjusted dynamically based on hardware conditions and data size. Get the current LOD point cloud data to be rendered ,Include Points. Divide into points according to their index order continuous subsets : Calculate the number of points contained in each subset , using the formula , where ceil represents the upward rounding function. For the t-th subset , whose starting point index is , the end point index is .create threads , the value range of t is .

[0058] In each thread , perform the following steps: Get the subset that the thread is responsible for . According to the coordinate transformation matrix and lighting parameters ,right Perform coordinate transformation and lighting calculation for each point in: For each point point, multiply its coordinates by , get the transformed coordinates . According to the normal vector of point point , lighting parameters And the transformed coordinates , use the Phong lighting model to calculate the light intensity of the point . The transformed coordinates and light intensity As the new coordinates and color values ​​of point point, add it to the transformed point cloud subset Thread After the calculation is completed, Join the thread synchronization queue middle.

[0059] After all calculation threads are completed, the transformed point cloud subsets calculated by each thread are merged into a complete point cloud data set: Create an empty point cloud data set . Synchronize queue from thread Take out the transformed point cloud subset calculated by each thread in turn .Will The point cloud data in is merged into Repeat the above steps until the thread synchronization queue Empty to get the final merged point cloud data The merged point cloud data Submit to the renderer: Submitted to the renderer through the renderer's data interface. The renderer processes the submitted point cloud data. Rendering is performed to generate the final point cloud visualization image. In response to user interactive operation events, coordinate transformation and lighting calculation are performed on the LOD point cloud data in parallel, and finally an interactive point cloud visualization result is generated. By dividing the point cloud data into multiple subsets and performing calculations in independent threads, the parallel processing capabilities of multi-core CPUs can be fully utilized to improve computing efficiency. At the same time, by dynamically adjusting the number of threads and the size of the point cloud subsets, load balancing can be performed according to hardware conditions and data scale to ensure real-time performance and response speed. During the data merging and rendering submission stages, thread synchronization and data reorganization can ensure the correctness and consistency of the point cloud data, avoiding concurrent access and data contention issues.

Claims

1. A point cloud rendering method based on multithreading and LOD, characterized in that: include: According to the total number of bytes of the point cloud file bytes , divide the point cloud data into n data blocks, the number of bytes of each data block is data size ; Calculate the starting position of each data block in the point cloud file, create n threads, and each thread reads a data block in parallel; For n data blocks, the octree algorithm is used to filter at different scales, and m LOD point cloud data with different detail levels are extracted from the octree of each data block; the detail level is determined by the spatial division granularity of the octree. The smaller the granularity, the higher the detail level and the denser the point cloud data. Calculate the ratio of the current point cloud size to the rendering viewport ratio , and the distance from the viewpoint to the camera distance , according to the preset viewport ratio threshold viewport threshold and the visual distance threshold distance threshold , calculate the target_lod_level of the LOD point cloud detail level to be rendered; according to the target_lod_level of the LOD point cloud detail level to be rendered, select the point cloud data of the corresponding detail level from the m LOD point cloud data as the LOD point cloud data to be rendered data ; Get the user's interactive operation events, and for each interactive operation event, perform parallel coordinate transformation and lighting calculation on the LOD point cloud data render_data to be rendered in different threads; after all calculation threads are completed, submit the calculation results to the renderer, and the renderer renders the submitted LOD point cloud data; where interactive operation events include rotation, scaling and translation; Extract m LOD point cloud data of different levels of detail from the octree of each data block, including: For shared memory blocks shared memory For each data block in the octree, the octree algorithm is used to divide the space and generate the corresponding octree k ,The root node of the octree represents the bounding box of the entire data block, and the child nodes represent the bounding boxes of the subspaces after recursive partitioning; Set m different space partition granularities lod threshold_l , where l represents the LOD level, and its value range is [1, m]; For each data block, the octree k , perform m filtering to obtain n×m LOD point cloud data lod data_k_l , where k represents the number of the data block and l represents the LOD level; For each LOD level l, merge the LOD point cloud data corresponding to all data blocks to obtain the LOD point cloud data lod of the lth layer data_l : lot data_l =merge(lot data_1_l ,lot data_2_l ,......,lot data_n_l ) Merge is a function that merges multiple point cloud data sets into a complete point cloud data set. The merged point cloud data set contains the point cloud data in all input sets and does not contain duplicate point cloud data. Set m different space partition granularities lod threshold_l ,include: Determine the spatial division granularity lod of the first layer LOD threshold_l And the scaling factor lod of the spatial division granularity between two adjacent LOD levels scale ; According to lod threshold_l and lod scale , calculate the spatial division granularity lod of the lth layer LOD threshold_l , the calculation formula is: lot threshold_l =lot threshold_1 ×pow(lot scale ,l-1) Wherein, pow represents the power function; For each data block, the octree k , perform m filtering, including: Starting from the root node of the tree, recursively traverse each node of the octree; For the currently accessed node node, calculate the spatial density of the point cloud data contained in the corresponding node node density , the calculation formula is: Among them, node point_count Indicates the number of point cloud data contained in the node node. volume Indicates the volume of the spatial bounding box corresponding to the node node; Calculate the normal vector variance of the local point cloud data of node node normal_variance , as the geometric complexity measure of the corresponding node, is calculated as: Among them, normal i Represents the normal vector of the i-th point in the node node, normal mean Represents the average value of the normal vectors of all points in the node node; According to the spatial density of node density , normal vector variancenode normal_variance and the lth space partition granularity lod threshold_l , calculate the importance of node details importance , the calculation formula is: The node importance and the preset threshold importance threshold For comparison: If node importance Less than or equal to importance threshold , then add the point cloud data of node node to the LOD point cloud data lod of layer l data_k_l and stop traversing the child nodes of the corresponding node; If node importance Greater than importance threshold , then continue to recursively traverse the child nodes of node node.

2. The point cloud rendering method based on multithreading and LOD according to claim 1, characterized in that: The formula for calculating the target_lod_level of the LOD point cloud detail level to be rendered is: Among them, view distance Indicates the distance from the viewpoint to the camera; distance threshold Represents the preset reference distance, which is used to control the rate of change of the LOD detail level; viewport ratio It represents the ratio of the current point cloud size to the rendering viewport; ζ represents a parameter; C represents a preset constant used to adjust the variation of the LOD detail level.

3. The point cloud rendering method based on multithreading and LOD according to claim 2, characterized in that: Divide the point cloud data into n data blocks, and the number of bytes in each data block is data size ,include: Get the total number of bytes of the point cloud file bytes ; According to the preset data block size data size , calculate the number of data blocks n, the calculation formula is: Among them, ceil represents the upward rounding function; According to the number of data blocks n, the point cloud data is divided into n data blocks, and the number of bytes of each data block does not exceed block size , the number of bytes of the point cloud data of the kth data block is block bytes_k : Among them, k represents the number of the data block, the value range is [1, n], and min represents the minimum value function.

4. The point cloud rendering method based on multithreading and LOD according to claim 3, characterized in that: Calculate the starting position of each data block in the point cloud file and create n threads, including: For the kth data block, calculate its starting position block in the point cloud file start_k , the calculation formula is: block start_k =(k-1)×block size Wherein, k represents the number of the data block, and its value range is [1, n]; Create n threads, each thread corresponds to a data block, and the kth thread is responsible for reading the point cloud data of the kth data block; In the kth thread, according to the starting position block of the kth data block start_k and data block size block bytes_k , use the file random access method to read the point cloud data of the data block from the point cloud file, and the number of bytes of data read is block bytes_k ; Store the point cloud data read by the kth thread in the pre-allocated shared memory block shared memory In , the starting address of the shared memory block is: memory start_k =shared memory +(k-1)×block size Among them, memory start_k Indicates the starting address of the kth data block in the shared memory block; All threads perform read operations in parallel, and after all threads have completed, the shared memory block shared memory The complete point cloud data is stored in for use in subsequent steps.

5. The point cloud rendering method based on multithreading and LOD according to claim 1, characterized in that: For each interactive operation event, render the LOD point cloud data to be rendered in different threads data Perform parallel coordinate transformation and lighting calculations, including: Get user interaction events; For each interactive operation event, calculate the corresponding coordinate transformation matrix transform according to the event type and parameters matrix and lighting parameters light params ; For each interactive operation event, create num threads threads, num threads is a positive integer. Each thread renders the LOD point cloud data to be rendered. data A continuous subset of data subset Perform coordinate transformation and lighting calculation; In each thread, according to the coordinate transformation matrix transform matrix and lighting parameters light params , the subset data that the thread is responsible for subset Perform coordinate transformation and illumination calculation on each point in to obtain the transformed point cloud subset transformed subset ; After all the computation threads are completed, the transformed point cloud subsets calculated by each thread are merged transformed_data Merge into a complete point cloud data set: merged transformed_data =merge(transformed subset_1 ,transformed subset_2 , ......,transformed subset_numthreads ) Among them, transformed subset_t Represents the transformed point cloud subset calculated by the i-th thread, and the value range of t is [1,mum threads ], the function of merge function is to merge multiple point cloud subsets into a complete point cloud data set according to the original point index order; The merged point cloud data transformed_data Submit to the renderer; The renderer merges the submitted point cloud data transformed_data Rendering is performed to generate the final point cloud visualization image.

6. The point cloud rendering method based on multithreading and LOD according to claim 5, characterized in that: Divide each thread into a continuous subset of data subset , through the following steps: Render data The point cloud data in is divided into num according to the index order of the points threads A continuous subset of data subset_t , the value range of t is [1,num threads ]; Each subset of data subset_t The number of point cloud data contained in subset point_count_t for: Among them, render point_count Represents render data The total number of point cloud data, ceil represents the upward rounding function; The tth thread is responsible for processing the tth subset data subset_t .

7. The point cloud rendering method based on multithreading and LOD according to claim 6, characterized in that: The subset of data that the thread is responsible for subset Perform coordinate transformation and illumination calculation on each point in to obtain the transformed point cloud subset transformed subset ,include: For the subset data subset For each point in , multiply the coordinates of the point by the coordinate transformation matrix transform matrix , get the transformed coordinate point transformed ; According to the normal vector point normal , Light parameters params And the transformed coordinate point transformed , calculate the light intensity point intensity , the lighting calculation adopts the Phong lighting model; The transformed coordinate point transformed and light intensity point intensity As the new coordinates and color value of point point, add the transformed point cloud subset transformed subset middle.

Images