Point cloud rendering method based on multiple threads and LOD

Abstract

The invention discloses a multi-thread and LOD-based point cloud rendering method, and relates to the field of video rendering, and the method comprises the steps: dividing point cloud data into n data blocks; calculating an initial position of each data block in the point cloud file, and creating n threads; extracting m pieces of LOD point cloud data with different levels of detail from the octree of each data block; calculating the proportion # imgabs0 # of the size of the current point cloud in a rendering viewport and the distance # imgabs1 # from a viewpoint to a camera, and calculating the detail level # imgabs4 # of the current to-be-rendered LOD point cloud according to a preset viewport proportion threshold value # imgabs2 # and a preset sight distance threshold value # imgabs3 #; according to # imgabs5 #, selecting point cloud data of a corresponding detail level from the m pieces of LOD point cloud data; for each interactive operation event, parallel coordinate transformation and illumination calculation are carried out on the LOD point cloud data to be rendered currently in different threads; submitting a calculation result to a renderer for rendering; aiming at low rendering efficiency in the prior art, the method and the device can efficiently render large-scale point cloud data.

Description

Technical Field

[0001] The present application relates to the field of video rendering, and in particular to a point cloud rendering method based on multi-threading and LOD. Background Art

[0002] With the rapid development of 3D scanning technology and sensor technology, it is becoming increasingly convenient to obtain large-scale point cloud data. These massive point cloud data have been widely used in fields such as virtual reality and autonomous driving, providing an important data foundation for technological progress and application innovation in related industries. However, in practical applications, the efficient processing and real-time rendering of large-scale point cloud data still face many technical challenges.

[0003] In the process of loading and rendering point cloud data, there are often significant performance bottlenecks and time consumption problems. Especially in the rendering stage, each point needs to go through complex operations such as coordinate transformation, lighting calculation, and projection before it can be correctly displayed on the screen. This process is not only computationally intensive, but also easily leads to congestion in the graphics rendering pipeline, seriously affecting rendering efficiency and user experience.

[0004] Especially in real-time application scenarios, such as real-time environmental perception of autonomous vehicles and instant scene reconstruction in virtual reality, there are extremely high requirements for the speed and frame rate stability of point cloud rendering. However, even with high-performance GPU acceleration, real-time rendering of large-scale point cloud data often becomes a technical bottleneck restricting the development of related applications.

[0005] In the existing technology, although there are some improvement solutions for the problem of point cloud rendering efficiency, such as using spatial index structures such as octrees to organize and manage point cloud data to reduce the number of traversals during the rendering process; or using technologies such as frustum culling and backface culling to remove invisible point cloud data to reduce the rendering burden. However, these methods still have problems such as low computing efficiency and large memory usage when processing large-scale point cloud data, and it is difficult to meet the performance requirements of real-time interactive applications. Summary of the invention

[0006] In response to the problem of low rendering efficiency in the prior art, the present application provides a point cloud rendering method based on multi-threading and LOD. Under the multi-threaded parallel framework, the LOD detail level and spatial division granularity are adaptively adjusted to efficiently render large-scale point cloud data.

[0007] The purpose of this application is achieved through the following technical solutions.

[0008] The present application provides a point cloud rendering method based on multithreading and LOD, comprising: , divide the point cloud data into n data blocks, the number of bytes of each data block is ; Calculate the starting position of each data block in the point cloud file, create n threads, and each thread reads a data block in parallel; For the n data blocks, use the octree algorithm to filter at different scales, and extract m LOD point cloud data with different detail levels from the octree of each data block; Among them, the detail level is determined by the spatial division granularity of the octree. The smaller the granularity, the higher the detail level and the denser the point cloud data; Calculate the proportion of the current point cloud size to the rendering viewport , and the distance from the viewpoint to the camera , based on a preset viewport ratio threshold and sight distance threshold , calculate the LOD point cloud detail level to be rendered ; According to the LOD point cloud detail level to be rendered , select the point cloud data corresponding to the level of detail from the m LOD point cloud data as the current LOD point cloud data to be rendered ; Get the user's interactive operation events, and for each interactive operation event, perform parallel coordinate transformation and lighting calculation on the LOD point cloud data render_data to be rendered in different threads; after all calculation threads are completed, submit the calculation results to the renderer, and the renderer renders the submitted LOD point cloud data; among them, interactive operation events include rotation, scaling and translation.

[0009] Among them, LOD is the abbreviation of Level of Detail, which means multiple levels of detail. LOD point cloud data refers to a series of point cloud data sets with different levels of detail generated by simplifying and sampling the original point cloud. LOD point cloud data is usually organized in data structures such as octrees, and nodes at different levels represent point cloud representations of different granularities. During rendering, the appropriate LOD level is dynamically selected for display based on factors such as viewpoint position and viewport size to improve rendering efficiency. The level of detail represents the degree of simplification of the LOD point cloud data and is an indicator of the fineness of the point cloud. In a spatial division structure such as an octree, the depth of the tree corresponds to different levels of detail. The leaf nodes of the tree represent the original and most complete point cloud data with the highest level of detail; the root of the tree represents the coarsest point cloud representation with the lowest level of detail. During the rendering process, selecting the appropriate level of detail based on factors such as the viewpoint position can reduce computational overhead and improve rendering performance while ensuring visual quality. Indicates the proportion of the currently rendered point cloud in the screen viewport. It is generally calculated by dividing the screen pixel area of ​​the current point cloud by the total pixel area of ​​the viewport. This ratio reflects the relative size of the point cloud rendering object in the field of view and is an important parameter affecting rendering performance and quality. The larger the ratio, the more prominent the currently rendered point cloud is in the field of view, and a higher level of detail should be used; the smaller the ratio, the less important the point cloud rendering object is in the field of view, and a lower level of detail can be used. Indicates the distance from the current viewpoint (observer position) to the camera. In 3D rendering, the camera represents the point of observation. The distance of the viewpoint from the camera directly affects the size and clarity of the object on the screen. The farther the distance, the smaller the pixel area of ​​the object on the screen, and the lower the level of detail required for rendering; the closer the distance, the larger the pixel area of ​​the object on the screen, and the higher the level of detail required for rendering. Therefore, It is another key parameter that determines the LOD level selection.

[0010] Further, calculate the LOD point cloud detail level to be rendered The formula is:

[0011] ;in, Indicates the distance from the viewpoint to the camera; Represents the preset reference distance, which is used to control the rate of change of the LOD detail level; Indicates the ratio of the current point cloud size to the rendering viewport; Represents a parameter; C represents a preset constant used to adjust the range of changes in the LOD detail level.

[0012] in, Is a preset distance value that controls the rate at which the LOD level of detail changes. When the viewpoint distance is equal to this value, the LOD level of detail will change by one unit. Parameters Used to control the relationship between LOD level of detail and viewport scale. The value range of is usually between 0 and 1. =0, the formula degenerates to only consider the line of sight factor; when =1, the formula degenerates to only consider the viewport ratio factor. In general, l can be set to 0.5, indicating that the view distance and viewport ratio each account for 50% of the LOD detail level. However, if the point cloud density in the scene is uneven, or if you want to emphasize the details near or far, you can adjust the value of l appropriately. For example, if you want the point cloud near to maintain high details, while the point cloud far away can be appropriately simplified, you can increase l, such as 0.6 to 0.8.

[0013] Preset Constants It is used to adjust the variation of LOD detail level, that is, to control the difference in details between different LOD levels. The larger it is, the greater the difference in detail between adjacent LOD levels; The smaller it is, the smaller the difference in detail between adjacent LOD levels. The value range of needs to be determined according to the specific point cloud data and the desired visual effect. Generally speaking, It can be set to an integer between 1 and 5. If the density of the point cloud changes slowly, or you want the rendering effect to transition smoothly, you can Set it to a smaller value, such as 1 to 2; if the density of the point cloud changes dramatically, or you want to emphasize the differences between different LOD levels, you can Set it to something larger, like 3 to 5.

[0014] Furthermore, according to the total number of bytes of the point cloud file , divide the point cloud data into n data blocks, the number of bytes of each data block is , including: Get the total number of bytes of the point cloud file ; According to the preset data block size , calculate the number of data blocks n, the calculation formula is: ; Where ceil represents the upward rounding function; according to the number of data blocks n, the point cloud data is divided into n data blocks, and the number of bytes of each data block does not exceed , the number of bytes of the point cloud data of the kth data block is : ; Where k represents the number of the data block, the value range is [1, n], and min represents the minimum value function.

[0015] Furthermore, the starting position of each data block in the point cloud file is calculated, and n threads are created. Each thread reads a data block in parallel, including: for the kth data block, its starting position in the point cloud file is calculated , the calculation formula is: ; k represents the number of the data block, and its value range is [1, n]; n threads are created, each thread corresponds to a data block, and the kth thread is responsible for reading the point cloud data of the kth data block; in the kth thread, according to the starting position of the kth data block and data block size , use the file random access method to read the point cloud data of the data block from the point cloud file, and the number of bytes of data read is ; Store the point cloud data read by the kth thread in a pre-allocated shared memory block In , the starting address of the shared memory block is: ;in, Indicates the starting address of the kth data block in the shared memory block; all threads perform the read operation in parallel, and after all threads are completed, the shared memory block The complete point cloud data is stored in for use in subsequent steps.

[0016] Furthermore, the n data blocks are filtered at different scales using the octree algorithm, and m LOD point cloud data with different detail levels are extracted from the octree of each data block, including: For each data block in the octree, the octree algorithm is used to divide the space and generate the corresponding octree , the root node of the octree represents the bounding box of the entire data block, and the child nodes represent the bounding boxes of the subspaces after recursive partitioning; set m different space partitioning granularities ,in, Indicates the LOD level, the value range is [1, m]; for each data block, the octree , perform m filtering to obtain n×m LOD point cloud data , where k represents the number of the data block and l represents the LOD level; for each LOD level l, merge the LOD point cloud data corresponding to all data blocks to obtain the LOD point cloud data of the lth layer : ; Merge represents a function that merges multiple point cloud data sets into a complete point cloud data set. The merged point cloud data set contains the point cloud data in all input sets and does not contain duplicate point cloud data.

[0017] Furthermore, m different space division granularities are set , including: determining the spatial division granularity of the first layer LOD The scaling factor of the spatial division granularity between two adjacent LOD levels Preferably, by analyzing the density change of the point cloud data between two adjacent LOD levels, adaptively adjust The value of makes the spatial division granularity scaling between adjacent LOD levels more reasonable, ensuring smooth transition and gradual loss of details between different LOD levels. Specifically, the point cloud data density change between two adjacent LOD levels is calculated. Layer LOD and Layer LOD, you can select a fixed-size cube as the sampling area, count the points of the two LOD levels in the sampling area, and then calculate the ratio of the points to get the density change factor . ;in, Indicates The number of points in the sampling area of ​​the layer LOD, Indicates The number of points in the sampling area of ​​the layer LOD. The scaling factor of the spatial partition granularity after adaptive adjustment , update The spatial division granularity of layer LOD . ; Through adaptive adjustment The value of can make the spatial division granularity scaling between adjacent LOD levels more reasonable, ensuring smooth transition and gradual loss of details between different LOD levels.

[0018] according to and , calculate the The spatial division granularity of layer LOD , the calculation formula is: , where pow represents a power function; preferably, , The weight factor representing the spatial division granularity of the first LOD layer can be dynamically adjusted according to the importance of different LOD levels, the distribution characteristics of point cloud data, etc. By introducing the weight factor, the spatial division granularity of different LOD levels can be controlled more flexibly, and the generation quality and efficiency of LOD point cloud data can be improved.

[0019] Furthermore, for each data block, the octree , perform m filtering, including: starting from the root node of the tree, recursively traversing each node of the octree; for the currently visited node node, calculate the spatial density of the point cloud data contained in the corresponding node , the calculation formula is: ;in, Indicates the number of point cloud data contained in the node node. Represents the volume of the spatial bounding box corresponding to the node node; calculates the normal vector variance of the local point cloud data of the node node , as the geometric complexity measure of the corresponding node, is calculated as: ;in, Represents the normal vector of the i-th point in node node, Represents the average value of the normal vectors of all points in the node node; according to the spatial density of the node node , normal vector variance and Space partition granularity , calculate the importance of node details , the calculation formula is: ;Will With preset threshold For comparison: If Less than or equal to , then add the point cloud data of node node to the LOD point cloud data of layer l , and stop traversing the child nodes of the corresponding node; if Greater than , then continue to recursively traverse the child nodes of node node.

[0020] Furthermore, the user's interactive operation events are obtained, and for each interactive operation event, the LOD point cloud data to be rendered is processed in different threads. Perform parallel coordinate transformation and lighting calculation, including: obtaining user interaction events; for each interaction event, calculating the corresponding coordinate transformation matrix according to the event type and parameters and lighting parameters ; For each interactive operation event, create Threads, Is a positive integer. Each thread has a certain number of LOD point cloud data to be rendered. A continuous subset of Perform coordinate transformation and lighting calculation; in each thread, according to the coordinate transformation matrix and lighting parameters , the subset responsible for the thread Perform coordinate transformation and illumination calculation on each point in the point cloud to obtain the transformed point cloud subset ; After all calculation threads are completed, the transformed point cloud subsets calculated by each thread are Merge into a complete point cloud data set: ;in, Represents the transformed point cloud subset calculated by the i-th thread, and the value range of t is The merge function is used to merge multiple point cloud subsets into a complete point cloud data set according to the original point index order; the merged point cloud data Submit to the renderer; the renderer processes the submitted point cloud data Rendering is performed to generate the final point cloud visualization image.

[0021] Furthermore, each thread is divided into continuous subsets , through the following steps: The point cloud data in is divided into A continuous subset , the value range of t is ; Each subset The number of point cloud data included for: ;in, express The total number of point cloud data, ceil represents the upward rounding function; the tth thread is responsible for processing the tth subset .

[0022] Furthermore, in each thread, according to the coordinate transformation matrix and lighting parameters , the subset responsible for the thread Perform coordinate transformation and illumination calculation on each point in the point cloud to obtain the transformed point cloud subset , including: for the subset For each point in , multiply the coordinates of point by the coordinate transformation matrix , and get the transformed coordinates ; According to the normal vector of point point , lighting parameters And the transformed coordinates , calculate the light intensity of point point , the lighting calculation adopts the Phong lighting model; the transformed coordinates and light intensity As the new coordinates and color value of point point, add it to the transformed point cloud subset middle.

[0023] Compared with the prior art, the advantages of this application are:

[0024] By dividing large-scale point cloud data into multiple data blocks and using multi-threaded parallel reading, the method of the present application can significantly improve the loading and transmission efficiency of point cloud data. Compared with the traditional serial reading method, multi-threaded parallel reading can make full use of the multi-core processing power of the computer and accelerate the loading process of point cloud data. At the same time, by dividing the point cloud data into multiple data blocks, the size of a single data block can be reduced, thereby reducing the delay of data reading and improving the concurrency of data transmission.

[0025] The method of the present application generates point cloud data with multiple LOD detail levels by using the octree algorithm to perform spatial division and filtering at different scales for each data block. This multi-level LOD representation method can adaptively select the current LOD detail level to be rendered based on parameters such as viewpoint position and viewport size, and dynamically filter visible point cloud data. Compared with directly rendering the original point cloud data, this method can significantly reduce the rendering overhead of redundant data, reduce the data redundancy of point cloud rendering, and improve rendering efficiency.

[0026] During the point cloud rendering process, user interaction operations (such as translation, rotation, scaling, etc.) require real-time coordinate transformation and lighting calculation of point cloud data, which places high demands on rendering performance. The method of the present application uses multi-threading to perform parallel coordinate transformation and lighting calculation on visible point cloud data, and distributes the computing tasks to multiple threads for concurrent execution, which can significantly improve the real-time interactive response speed during point cloud rendering. Compared with single-threaded computing, multi-threaded parallel computing can make full use of the multi-core processing power of modern computers, accelerate the transformation and lighting calculation of point cloud data, and thus achieve a smoother and real-time user interaction experience.

[0027] This application adopts a data screening strategy based on importance, giving priority to rendering point cloud data that has a greater impact on visual effects. By calculating indicators such as the spatial density and geometric complexity of the point cloud data, its contribution to the rendering quality is evaluated, and screening is performed according to the importance threshold. This strategy can minimize the rendering of point cloud data that has less impact on visual effects while ensuring rendering quality, thereby reducing computing overhead and improving rendering performance. Compared with ordinary equal-interval sampling methods, data screening based on importance can more efficiently and specifically select key point cloud data to optimize rendering performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present application will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0029] Figure 1 is an exemplary flow chart of a point cloud rendering method based on multithreading and LOD according to some embodiments of the present application;

[0030] Figure 2 This is an exemplary flowchart of rendering interaction according to some embodiments of the present application. DETAILED DESCRIPTION

[0031] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0032] Figure 1 is an exemplary flow chart of a point cloud rendering method based on multithreading and LOD according to some embodiments of the present application, comprising: , divide the point cloud data into n data blocks, the number of bytes of each data block is ; Calculate the starting position of each data block in the point cloud file, create n threads, and each thread reads a data block in parallel; For the n data blocks, use the octree algorithm to filter at different scales, and extract m LOD point cloud data with different detail levels from the octree of each data block; Among them, the detail level is determined by the spatial division granularity of the octree. The smaller the granularity, the higher the detail level and the denser the point cloud data; Calculate the proportion of the current point cloud size to the rendering viewport , and the distance from the viewpoint to the camera , based on a preset viewport ratio threshold and sight distance threshold , calculate the LOD point cloud detail level to be rendered ; According to the LOD point cloud detail level to be rendered , select the point cloud data corresponding to the level of detail from the m LOD point cloud data as the current LOD point cloud data to be rendered ; Get the user's interactive operation events, and for each interactive operation event, perform parallel coordinate transformation and lighting calculation on the LOD point cloud data render_data to be rendered in different threads; after all calculation threads are completed, submit the calculation results to the renderer, and the renderer renders the submitted LOD point cloud data; among them, interactive operation events include rotation, scaling and translation.

[0033] Get the total number of bytes of the point cloud file :If the point cloud file is stored in the file system, you can use the file operation library or system API to get the number of bytes of the file, such as using the std::filesystem::file_size function in C++. If the point cloud file is stored in a database, you can query the database metadata or use the functions provided by the database to get the number of bytes of the file. According to the preset data block size , calculate the number of data blocks n: use the formula: , where ceil represents the ceiling function. In the code, you can use the ceil function provided by the math library or a custom ceiling function. According to the number of data blocks n, divide the point cloud data into n data blocks: create an array or vector blocks of length n to store the information of each data block. Traverse the data block number k, from 1 to n: calculate the starting byte position of the kth data block: . Calculate the number of bytes in the kth data block: , where min represents the minimum value function. The starting byte position and number of bytes of the kth data block are stored in blocks[k-1].

[0034] Calculate the starting position of each data block in the point cloud file, create n threads, and each thread reads a data block in parallel; including: Calculate the starting position of each data block in the point cloud file: Create an array or vector with a length of n , used to store the starting position of each data block. Traverse the data block number k, from 1 to n: calculate the starting position of the kth data block in the point cloud file , using the formula: .Will Stored in Create n threads to read the data block in parallel: Create an array or vector of threads of length n. Iterate over the data block number k, from 1 to n: Create the kth thread and store it in middle.

[0035] In the execution function of the kth thread: according to Get the starting position of the kth data block . File random access reads data blocks: According to the calculated number of bytes of the kth data block , determine the amount of data to be read. Use the file random access method to move the file pointer to the starting position of the kth data block. Starting from the current position of the file pointer, read continuously Bytes of data. The read operation uses the low-level file I / O functions provided by the system, such as read or fread, to read in bytes. The read data is stored in a temporary memory buffer. Store the read data in the shared memory block: Calculate the starting address of the kth data block in the shared memory block , the formula is: Use a memory copy function, such as memcpy, to copy the data in the temporary memory buffer to the corresponding location of the shared memory block. The starting address of the copy is , the number of bytes copied is The copy operation is performed in memory and is faster.

[0036] The thread exits after execution is completed: After the current thread completes the reading and copying of the data block, the thread execution is completed. When the thread exits, resources such as temporary memory buffers are released. The way the thread exits depends on the parallel programming framework or library used, such as or Wait for all threads to finish executing: The main thread or control thread traverses the thread array or vector threads and checks the execution status of each thread one by one. Use thread synchronization primitives such as or , waiting for each thread to finish executing. When all threads have finished executing, the reading and copying operations of all data blocks have been completed. Complete point cloud data in shared memory blocks: After all threads have finished executing, the shared memory blocks The complete point cloud data is stored in the shared memory block. The starting address of the shared memory block is , the total number of bytes is Subsequent data processing steps can directly access the shared memory block without having to read data from the file. The use of shared memory blocks can improve the efficiency of data access and reduce the overhead of file I / O operations.

[0037] For n data blocks, the octree algorithm is used to filter at different scales, and m LOD point cloud data with different detail levels are extracted from the octree of each data block; the detail level is determined by the spatial division granularity of the octree. The smaller the granularity, the higher the detail level and the denser the point cloud data. First, traverse each data block in the shared memory block: according to the shared memory block The starting address and size of each data block , calculate the number of data blocks n in the shared memory block. Use a loop or iterator to access each data block in the shared memory block one by one. For the kth data block, its starting address in the shared memory block is: According to the starting address and size of the data block, the point cloud data of the data block is read and stored in a temporary memory buffer.

[0038] Create an octree object: For the kth data block, create an empty octree object , used to store the spatial partitioning results of the data block. The octree object can be implemented using a class or structure, which contains attributes such as the root node pointer and the number of nodes. Initialized to NULL, indicating an empty octree. Calculate the bounding box of the data block: traverse all point cloud data in the kth data block and find the minimum and maximum values ​​in the three dimensions of x, y, and z. Suppose the coordinates of the point cloud data are represented as (x, y, z), then the minimum point of the bounding box is and maximum point The coordinates of are calculated as follows: ; ;in, , , Respectively represent the minimum values ​​of the x, y, and z coordinates of all points in the data block. , , Respectively represent the maximum values ​​of the x, y, and z coordinates of all the points in the data block.

[0039] The bounding box can be represented by a structure To represent, including the coordinates of the minimum point and the maximum point. Start the recursive space division process: the point cloud data of the data block and the bounding box of the root node Passed to the octree construction function .exist In the function, create a new node object node, which represents the octree node currently being processed. Assign the bounding box of the root node to the bounding box of the current node node.bounding_box. Recursively call Function to start the space partitioning process.

[0040] Recursively partition the octree space: In the function, whether the space needs to be further divided is determined according to the following conditions: If the bounding box size of the current node is less than or equal to the preset space division threshold , then mark the current node as a leaf node and stop dividing. If the number of point cloud data contained in the current node is less than or equal to the preset point count threshold , then mark the current node as a leaf node and stop dividing. If you need to further divide the space, perform the following steps: divide the bounding box node.bounding_box of the current node equally in the three dimensions of x, y, and z to obtain eight sub-bounding boxes For each sub-bounding box , divide the point cloud data contained in the current node into corresponding child nodes according to its coordinate range. For each child node, recursively call Function, pass in the corresponding child bounding box and point cloud data, and continue the space division. The recursive process continues until the condition of the leaf node is reached (bounding box size or point number threshold). Calculate the bounding box and point cloud data of the node during the generation of the octree: In the process of recursive division of space, calculate the corresponding bounding box node.bounding_box for each node. The bounding box is calculated by finding the minimum and maximum coordinate values ​​of the point cloud data contained in the node. When dividing the point cloud data into child nodes, the bounding box of the child node is updated at the same time to ensure that the bounding box can correctly contain all the point cloud data. For leaf nodes, the point cloud data they contain is stored in the node's attribute node.points for subsequent access and processing.

[0041] Use the octree algorithm to perform spatial division on each data block. First, create an empty octree object and calculate the bounding box of the data block as the bounding box of the root node. Then, pass the point cloud data of the data block and the bounding box of the root node to the construction function of the octree to start the recursive spatial division process. In the recursive process, determine whether the space needs to be further divided according to the bounding box size and point number threshold of the current node. If division is required, divide the bounding box of the current node into eight sub-bounding boxes and divide the point cloud data into the corresponding sub-nodes. The recursive process continues until the condition of the leaf node is reached. In the process of generating the octree, calculate the corresponding bounding box for each node, and store the point cloud data of the leaf node in the node's attributes. Through the spatial division of the octree, the point cloud data can be organized into a hierarchical structure, which is convenient for subsequent LOD point cloud data extraction and rendering. The storage of the node bounding box and point cloud data of the octree can be implemented through structures or classes to achieve efficient data access and processing.

[0042] Set different space division granularities: Determine the space division granularity of the first layer LOD , for example, set to 0.1, indicating that the spatial division granularity of the first LOD layer is one tenth of the data block side length. Determine the scaling factor of the spatial division granularity between two adjacent LOD levels , for example, set to 0.5, it means that each time you increase the LOD level, the spatial division granularity is reduced to half of the original. and , calculate the spatial division granularity of the lth layer LOD , using the power function: ; where l represents the LOD level, ranging from 1 to m, and m is the maximum number of LOD levels. is 0.1, When it is 0.5, the spatial division granularity of the second layer LOD 0.05, the spatial division granularity of the third level LOD is 0.025, and so on. Different spatial division granularities correspond to different LOD levels. The smaller the granularity, the higher the LOD level and the denser the point cloud data.

[0043] Filter the octree of each data block to obtain point cloud data at different LOD levels: For the octree of each data block , perform m filtering to obtain n × m LOD point cloud data , where k represents the number of the data block and l represents the LOD level. Starting from the root node of the tree, recursively traverse each node of the octree. For the currently visited node node, calculate the spatial density of the point cloud data contained in the corresponding node and the normal vector variance of the local point cloud data . Spatial density The calculation formula is: ,in Indicates the number of point cloud data contained in the node, and node_volume indicates the volume of the node. Normal vector variance The calculation formula is: ,in Represents the normal vector of each point in the node, Represents the average of the normal vectors of all points in the node.

[0044] Calculate the importance of the details of the node node based on the spatial density, geometric complexity and the lth spatial partition granularity of the node node. . The importance of details The calculation formula is: ,in For the The spatial division granularity of the layer LOD. With preset threshold Compare and decide whether to add the point cloud data of node node to the LOD point cloud data of layer l If Less than or equal to , then add the point cloud data of node node to , and stop traversing the child nodes of the corresponding node. Greater than , then continue to recursively traverse the child nodes of node node. Merge the LOD point cloud data corresponding to all data blocks: For each LOD level l, merge the LOD point cloud data corresponding to all data blocks to obtain the LOD point cloud data of the lth layer The merge operation is implemented through the merge function, which merges multiple point cloud data sets into a complete point cloud data set. The merged point cloud data set contains the point cloud data in all input sets and does not contain duplicate point cloud data.

[0045] The merging operation can be performed in a parallel manner, such as using parallel computing frameworks such as OpenMP or CUDA to improve the efficiency of data processing. According to the set spatial division granularity and LOD level, the octree of each data block is filtered to obtain point cloud data of different LOD levels. First, the spatial division granularity of the first layer LOD and the scaling factor between adjacent LOD levels are determined, and the spatial division granularity of different LOD levels is obtained by power function calculation. Then, the octree of each data block is recursively traversed to calculate the spatial density, normal vector variance and detail importance of the node, and the threshold is used to determine whether to add the point cloud data of the node to the point cloud data of the corresponding LOD level. Finally, for each LOD level, the LOD point cloud data corresponding to all data blocks are merged to obtain complete LOD point cloud data.

[0046] For each LOD level l, merge the LOD point cloud data corresponding to all data blocks to obtain the l-th layer of LOD point cloud data to calculate the current LOD point cloud detail level to be rendered The formula is: ;in, Indicates the distance from the viewpoint to the camera; Represents the preset reference distance, which is used to control the rate of change of the LOD detail level; Indicates the ratio of the current point cloud size to the rendering viewport; Represents a parameter; C represents a preset constant, which is used to adjust the variation of the LOD detail level; during the rendering process, the distance view_distance from the viewpoint to the camera is calculated in real time, which can be calculated by the position of the camera and the position of the viewpoint.

[0047] Set the preset reference distance , used to control the rate of change of the LOD detail level. It can be adjusted according to specific application scenarios and performance requirements. Calculate the ratio of the current point cloud size to the rendering viewport , which can be calculated by the bounding box of the point cloud and the size of the viewport. Set the parameter l to control the rate of change of the LOD detail level. The value range of l can be adjusted according to the specific application scenario and performance requirements, and is usually between 0 and 1. Set the preset constant C to adjust the range of change of the LOD detail level. The value of C can be adjusted according to the specific application scenario and performance requirements, and is usually between 0 and 1. log2 represents a logarithmic function with base 2, which is used to map the viewpoint distance to the range of change of the LOD detail level. Indicates the ratio of the viewpoint distance to the reference distance. The larger the value, the higher the LOD detail level. l is used to control the rate of change of the LOD detail level. The larger the value, the faster the LOD detail level changes. Indicates the impact of the current point cloud size on the LOD detail level. The larger the value, the higher the LOD detail level.

[0048] Figure 2 is an exemplary flowchart of rendering interaction shown in some embodiments of the present application, calculating the ratio of the current point cloud size to the rendering viewport : Get the width of the current rendering viewport and height , in pixels. Calculate the bounding box of the current point cloud and get its projection size on the X and Y axes and , expressed in the same units. Calculate the proportion of the point cloud on the X-axis and Y-axis and : ; ;Pick and The maximum value is the ratio of the current point cloud size to the rendering viewport : ; The value range is 0 to 1, indicating the relative size of the point cloud in the viewport.

[0049] Calculate the distance from the viewpoint to the camera : Get the current camera position , expressed in the form of three-dimensional coordinates (x, y, z). Get the position of the current viewpoint , expressed in the form of three-dimensional coordinates (x, y, z). Calculate the Euclidean distance between the camera position and the viewpoint position as the distance of the viewpoint from the camera : ; The units are consistent with the camera position and viewpoint position, usually expressed in meters or world coordinate units. and sight distance threshold , calculate the LOD point cloud detail level to be rendered : Set the preset viewport ratio threshold , which indicates the minimum acceptable size of the point cloud in the viewport. Less than , it means that the point cloud is too small in the viewport and the LOD level of detail needs to be reduced.

[0050] Set a preset sight distance threshold , represents the maximum acceptable distance between the viewpoint and the camera. Greater than , it means that the viewpoint is too far from the camera and the LOD level of detail needs to be lowered. Calculate the LOD point cloud level of detail to be rendered :if Less than ,or Greater than , then reduce the LOD detail level: ; Otherwise, keep the current LOD detail level: ; The value range is 1 to m, which indicates the level of detail of the LOD point cloud data to be rendered, and m is the maximum level of detail of the LOD point cloud data.

[0051] Through the point cloud container, the point cloud container is a data structure for storing and managing point cloud data. Its main function is to organize and maintain point cloud data of different LOD levels for fast retrieval and access during rendering. In the LOD-based point cloud rendering method, the original point cloud data is usually divided into multiple data blocks, and after simplification and sampling, LOD point cloud data of different levels of detail are generated. These LOD point cloud data need to be stored and managed effectively so that the appropriate level of detail can be dynamically selected according to the viewpoint position and viewport size during rendering. Point cloud containers usually use a tree-like hierarchical structure, such as an octree or a kd-tree, to organize point cloud data of different LOD levels. Each tree node represents a spatial region and contains a subset of the point cloud within the region. The parent node represents a lower LOD level, and the child node represents a higher LOD level. By traversing the tree structure, you can quickly locate the LOD level suitable for the current rendering parameters and obtain the corresponding point cloud data.

[0052] According to the LOD point cloud detail level to be rendered , select the point cloud data corresponding to the level of detail from the m LOD point cloud data as the current LOD point cloud data to be rendered :according to The value of m LOD point cloud data arrive Select the point cloud data corresponding to the detail level: . Select the point cloud data Pass it to the rendering pipeline for rendering and displaying the point cloud. During the rendering process, the current point cloud size is calculated in real time as a percentage of the rendering viewport and the distance from the viewpoint to the camera, and the LOD point cloud detail level to be rendered is calculated based on the preset threshold. First, obtain the size of the rendering viewport and the bounding box of the point cloud, and calculate the relative size of the point cloud in the viewport. Then, the Euclidean distance view_distance between the camera position and the viewpoint position is calculated. and Compare the result with the preset threshold to calculate the LOD point cloud detail level to be rendered Finally, select the point cloud data corresponding to the level of detail from the m LOD point cloud data as the point cloud data to be rendered. , and passed to the rendering pipeline for rendering and display.

[0053] Get the user's interactive operation events, and for each interactive operation event, process the LOD point cloud data to be rendered in different threads Perform parallel coordinate transformation and lighting calculations; after all calculation threads are completed, submit the calculation results to the renderer, which renders the submitted LOD point cloud data; among them, interactive operation events include rotation, scaling and translation. First, obtain the user's interactive operation events: listen to the user's mouse, keyboard or touch events, such as mouse press, move, release, keyboard key, touch screen, etc. For each event, obtain the event type (such as rotation, scaling, translation) and related parameters (such as rotation angle, scaling factor, translation vector). and parameters Encapsulated as an interactive operation event , add to the event queue Waiting for processing.

[0054] For each interactive operation event, calculate the corresponding coordinate transformation matrix based on the event type and parameters. and lighting parameters : From the event queue Get an interactive operation event . According to the event type and parameters , calculate the corresponding coordinate transformation matrix : For rotation events: Get the rotation axis and rotation angle (in radians). and rotation angle , construct the rotation matrix : is the unit vector (x, y, z), is the radian value; ; ; ; For zoom events: Get the zoom factor , and , which represent the scaling ratios along the X-axis, Y-axis, and Z-axis respectively. , and , construct the scaling matrix : For translation events: Get the translation vector , represents the translation along the X-axis, Y-axis, and Z-axis. According to the translation vector , construct the translation matrix : translation vector for ; . The rotation matrix , scaling matrix and translation matrix Multiply to get the final coordinate transformation matrix : .

[0055] According to event type and parameters , calculate the corresponding lighting parameters : For rotation events: According to the rotation matrix , update the light source direction . Assume that the initial light source direction is , the updated light source direction is: ; Updated light source direction is the rotation matrix Multiply by the initial light direction, ignoring the fourth component.

[0056] For zoom events: according to the zoom factor , update the light intensity Assuming the initial light intensity is intensity, the updated light intensity is: ; Updated light intensity The product of the initial light intensity and the scaling factor. For translation events: Lighting parameters Unchanged. Get the coordinate transformation matrix corresponding to each interactive operation event and lighting parameters . Coordinate transformation matrix It will be used to transform the coordinates of the point cloud data, converting the point cloud data from the original coordinate system to the transformed coordinate system. It will be used to update the light source direction and intensity in the lighting calculation, affecting the lighting effect of the point cloud data.

[0057] For each interaction event, create threads, each thread processes the LOD point cloud data to be rendered. A continuous subset of Performing coordinate transformation and lighting calculations: Determining the number of threads , can be adjusted dynamically according to hardware conditions and data size. Get the current LOD point cloud data to be rendered ,Include Points. Divide into points according to their index order A continuous subset : Calculate the number of points contained in each subset , using the formula , where ceil represents the upward rounding function. For the tth subset , whose starting point index is , the end point index is .create Threads , the value range of t is .

[0058] In each thread , perform the following steps: Get the subset that the thread is responsible for . According to the coordinate transformation matrix and lighting parameters ,right Perform coordinate transformation and lighting calculation for each point in: For each point point, multiply its coordinates by , and get the transformed coordinates . According to the normal vector of point point , lighting parameters And the transformed coordinates , use the Phong lighting model to calculate the lighting intensity of the point . The transformed coordinates and light intensity As the new coordinates and color value of point point, add it to the transformed point cloud subset Thread After the calculation is completed, Join the thread synchronization queue middle.

[0059] After all calculation threads are completed, the transformed point cloud subsets calculated by each thread are merged into a complete point cloud data set: Create an empty point cloud data set . Synchronize the queue from the thread Take out the transformed point cloud subset calculated by each thread in turn .Will The point cloud data in is merged into Repeat the above steps until the thread synchronizes the queue Empty to get the final merged point cloud data The merged point cloud data Submit to the renderer: Submitted to the renderer through the renderer's data interface. The renderer processes the submitted point cloud data. Rendering is performed to generate the final point cloud visualization screen. In response to the user's interactive operation events, coordinate transformation and lighting calculation are performed on the LOD point cloud data in parallel, and finally an interactive point cloud visualization result is generated. By dividing the point cloud data into multiple subsets and performing calculations in independent threads, the parallel processing capabilities of multi-core CPUs can be fully utilized to improve computing efficiency. At the same time, by dynamically adjusting the number of threads and the size of the point cloud subsets, load balancing can be performed according to hardware conditions and data scale to ensure real-time performance and response speed. In the data merging and rendering submission stages, thread synchronization and data reorganization can be used to ensure the correctness and consistency of point cloud data and avoid concurrent access and data competition issues.

Claims

1. A point cloud rendering method based on multithreading and LOD, characterized in that: include: According to the total number of bytes of the point cloud file , divide the point cloud data into n data blocks, the number of bytes in each data block is ; Calculate the starting position of each data block in the point cloud file, create n threads, and each thread reads a data block in parallel; For n data blocks, the octree algorithm is used to filter at different scales, and m LOD point cloud data with different detail levels are extracted from the octree of each data block; the detail level is determined by the spatial division granularity of the octree. The smaller the granularity, the higher the detail level and the denser the point cloud data. Calculate the ratio of the current point cloud size to the rendering viewport , and the distance of the viewpoint from the camera , based on a preset viewport ratio threshold and sight distance threshold , calculate the LOD point cloud detail level to be rendered ; According to the LOD point cloud detail level to be rendered , select the point cloud data corresponding to the level of detail from the m LOD point cloud data as the current LOD point cloud data to be rendered ; Get the user's interactive operation events, and for each interactive operation event, perform parallel coordinate transformation and lighting calculation on the LOD point cloud data render_data to be rendered in different threads; after all calculation threads are completed, submit the calculation results to the renderer, and the renderer renders the submitted LOD point cloud data; among them, interactive operation events include rotation, scaling and translation.

2. The point cloud rendering method based on multithreading and LOD according to claim 1, characterized in that: Calculate the LOD point cloud detail level to be rendered The formula is: ； in, Indicates the distance from the viewpoint to the camera; Represents the preset reference distance, which is used to control the rate of change of the LOD detail level; Indicates the ratio of the current point cloud size to the rendering viewport; Represents a parameter; C represents a preset constant used to adjust the range of changes in the LOD detail level.

3. The point cloud rendering method based on multithreading and LOD according to claim 2, characterized in that: Divide the point cloud data into n data blocks, and the number of bytes in each data block is ,include: Get the total number of bytes of the point cloud file ; According to the preset data block size , calculate the number of data blocks n, the calculation formula is: ； Among them, ceil represents the upward rounding function; According to the number of data blocks n, the point cloud data is divided into n data blocks, and the number of bytes of each data block does not exceed , the number of bytes of the point cloud data of the kth data block is : ； Among them, k represents the number of the data block, the value range is [1, n], and min represents the minimum value function.

4. The point cloud rendering method based on multithreading and LOD according to claim 3, characterized in that: Calculate the starting position of each data block in the point cloud file and create n threads, including: For the kth data block, calculate its starting position in the point cloud file , the calculation formula is: ； Wherein, k represents the number of the data block, and its value range is [1, n]; Create n threads, each thread corresponds to a data block, and the kth thread is responsible for reading the point cloud data of the kth data block; In the kth thread, according to the starting position of the kth data block and data block size , use the file random access method to read the point cloud data of the data block from the point cloud file, and the number of bytes of data read is ; Store the point cloud data read by the kth thread in a pre-allocated shared memory block In , the starting address of the shared memory block is: ； in, Indicates the starting address of the kth data block in the shared memory block; All threads perform read operations in parallel, and after all threads have completed, the shared memory block The complete point cloud data is stored in for use in subsequent steps.

5. The point cloud rendering method based on multithreading and LOD according to claim 4, characterized in that: Extract m LOD point cloud data of different levels of detail from the octree of each data block, including: For shared memory blocks For each data block in the octree, the octree algorithm is used to divide the space and generate the corresponding octree ,The root node of the octree represents the bounding box of the entire data block, and the child nodes represent the bounding boxes of the subspaces after recursive partitioning; Set m different space division granularities ,in, Indicates the LOD level, the value range is [1, m]; For each data block, the octree , perform m filtering to obtain n×m LOD point cloud data , where k represents the number of the data block and l represents the LOD level; For each LOD level l, merge the LOD point cloud data corresponding to all data blocks to obtain the LOD point cloud data of the lth layer : ； Among them, merge represents a function that merges multiple point cloud data sets into a complete point cloud data set. The merged point cloud data set contains the point cloud data in all input sets and does not contain duplicate point cloud data.

6. The point cloud rendering method based on multithreading and LOD according to claim 5, characterized in that: Set m different space division granularities ,include: Determine the spatial division granularity of the first level LOD The scaling factor of the spatial division granularity between two adjacent LOD levels ; according to and , calculate the spatial division granularity of the lth layer LOD , the calculation formula is: ； Here, pow represents a power function.

7. The point cloud rendering method based on multithreading and LOD according to claim 6, characterized in that: For each data block, the octree , perform m filtering, including: Starting from the root node of the tree, recursively traverse each node of the octree; For the currently visited node, calculate the spatial density of the point cloud data contained in the corresponding node , the calculation formula is: ； in, Indicates the number of point cloud data contained in the node node. Indicates the volume of the spatial bounding box corresponding to the node node; Calculate the normal vector variance of the local point cloud data of node node , as the geometric complexity measure of the corresponding node, is calculated as: ； in, Represents the normal vector of the i-th point in node node, Represents the average value of the normal vectors of all points in the node node; According to the spatial density of the node , normal vector variance and Space partition granularity , calculate the importance of node details , the calculation formula is: ； Will With preset threshold For comparison: if Less than or equal to , then add the point cloud data of node node to the LOD point cloud data of layer l and stop traversing the child nodes of the corresponding node; if Greater than , then continue to recursively traverse the child nodes of node node.

8. The point cloud rendering method based on multithreading and LOD according to claim 5, characterized in that: For each interactive operation event, the LOD point cloud data to be rendered is processed in different threads. Perform parallel coordinate transformation and lighting calculations, including: Get user interaction events; For each interactive operation event, calculate the corresponding coordinate transformation matrix based on the event type and parameters. and lighting parameters ; For each interaction event, create Threads, Is a positive integer. Each thread has a certain number of LOD point cloud data to be rendered. A continuous subset of Perform coordinate transformation and lighting calculation; In each thread, according to the coordinate transformation matrix and lighting parameters , the subset responsible for the thread Perform coordinate transformation and lighting calculation on each point in the point cloud to obtain the transformed point cloud subset ; After all the calculation threads are completed, the transformed point cloud subsets calculated by each thread are Merge into a complete point cloud data set: ; in, Represents the transformed point cloud subset calculated by the i-th thread, and the value range of t is ,The function of the merge function is to merge multiple point cloud subsets into a complete point cloud data set according to the original point index order; The merged point cloud data Submit to the renderer; The renderer processes the submitted point cloud data. Rendering is performed to generate the final point cloud visualization image.

9. The point cloud rendering method based on multithreading and LOD according to claim 8, characterized in that: Divide each thread into a continuous subset , through the following steps: Will The point cloud data in is divided into A continuous subset , the value range of t is ; Each subset The number of point cloud data included for: ； in, express The total number of point cloud data, ceil represents the upward rounding function; The tth thread is responsible for processing the tth subset .

10. The point cloud rendering method based on multithreading and LOD according to claim 9, characterized in that: Subset responsible for threads Perform coordinate transformation and lighting calculation on each point in the point cloud to obtain the transformed point cloud subset ,include: For a subset For each point in , multiply the coordinates of point by the coordinate transformation matrix , and get the transformed coordinates ; According to the normal vector of point point , lighting parameters And the transformed coordinates , calculate the light intensity of point point , the lighting calculation adopts the Phong lighting model; The transformed coordinates and light intensity As the new coordinates and color value of point point, add it to the transformed point cloud subset middle.

Images