Mass point cloud real-time visualization method for uniform sampling

Abstract

The invention discloses a uniform sampling mass point cloud real-time visualization method, and relates to the technical field of point cloud visualization, and the method comprises the steps: carrying out the multi-thread Morton coding of original point cloud data, and obtaining a plurality of binary segment files; combining the plurality of binary segment files to obtain a target file; performing hierarchical uniform sampling on the point cloud data in the target file to obtain sampling point cloud data; constructing an LOD three-dimensional model based on the sampling point cloud data and the target file; storing each level and the focal length corresponding to the level into an array; carrying out an inclusion or intersection test by utilizing a view cone and a node bounding box, and adding LOD nodes which meet conditions and have levels smaller than or equal to a target level into a loading list; and the data is switched and uploaded to the GPU in real time through the double-buffer FBO. The technical problem that a complete technical chain which faces a common terminal and runs through importing, indexing, loading and rendering is not formed in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of point cloud visualization, and particularly relates to a method for real-time visualization of massive point clouds with uniform sampling. BACKGROUND

[0002] Currently, the production of three-dimensional point cloud data is growing exponentially. For example, a vehicle-grade LiDAR outputs 2 million points per second, and a single urban oblique photography project easily breaks through 10 billion points. In recent years, the global point cloud data volume will reach 73 ZB. Under the driving of applications such as smart cities, autonomous driving, digital cultural heritage, and metaverse, the industry's demand for "smoothly browsing massive point clouds on low-cost terminals" is becoming increasingly urgent. However, due to the bottlenecks of CPU, video memory, and mechanical hard disk random I / O, the existing public technologies cannot simultaneously meet the requirements of "minute-level import + precise LOD + 60FPS stable rendering" in an ordinary PC environment with 8GB RAM and 6GB VRAM.

[0003] Many patents have attempted to alleviate the above-mentioned contradictions: the patent application US11010930B2 adopts the ROI dynamic switching idea, but its octree is still constructed offline by a serial script; the patent application with publication number CN109345619A stores point cloud slices as multi-level files, but does not consider the coarse loading granularity caused by the camera focal length; the patent application EP3474514A1 relies on cloud snapshot lazy loading, and the client still requires high memory; the granted patents (CN111882657B, CN111402429B) focus on scale calibration or network transmission protocol, and pay insufficient attention to real-time rendering on low-end hardware. Industry commonly used tools are also restricted by these bottlenecks: the single-threaded import speed of PotreeConverter is only 3-4 million points per minute; CloudCompare can open up to 100 million points under 8GB RAM; Cesium 3D iles can only smoothly display 3 billion points in an RX 3060 environment. Therefore, current patents and products mostly focus on offline processing or high-end hardware scenarios, and have not formed a complete technical chain that faces ordinary terminals and runs through the "import-index-load-render" process. The existing point cloud visualization systems mainly face high-performance workstations or servers, and in the general PC environment with limited software and hardware resources, they often cannot meet the real-time interaction requirements due to the following technical bottlenecks: (1) single-threaded preprocessing leads to long import time; (2) large random I / O between disk and memory; (3) uneven point distance of multi-level LOD; (4) decoupling of loading granularity and camera parameters; (5) GPU data replacement blocks the rendering pipeline. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a method for real-time visualization of massive point clouds with uniform sampling, to solve the above-mentioned technical problems.

[0005] To achieve the above object, the technical scheme adopted by the present application is as follows: A uniform sampling massive point cloud real-time visualization method, comprising the following steps: S1, obtaining original point cloud data; performing multi-threaded Morgen coding on the original point cloud data to obtain a plurality of binary segment files; S2, merging the plurality of binary segment files by using a k-way merging algorithm to obtain a target file; S3, performing hierarchical uniform sampling on the point cloud data in the target file to obtain sampled point cloud data; S4, constructing a LOD three-dimensional model: grouping the sampled point cloud data to the corresponding LOD node according to the corresponding level, and recording the average point distance of the sampled point cloud data in the corresponding LOD node in the target file and the node bounding box in each LOD node; S5, based on the mapping relationship between the focal length and the average point distance, each average point distance corresponds to a level, storing each level and the focal length corresponding to each level in an array; S6, using a view frustum and a node bounding box to do containment or intersection test, adding the LOD node meeting the conditions and the level less than or equal to the target level to the loading list; S7, the background thread asynchronously reads the sampled point cloud data corresponding to the LOD node in the loading list to the CPU memory, and uploads the data to the GPU in real time through double-buffer FBO.

[0006] Further, S1 comprises: S11, setting parameters: setting the size of the point cloud block according to the hardware memory, the bit width of Morgen coding is 21 bits, the preset level is 8 layers, and the circular buffer threshold is 1 million points; S12, according to the number of CPU cores, allocating the same number of threads as the number of CPU cores and allocating an independent scalable circular buffer area for each thread; S13, loading the original point cloud file into the memory, dividing all the original point cloud data in the original point cloud file into a plurality of point cloud blocks according to the size of the point cloud block, and distributing the plurality of point cloud blocks to the multi-threaded work queue; S14, each thread uses the Morgen coding algorithm to divide the current point cloud block into multiple Morgen point cloud blocks at all levels, each Morgen point cloud block has a Morgen code; store the plurality of Morgen point cloud blocks in the scalable circular buffer area, when the total number of point clouds corresponding to the plurality of Morgen point cloud blocks in the scalable circular buffer area reaches the circular buffer threshold, the thread triggers a sequential disk writing, sorts the point clouds in the scalable circular buffer area in ascending order of Morgen code, saves the sorted point clouds to a binary segment file, and when all threads process all point cloud blocks, a plurality of binary segment files are obtained.

[0007] Further, S2 comprises: S21, read the Morton point cloud block information in each binary segment file one by one, the Morton point cloud block information comprising: level, Morton code, total number of points, and offset of each point in the binary segment file; S22, count the total number of points corresponding to the Morton point cloud block under the same level, and calculate the overall capacity required by the target file according to the total number of points; S23, reserve a continuous storage interval in the target file for each Morton point cloud block under the same level, and record the starting byte position of each interval; S24, calculate the final write address of each Morton point cloud block in the target file according to the starting byte position of each interval, and calculate the cumulative offset in the interval; S25, generate a newly named binary segment file as the target file; S26, copy the data of each Morton point cloud block to the target file according to the starting byte position and the final write address; S27, write an index record for each interval, the contents of the index record including: level, Morton code, starting address of the interval, total number of points; count the total number of all index records, total number of points in all index records, maximum and minimum values of X, Y and Z coordinates of all index records, and obtain statistical information; update the statistical information to the header information of the target file.

[0008] Further, S3 comprises: S31, for each bounding box of the Morton point cloud block in the target file and the target level, calculate the voxel edge length and the diagonal length of the point cloud data; S32, construct a first set for recording occupied voxels, and a second set for recording the first reserved point coordinate information in the occupied voxels, both the first set and the second set reserving a capacity according to the input point number divided by 8; S33, list 27 voxel coordinate offsets consisting of 1 voxel and 26 adjacent voxels of the voxel in advance; S34, for each input point cloud, perform the following operations: locate the voxel key where the current point cloud is located; if the voxel has been occupied, skip the current point; if the voxel has not been occupied, check whether there is a reserved point in the 26 adjacent voxels and the distance is less than the voxel diagonal, if there is a reserved point and the distance is less than the voxel diagonal, skip the current point; when the current point is not skipped, write the current point to the second set and record the mapping of the voxel key to the coordinates of the current point, mark the voxel as occupied in the first set, and append the real level and the average point distance; S35, output the second set and clip to the reserved capacity to obtain the sampling point cloud data.

[0009] Further, in S31, the calculation formula of the voxel side length s(L) is: ; Wherein, L is the target level, L0 is the length of one side of the bounding box.

[0010] Further, in S6, the condition means that a Morton point cloud block meets the following two conditions: Bounding box condition: one of the eight vertices of a Morton point cloud block is in the bounding box; Angle condition: one of the eight vertices of a Morton point cloud block is in the view frustum, and the vertex in the view frustum satisfies the following formula: ; Wherein, a is the vector of the line connecting the camera center to the observation center, b is the vector of the line connecting the camera center to the vertex, and FOV is the field of view angle.

[0011] Further, in S5, the mapping relationship between the focal length and the average point distance is: ; Wherein, is the focal length, is the average point distance, is the viewing angle, is the number of pixels on the longitudinal tangent of the screen within the field of view of the viewpoint.

[0012] Further, S7 comprises: a background thread asynchronously reads the sampling point cloud data corresponding to the LOD node in the loading list to the CPU memory; the main thread loads the sampling point cloud data in the CPU memory into the VBO through the API function of OpenGL; the data in the VBO is offline rendered to the FBO through the API function of OpenGL; in the display process, the OpenGL operation default buffer, the data in the FBO is drawn to the default buffer for drawing.

[0013] Compared with the prior art, the present application has the following beneficial effects: The present invention is scientifically and rationally designed and easy to use. It accelerates offline sorting through multi-threaded Morton encoding and threshold disk writing, solving the technical problem of slow import in existing technologies; maintains the exponential increase of LOD point distance through hierarchical uniform sampling technology to avoid zooming flicker; accurately limits video memory and bandwidth through focal length-level mapping and dual-threshold frustum clipping; solves the GPU-CPU synchronization blocking problem through double-buffered FBO hot switching, forming a complete technology chain for ordinary terminals that runs through "import-index-load-rendering"; actual tests show that on an ordinary laptop (8GB RAM, GX16606GB), 10 billion points can be imported within 2 hours, and browsed smoothly at a stable frame rate of 60FPS, breaking through the technical upper limit of existing patents and tools, and providing a practical solution for the lightweight application of massive point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0015] Explanation of terms: K-way merge is an extended merge sort algorithm; LOD is a method of creating multiple models with different levels of detail for the same object; CPU is the central processing unit; GPU is a graphics processing unit; OpenGL is a cross-programming language and cross-operating system 2D and 3D graphics rendering application programming interface; API is an application programming interface; VBO is the video memory area used to store vertex data in OpenGL; FBO is an object created in OpenGL that simulates the default frame buffer functionality and structure.

[0016] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0017] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation or be constructed and operated in a particular orientation, therefore it cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0018] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; of course, it can also be mechanically connected, or it can also be electrically connected; in addition, it can also be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0019] As shown in Figure 1 The present application provides a kind of massive point cloud real-time visualization method of uniform sampling, comprising the following steps: S1, obtaining original point cloud data; the original point cloud data is subjected to multi-threaded Morton coding, to obtain multiple binary segment files; S2, the multiple binary segment files are merged using k-way merging algorithm to obtain a target file; S3, the point cloud data in the target file is subjected to hierarchical uniform sampling to obtain sampled point cloud data; S4, constructing LOD three-dimensional model: the sampled point cloud data is grouped to the corresponding LOD node according to the corresponding level, and the average point distance of the sampled point cloud data in the corresponding LOD node in the target file is recorded in each LOD node Node bounding box; S5, based on the mapping relationship of focal length and average point distance, each average point distance corresponds to a level, each level and the focal length corresponding to each level are stored in an array; S6, using a view cone and a node bounding box to do containment or intersection test, the LOD node that meets the condition and whose level is less than or equal to the target level is added to the loading list; S7, the background thread asynchronously reads the sampled point cloud data corresponding to the LOD node in the loading list to the CPU memory, and uploads the data to the GPU in real time through double-buffer FBO.

[0020] The embodiment solves the technical problem of slow import of the prior art by multi-threaded Morton coding, multi-threaded parallel block of original point cloud data, maintaining LOD point distance in a fold increase by hierarchical uniform sampling technology, avoiding zooming flicker, accurately limiting memory and bandwidth by focal length and level mapping and double threshold view frustum clipping, and solving the GPU-CPU synchronization blocking problem by double buffering FBO hot switching. The embodiment solves the technical problem that the prior art has not formed a complete technical chain of import, indexing, loading and rendering for ordinary terminals.

[0021] In some embodiments, S1 comprises: S11, setting parameters: setting the size of the point cloud block according to the hardware memory, the bit width of the Morton coding is 21 bits, the preset level is 8 layers, and the circular buffer threshold is 1 million points; S12, according to the number of CPU cores of the computer, allocating the same number of threads as the number of CPU cores and allocating an independent scalable circular buffer area for each thread; S13, loading the original point cloud file into the memory, dividing all the original point cloud data in the original point cloud file into multiple point cloud blocks according to the size of the point cloud block, and distributing the multiple point cloud blocks to the multi-threaded work queue; S14, each thread uses the Morton coding algorithm to divide the current point cloud block into multiple blocks at all levels, obtains multiple Morton point cloud blocks at all levels, each Morton point cloud block has a Morton code, and stores the multiple Morton point cloud blocks in the scalable circular buffer area. When the total number of points corresponding to the multiple Morton point cloud blocks in the scalable circular buffer area reaches the circular buffer threshold, the thread triggers a sequential disk writing, sorts the point cloud in the scalable circular buffer area in ascending order of the Morton code, saves the sorted point cloud to a binary segment file, and obtains multiple binary segment files after all threads process all point cloud blocks.

[0022] The embodiment solves the technical problem of slow import of the prior art by multi-threaded Morton coding and threshold disk writing speedup.

[0023] Preferably, S2 comprises: S21, reading the Morton point cloud block information in each binary segment file one by one, the Morton point cloud block information comprising: level, Morton code, total number of points, and offset of each point in the binary segment file; S22, counting the total number of points corresponding to the Morton point cloud blocks at the same level, and calculating the overall capacity required for the target file according to the total number of points; S23, reserving a continuous storage interval in the target file for each Morton point cloud block at the same level, and recording the starting byte position of each interval; S24, according to the start byte position of each segment, calculate the final write address of each Morton point cloud block in the target file, and calculate the cumulative offset in the segment at the same time; S25, generate a new named binary segment file as the target file; S26, according to the start byte position and the final write address, copy the data of each Morton point cloud block into the target file at the same time; S27, write an index record for each segment, the content of the index record includes: level, Morton code, segment start address, total number of point clouds; Calculate the total sum of all index records, the total number of points in all index records, the maximum and minimum values of the X, Y and Z coordinates of all index records, and get the statistical information; Update the statistical information to the header information of the target file.

[0024] Preferably, S3 comprises: S31, for each bounding box of the Morton point cloud block in the target file and the target level, calculate the voxel edge length and the diagonal length of the point cloud data; S32, construct a first set for recording occupied voxels, and a second set for recording the first reserved point coordinate information in the occupied voxels, both the first set and the second set reserve a capacity according to the input point number divided by 8; S33, list 27 voxel coordinate offsets composed of 1 body voxel and 26 adjacent voxels of the body voxel in advance; S34, for each input point cloud, locate the voxel key where the current point cloud is located; if the voxel has been occupied, skip the current point; if the voxel has not been occupied, check whether there is a reserved point in the 26 adjacent voxels and the distance is less than the voxel diagonal, if there is a reserved point and the distance is less than the voxel diagonal, skip the current point; when the current point is not skipped, write the current point into the second set and record the mapping of the voxel key of the current point to the coordinate, mark the voxel as occupied in the first set, and append the real level and the average point distance; S35, output the second set and clip to the reserved capacity to obtain the sampling point cloud data.

[0025] Preferably, in S31, the calculation formula of the voxel edge length s(L) is: ; Wherein, L is the target level, L0 is the length of one side of the bounding box.

[0026] Preferably, in S6, the condition means that a Morton point cloud block meets the following two conditions at the same time: Bounding box condition: one of the eight vertices of a Morton point cloud block is in the bounding box; Angle condition: meet one Morton point cloud block eight vertexes, one vertex in the view frustum, the vertex in the view frustum, meet the following formula: ; Wherein, a is the vector of the connection line of the camera center to the observation center, b is the vector of the connection line of the camera center to the vertex, and FOV is the field of view angle.

[0027] Preferably, in S5, the mapping relationship between the focal length and the average point distance is: ; Wherein, is the focal length, is the average point distance, is the view angle, is the number of pixels of the screen on the longitudinal tangent in the field of view of the viewpoint.

[0028] Preferably, S7 comprises: a background thread asynchronously reads the sampling point cloud data corresponding to the LOD node in the loading list to the CPU memory; the main thread loads the sampling point cloud data in the CPU memory into the VBO through the API function of OpenGL; the data in the VBO is offline rendered to the FBO through the API function of OpenGL; in the display process, the OpenGL operation default buffer, the data in the FBO is drawn to the default buffer for drawing.

[0029] Finally, it should be noted that: the above embodiments are only the preferred embodiments of the present application for describing the technical solutions of the present application, but not limiting them, and of course, not limiting the patent scope of the present application; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application; that is, any modification or polishing without substantial significance in the main design idea and spirit of the present application, and the technical problems solved are still consistent with the present application, and should be included in the protection scope of the present application; in addition, the technical solutions of the present application are directly or indirectly applied to other related technical fields, which are also included in the patent protection scope of the present application.

Claims

1. A real-time visualization method for uniformly sampled massive point clouds, characterized by: The steps include: S1. Obtain original point cloud data; perform multi-threaded Morton encoding on the original point cloud data to obtain multiple binary segment files; S2. Use the k-way merge algorithm to merge multiple binary segment files to obtain the target file; S3, performing hierarchical uniform sampling on the point cloud data in the target file to obtain sampled point cloud data; S4, grouping the sampled point cloud data into corresponding LOD nodes according to the corresponding levels, and recording in each LOD node the average point distance and node bounding box of the sampled point cloud data in the corresponding LOD node in the target file; S5. Based on the mapping relationship between focal length and average point distance, each average point distance corresponds to a level, and each level and a focal length corresponding to each level are stored in an array; S6. Perform inclusion or intersection tests with the view frustum and the node bounding box, and add LOD nodes that meet the conditions and whose levels are less than or equal to the target level to the load list; S7. The background thread asynchronously reads the sampling point cloud data corresponding to the LOD node in the load list into the CPU memory, and uploads the data to the GPU in real time through the double-buffered FBO.

2. The real-time visualization method for uniformly sampled massive point clouds according to claim 1, characterized in that: S1 includes: S11. Set parameters: Set the point cloud block size according to the hardware memory, the bit width of Morton encoding is 21 bits, the preset layer is 8 layers, and the circular buffer threshold is 1 million points; S12. Allocate threads equal to the number of CPU cores in the computer and allocate an independent expandable circular buffer to each thread; S13, loading the original point cloud file into the memory, dividing all the original point cloud data in the original point cloud file into multiple point cloud blocks according to the point cloud block size; and distributing the multiple point cloud blocks to the multi-threaded work queue; S14. Each thread uses the Morton coding algorithm to divide the current point cloud block into blocks at all levels to obtain multiple Morton point cloud blocks at all levels, each Morton point cloud block has a Morton code; the multiple Morton point cloud blocks are stored in an expandable circular buffer. When the total number of point clouds corresponding to the multiple Morton point cloud blocks in the expandable circular buffer reaches the circular buffer threshold, the thread triggers a sequential write to the disk, sorts the point clouds in the expandable circular buffer in ascending order of the Morton codes, and saves the sorted point clouds to binary segment files. When all threads have processed all point cloud blocks, multiple binary segment files are obtained.

3. The real-time visualization method for uniformly sampled massive point clouds according to claim 2, characterized in that: S2 includes: S21, reading the Morton point cloud block information in each binary segment file one by one, the Morton point cloud block information including: level, Morton code, total number of point clouds, and offset of each point in the binary segment file; S22, counting the total number of point clouds corresponding to the Morton point cloud blocks at the same level, and calculating the overall capacity required for the target file based on the total number of point clouds; S23, reserving a continuous storage interval in the target file for each Morton point cloud block at the same level, and recording the starting byte position of each segment; S24. Calculate the final write address of each Morton point cloud block in the target file based on the starting byte position of each segment, and simultaneously calculate the cumulative offset within the segment; S25. Generate a newly named binary segment file as the target file; S26, copying the data of each Morton point cloud block to the target file simultaneously according to the starting byte position and the final write address; S27. Write an index record for each segment. The content of the index record includes: level, Morton code, segment starting address, and total number of point clouds; count the sum of all index records, the total number of points in all index records, and the maximum and minimum values ​​of the X, Y, and Z coordinates of the total point clouds in all index records to obtain statistical information; update the statistical information to the header information of the target file.

4. The real-time visualization method for uniformly sampled massive point clouds according to claim 3, characterized in that S3 include: S31, for each Morton point cloud block in the target file, the bounding box and the target level are calculated, and the voxel side length and diagonal length of the point cloud data are calculated; S32: construct a first set for recording occupied voxels and a second set for recording coordinate information of the first reserved point in the occupied voxels, wherein the first set and the second set both reserve capacity according to the number of input points divided by 8; S33, pre-listing 27 voxel coordinate offsets consisting of one main voxel and its 26 neighboring voxels; S34. For each input point cloud, perform the following operations: locate the voxel key where the current point cloud is located; if the voxel is occupied, skip the current point; if the voxel is not occupied, check whether there is a reserved point in any of the 26 neighboring voxels and the distance is less than the voxel diagonal; if so, skip the current point; if the current point is not skipped, write the current point into the second set and record the mapping from the voxel key to the coordinates of the current point, mark the voxel as occupied in the first set, and append the real level and average point distance; S35: Output the second set and trim it to the reserved capacity to obtain sampling point cloud data.

5. The real-time visualization method for uniformly sampled massive point clouds according to claim 4, characterized in that: In S31, the calculation formula of voxel side length s(L) is: ； Where L is the target level and L0 is the length of one side of the bounding box.

6. The real-time visualization method for uniformly sampled massive point clouds according to claim 4, characterized in that: In S6, meeting the conditions means that a Morton point cloud block meets the following two conditions at the same time: Bounding box condition: a Morton point cloud block has eight vertices and one vertex is inside the bounding box; Angle condition: A Morton point cloud block has eight vertices, one of which is within the viewing cone. The vertices within the viewing cone satisfy the following formula: ； Where a is the vector of the line connecting the camera center to the observation center, b is the vector of the line connecting the camera center to the vertex, and FOV is the field of view angle.

7. The real-time visualization method for uniformly sampled massive point clouds according to claim 1, characterized in that: In S5, the mapping relationship between focal length and average point distance is: ； in, is the focal length, is the average point distance, For perspective, It is the number of pixels on the vertical tangent of the screen within the viewpoint field of view.

8. The real-time visualization method for uniformly sampled massive point clouds according to claim 1, characterized in that: S7 includes: the background thread asynchronously reads the sampling point cloud data corresponding to the LOD node in the loading list into the CPU memory; the main thread loads the sampling point cloud data in the CPU memory into the VBO through the OpenGL API function; the data in the VBO is drawn offline into the FBO through the OpenGL API function; during the display process, OpenGL operates the default buffer and draws the data in the FBO into the default buffer for drawing.

Images