Method for rendering large-scale point cloud at web end

Abstract

The invention discloses a method for rendering a large-scale point cloud at a web end, and belongs to the technical field of large-scale point cloud rendering, and the method comprises the steps: S1, carrying out the data preprocessing of point cloud data, and carrying out the octree division of the whole point cloud; s2, on the basis of octree division, performing Lod hierarchical data division on leaf nodes of the whole octree; s3, data which needs to be rendered currently are screened out from the octree through view cone cutting; s4, high-level data and low-level data are separated from the screened data needing to be rendered, the low-level data are used for real-time rendering, and the high-level data are used for non-real-time rendering; and S5, combining the real-time rendering result and the non-real-time rendering result, and outputting all rendering results. According to the method, a similar double-buffer structure is used, and large-scale point clouds can be rendered in a real-time and non-real-time mixed mode at a web end.

Description

Technical Field

[0001] The present invention belongs to the technical field of large-scale point cloud rendering, and in particular to a method for rendering large-scale point clouds on a web terminal. Background Art

[0002] Large-scale point clouds are a collection of data consisting of a large number of points in three-dimensional space. These points usually contain three-dimensional coordinate information (X, Y, Z), and may also have other attribute information such as color, intensity, normal vector, etc. Large-scale point clouds can be obtained through laser scanning, photogrammetry and other technologies, and are widely used in geographic information systems, three-dimensional modeling, virtual reality, autonomous driving and other fields. Large-scale point cloud rendering refers to the process of presenting a point cloud containing a large amount of three-dimensional point data on the screen in the form of an image in computer graphics.

[0003] Due to the huge number of points, large-scale point clouds face great challenges in storage, transmission, and rendering. For example, efficient data structures and algorithms are needed to manage point cloud data to reduce storage requirements and increase processing speed; when rendering, multiple aspects such as data processing, rendering algorithms, and performance optimization need to be considered comprehensively to achieve efficient and high-quality point cloud visualization.

[0004] In the prior art, the main methods for rendering large-scale point clouds are: traditional single-buffer rendering, which only uses a single buffer for rendering. When processing large-scale point clouds, problems such as freezing and unstable frame rates may occur, making it difficult to meet real-time requirements; simple multi-threaded rendering, which uses multiple threads to process different parts of point cloud data for rendering, but may have difficulties in coordination and synchronization, and lacks support for real-time and non-real-time mixed rendering.

[0005] The above-mentioned existing rendering methods can only perform single real-time rendering or non-real-time rendering, and cannot simultaneously implement real-time and non-real-time mixed rendering on the web side to meet the needs of different scenarios.

[0006] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a method for rendering large-scale point clouds on a web side, which uses a double-buffer structure and can render large-scale point clouds in a real-time and non-real-time mixed manner on the web side.

[0008] To achieve the above object, the present invention provides a method for rendering a large-scale point cloud on a web terminal, comprising the following steps:

[0009] S1: Preprocess the point cloud data and divide the whole point cloud into octrees;

[0010] S2: Based on the octree division, the leaf nodes of the entire octree are divided into Lod level data;

[0011] S3: Through frustum pruning, filter out the data that needs to be rendered from the octree;

[0012] S4: Separate high-level and low-level data from the filtered data to be rendered, the low-level data is used for real-time rendering, and the high-level data is used for non-real-time rendering; the low-level data is selected according to the distance of the point cloud data from the camera, and the high-level data is all the input point cloud data;

[0013] S5: Merge the real-time rendering and non-real-time rendering results and output all rendering results.

[0014] In one embodiment of the present invention, step S1 includes the following sub-steps:

[0015] S101: Specifies the maximum depth of the octree and the maximum number of points that each leaf node can hold;

[0016] S102: Calculate the coordinate ranges of all points in the x-axis, y-axis, and z-axis directions, and construct a minimum outer bounding box;

[0017] S103: Divide the outer bounding box into eight sub-bounding boxes of the same size;

[0018] S104: reallocate each point in the point cloud data to a corresponding sub-bounding box according to its coordinate position;

[0019] S105: Confirm whether each sub-bounding box reaches the maximum depth of the octree specified in step S101 and the maximum number of points that each leaf node can hold; if so, stop segmenting; if not, repeat steps S103 and S104 for the sub-bounding boxes until the standard specified in step S101 is met.

[0020] In one embodiment of the present invention, step S2 includes the following sub-steps:

[0021] S201: confirming the sampling range [i, j] of the point cloud, where i is the starting subscript of the sampled point cloud in the overall point cloud, and j is the ending subscript of the sampled point cloud in the overall point cloud;

[0022] S202: Randomly sample a point cloud in the determined sampling range [i, j];

[0023] S203: exchanging the sampled point cloud with the point represented by the starting subscript;

[0024] S204: Reduce the point cloud sampling area and repeat steps S202 and S203 until the sampling reaches a predetermined level.

[0025] In one embodiment of the present invention, the predetermined level is 12 levels.

[0026] In one embodiment of the present invention, step S3 includes the following sub-steps:

[0027] S301: taking the bounding box of the octree root node as the initial operation;

[0028] S302: Determine whether the bounding box is within the range of the viewing cone. If not, discard all child nodes represented by the bounding box. If so, traverse all child nodes of the bounding box and then update the bounding box.

[0029] S303: Filter out leaf nodes with real point cloud data from the child nodes to be rendered obtained in step S302 as data basis for actual rendering.

[0030] In one embodiment of the present invention, the bounding box of the octree root node is a bounding box enclosed by the maximum and minimum values ​​of the x-axis, the maximum and minimum values ​​of the y-axis, and the maximum and minimum values ​​of the z-axis in the entire point cloud coordinates.

[0031] In one embodiment of the present invention, the operation of updating the bounding box uses the bounding box of the child node as the input of the program.

[0032] In one embodiment of the present invention, step S4 includes the following sub-steps:

[0033] S401: for the node with real point cloud data in the octree currently required to be rendered obtained in step S3, the Lod level of the low-level data is determined from the different Lod level data pre-generated based on step S2 according to the camera position, the frustum range and the maximum number of point clouds currently allowed to be rendered;

[0034] S402: Rendering low-level data in real time and rendering high-level data in non-real time.

[0035] In one embodiment of the present invention, the low-level data range is data with a Lod level of [6, 12].

[0036] In one embodiment of the present invention, two buffers of real-time rendering and non-real-time rendering are merged in a linear blending manner to generate a final rendering result.

[0037] Compared with the prior art, a method for rendering large-scale point clouds on the web according to the present invention, firstly, divides the point cloud data by octree and performs Lod level division, thereby realizing efficient data organization and management, and can quickly locate the required data and automatically select the appropriate detail level according to different situations, thereby greatly improving the rendering efficiency; secondly, the frustum clipping accurately screens the current data to be rendered, avoids useless processing of invisible areas, saves computing resources, and ensures the performance when processing large-scale point clouds; thirdly, separates high-level and low-level data for real-time and non-real-time mixed rendering, and the low-level data is used for real-time rendering to ensure the real-time nature of the interaction, so that the user's operation can get immediate feedback, and the high-level data is used for non-real-time rendering to provide high-quality pictures when needed, thereby enhancing the visual effect; finally, the two rendering result outputs are merged, which not only ensures the smoothness of the operation but also presents high-quality images, and can automatically adjust the rendering strategy in different scenes to bring the best visual experience to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of a method for rendering a large-scale point cloud on a web side according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0040] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0041] like Figure 1 As shown, a method for rendering a large-scale point cloud on a web terminal according to a preferred embodiment of the present invention includes the following steps:

[0042] S1: Preprocess the point cloud data and divide the whole point cloud into octrees.

[0043] S2: Based on the octree division, the leaf nodes of the entire octree are divided into Lod (Level of Detail) level data.

[0044] S3: Through frustum pruning, the data that needs to be rendered is filtered out from the octree.

[0045] S4: Separate high-level and low-level data from the filtered data to be rendered, the low-level data is used for real-time rendering, and the high-level data is used for non-real-time rendering; the low-level data is selected according to the distance of the point cloud data from the camera, and the high-level data is all the input point cloud data;

[0046] S5: Merge the real-time rendering and non-real-time rendering results and output all rendering results.

[0047] Furthermore, in step S1, before starting rendering, the original large-scale point cloud data is first preprocessed. Octree partitioning is a method of recursively subdividing a three-dimensional space into eight sub-areas. By placing the point cloud data into an octree structure, the data can be organized and managed more efficiently. The advantage of this is that points in a specific area can be quickly located, and in subsequent processing steps, part of the data can be selectively processed as needed to improve efficiency.

[0048] Specifically, step S1 includes the following sub-steps:

[0049] S101: Specify the maximum depth of the octree and the maximum number of points that each leaf node can hold.

[0050] The maximum number of points that each leaf node can hold is related to the performance of the computer's graphics card, and is generally an empirical constant of about 100,000. Such a threshold value is taken to prevent too many DrawCalls from being generated during the final rendering (DrawCall is a command sent by the CPU to the GPU during the GPU rendering process to render a grid), which in turn affects performance. The maximum depth of the octree is generally 6.

[0051] Choosing the above two parameters can achieve a balance between dynamic tree building and real-time rendering.

[0052] This step is to determine the parameters of the octree structure. The octree is a tree-like data structure used to organize three-dimensional spatial data. By specifying the maximum depth, the subdivision level of the octree can be controlled to avoid excessive subdivision leading to high computational complexity. At the same time, determining the maximum number of points that each leaf node can accommodate can ensure the reasonable distribution of data in the octree, which is convenient for subsequent processing and rendering.

[0053] S102: Calculate the coordinate ranges of all points in the x-axis, y-axis, and z-axis directions, and construct a minimum outer bounding box.

[0054] First, the minimum and maximum coordinate values ​​of all points in the point cloud data in the x, y, and z coordinate directions are calculated to determine a minimum outer bounding box that can completely surround all points. This outer bounding box provides a basic framework for the subsequent division of point cloud data into different nodes of the octree.

[0055] S103: Divide the outer bounding box into eight sub-bounding boxes of the same size.

[0056] S104: reallocate each point in the point cloud data to a corresponding sub-bounding box according to its coordinate position.

[0057] In this way, the specific position of each point in the octree can be determined, providing accurate information for subsequent processing and rendering.

[0058] S105: Confirm whether each sub-bounding box reaches the maximum depth of the octree specified in step S101 and the maximum number of points that each leaf node can hold; if reached, stop segmentation; if not, repeat steps S103 and S104 for the sub-bounding boxes until the threshold standard is reached.

[0059] Check whether each sub-bounding box meets the preset maximum depth of the octree and the maximum number of points that each leaf node can hold. If the sub-bounding box meets the above threshold standard, it means that the sub-bounding box can be used as a leaf node of the octree and no further subdivision is performed. If it does not meet the standard, it is necessary to continue to segment the sub-bounding box and repeat steps S103 and S104 until all sub-bounding boxes meet the threshold standard.

[0060] This step can ensure that the structure of the octree can adapt to the distribution of point cloud data and improve the efficiency of subsequent processing and rendering.

[0061] Furthermore, step S2 is to dynamically adjust the display details of the point cloud according to different observation distances and scene requirements. Therefore, step S2 performs Lod division on the leaf nodes of the octree, which can divide the point cloud data into different levels of detail. The point cloud data held at each level, starting from the overall point cloud, tends to decrease by two times. When the observation distance is far, data with a lower level of detail can be used for rendering to reduce the amount of calculation; when the observation distance is close, data with a higher level of detail can be used to provide richer details.

[0062] Specifically, step S2 includes the following sub-steps:

[0063] S201: confirm the sampling range [i, j] of the point cloud, where i is the starting subscript of the sampled point cloud in the overall point cloud, and j is the ending subscript of the sampled point cloud in the overall point cloud.

[0064] In the initial state, the sampling range of the point cloud is the total number of point clouds, that is, in the initial state, i is always 0, and j is the number of points in the point cloud.

[0065] S202: Randomly sample a point cloud in the determined sampling range [i, j].

[0066] S203: Exchange the sampled point cloud with the point represented by the starting subscript.

[0067] The exchange operation in this step is to exchange the sampled point cloud with the points represented by the starting range, the purpose of which is to reduce the spatial complexity in the point cloud processing process.

[0068] It can be understood like this, for example, a point is randomly sampled in the range [i, j], and assuming that the subscript of this point is k, then k is exchanged with i (the exchange here refers to the exchange of the actual points represented by i and k, not the exchange of the values ​​of i and k themselves). Then, in the entire array, it can be logically divided into two parts, the part before i is the sampled part, and the part after i is the unsampled part. In this way, the sampling can be completed without opening up additional memory space.

[0069] The range after the exchange is the reduced point cloud sampling area. From the program's perspective, it can be understood that i increases by one each time a sample is taken.

[0070] S204: Reduce the point cloud sampling area, and repeat steps S202 and S203 until the predetermined level is sampled. In step S204, the predetermined level is preferably 12 levels.

[0071] The selection of level 12 is an empirical parameter obtained through multiple tests. The logic of level sampling is: two-fold decreasing, that is, the number of point clouds in the previous level is twice the number of point clouds in the next level.

[0072] In step S204, after each execution of the previous steps, the sampling area is reduced so that the range of the next sampling becomes smaller. Then the random sampling step is repeated, and this process is continued until a predetermined level is reached, and the sampling and processing process is stopped.

[0073] Furthermore, the frustum clipping in step S3 is a clipping technique based on the observer's perspective. By determining the scope of the observer's frustum, the point cloud data outside the frustum can be quickly excluded, and only the points within the frustum are rendered. This can greatly reduce the amount of data to be processed and improve rendering efficiency. Especially for large-scale point cloud data, frustum clipping can significantly reduce the computational burden.

[0074] Specifically, step S3 includes the following sub-steps:

[0075] S301: Take the bounding box of the octree root node as the initial operation.

[0076] The bounding box of the octree root node is a bounding box surrounded by the maximum and minimum values ​​of the x-axis, the maximum and minimum values ​​of the y-axis, and the maximum and minimum values ​​of the z-axis in the entire point cloud coordinates.

[0077] The reason why the bounding box of the root node of the octree is selected as the initial value of the calculation in step S301 is that the bounding box of the root node of the octree can contain the point cloud of the entire tree. Therefore, once it is determined that the bounding box is not within the camera's frustum, it means that all the point clouds in the entire octree are invisible to the camera. Then there is no need for this part of the data to enter the GPU for rendering. Starting from the root node, the point clouds that do not need to be rendered can be quickly cut off to reduce the rendering pressure.

[0078] S302: Determine whether the bounding box is within the range of the viewing cone. If not, discard all child nodes represented by the bounding box. If so, traverse all child nodes of the bounding box and then update the bounding box.

[0079] The operation of updating the bounding box uses the bounding box of the child node as the input of the program to determine visibility.

[0080] Determine whether the bounding box of the root node is within the viewing cone. The viewing cone is a conical area determined from the observer's perspective, and only objects within this area need to be rendered. If the bounding box is not within the viewing cone, then all child nodes represented by the bounding box can be directly discarded, because they are definitely not within the viewing cone and do not need to be rendered. If the bounding box is within the viewing cone, it is necessary to traverse all child nodes of the bounding box. During the traversal process, the bounding box is continuously updated, that is, the bounding box of the child node currently being processed is used as input, and visibility judgment is performed again. In this way, the octree structure is gradually deepened to determine which parts of the point cloud data need to be rendered.

[0081] S303: Filter out leaf nodes with real point cloud data from the child nodes to be rendered obtained in step S302 as data basis for actual rendering.

[0082] After processing in step S302, a series of child nodes that may need to be rendered are obtained. However, these child nodes may not be the final leaf nodes, and may contain more child nodes. Therefore, it is necessary to further filter out the leaf nodes with real point cloud data from these child nodes. These leaf nodes are the basis for actual rendering because they directly correspond to specific point cloud data points.

[0083] Furthermore, in step S4, data of different detail levels are further separated from the data to be rendered after frustum cropping. Low-level, low-volume data usually has fewer points and simpler geometric shapes, and is suitable for real-time rendering to ensure that basic point cloud images can be quickly responded to and displayed during user interaction. High-level, high-volume data contains more details and more complex geometric information, and is used for non-real-time rendering. When the user does not need real-time interaction, high-quality rendering can be performed to provide more exquisite visual effects.

[0084] Specifically, step S4 includes:

[0085] S401: For the nodes with real point cloud data in the octree currently required to be rendered obtained in step S3, factors such as the camera position, the range of the viewing cone, and the maximum number of point clouds currently allowed to be rendered should be comprehensively considered, and the most suitable Lod level for rendering should be determined from the different Lod level data pre-generated based on the previous step S2.

[0086] The position of the camera is operated by the user and is not fixed. Therefore, the viewing cone range of the camera can be controlled in the present invention.

[0087] The longest axis of the outer bounding box is selected as the actual length of the X-axis of the far plane of the frustum. The actual length of the Y-axis of the far plane of the frustum is calculated according to the screen aspect ratio of the current viewport. After calculating the actual lengths of the X-axis and Y-axis of the original plane of the frustum, an optimal orthogonal frustum parameter can be calculated based on these two parameters. In actual operation, the camera near plane is generally set to 0.1 and the far plane is set to 1000. The frustum calculation logic is consistent with that of the standard orthogonal camera.

[0088] In the present invention, the corresponding lod level is determined by the point cloud distance and the camera distance.

[0089] S402: Render low-level data in real time, because the amount of low-level data is small and real-time performance can be guaranteed; at the same time, render high-level data in non-real time to generate high-quality images when real-time interaction is not required.

[0090] Through the above steps, two different buffers, real-time buffer and non-real-time buffer, are generated.

[0091] In the above steps, the high-level data is all the input point cloud data, representing the highest level of detail. The low-level data is selected based on the distance of the point cloud data from the camera. When the distance is far from the camera, the level data with fewer point clouds is selected, which can reduce the amount of data rendered and improve efficiency.

[0092] Taking the LOD level as an example, 0 is the highest level and 12 is the lowest level. The data range of the low level is the data of the LOD level [6,12]. The higher the level, the closer the distance to the camera is, and the more point clouds it has. Generally, the level of 10 meters from the camera is level 6, the level of 30 meters is level 7, the level of 50 meters is level 8, and so on, up to level 12.

[0093] Furthermore, in step S5, the two buffers of real-time rendering and non-real-time rendering are merged in a linear blending manner to generate a final rendering result.

[0094] By linear blending, the image information in the two buffers can be combined according to a certain ratio or rule. For example, the pixel color, depth and other information in the two buffers can be weighted averaged according to a specific weight value, or the real-time rendering result can be gradually transitioned to the non-real-time rendering result according to certain conditions.

[0095] The final rendering result has both the interactivity and fast response capability brought by real-time rendering and the high-quality image details of non-real-time rendering, presenting users with a richer and more realistic large-scale point cloud rendering picture on the web.

[0096] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A method for rendering large-scale point clouds on a web terminal, characterized in that: The following steps are involved: S1: Preprocess the point cloud data and divide the whole point cloud into octrees; S2: Based on the octree division, the leaf nodes of the entire octree are divided into Lod level data; S3: Through frustum pruning, filter out the data that needs to be rendered from the octree; S4: Separate high-level and low-level data from the filtered data to be rendered, the low-level data is used for real-time rendering, and the high-level data is used for non-real-time rendering; the low-level data is selected according to the distance of the point cloud data from the camera, and the high-level data is all the input point cloud data; S5: Merge the real-time rendering and non-real-time rendering results and output all rendering results.

2. The method for rendering a large-scale point cloud on a web terminal according to claim 1, characterized in that: Step S1 includes the following sub-steps: S101: Specifies the maximum depth of the octree and the maximum number of points that each leaf node can hold; S102: Calculate the coordinate ranges of all points in the x-axis, y-axis, and z-axis directions, and construct a minimum outer bounding box; S103: Divide the outer bounding box into eight sub-bounding boxes of the same size; S104: reallocate each point in the point cloud data to a corresponding sub-bounding box according to its coordinate position; S105: Confirm whether each sub-bounding box reaches the maximum depth of the octree specified in step S101 and the maximum number of points that each leaf node can hold; if so, stop segmenting; if not, repeat steps S103 and S104 for the sub-bounding boxes until the standard specified in step S101 is met.

3. The method for rendering a large-scale point cloud on a web terminal as claimed in claim 2, characterized in that: Step S2 includes the following sub-steps: S201: confirming the sampling range [i, j] of the point cloud, where i is the starting subscript of the sampled point cloud in the overall point cloud, and j is the ending subscript of the sampled point cloud in the overall point cloud; S202: Randomly sample a point cloud in the determined sampling range [i, j]; S203: exchanging the sampled point cloud with the point represented by the starting subscript; S204: Reduce the point cloud sampling area and repeat steps S202 and S203 until the sampling reaches a predetermined level.

4. The method for rendering a large-scale point cloud on a web terminal as claimed in claim 3, characterized in that: The predetermined level is 12 levels.

5. The method for rendering a large-scale point cloud on a web terminal as claimed in claim 3, characterized in that: Step S3 includes the following sub-steps: S301: taking the bounding box of the octree root node as the initial operation; S302: Determine whether the bounding box is within the range of the viewing cone. If not, discard all child nodes represented by the bounding box. If so, traverse all child nodes of the bounding box and then update the bounding box. S303: Filter out leaf nodes with real point cloud data from the child nodes to be rendered obtained in step S302 as data basis for actual rendering.

6. The method for rendering a large-scale point cloud on a web terminal according to claim 5, characterized in that: The bounding box of the octree root node is a bounding box enclosed by the maximum and minimum values ​​of the x-axis, the maximum and minimum values ​​of the y-axis, and the maximum and minimum values ​​of the z-axis in the entire point cloud coordinates.

7. The method for rendering a large-scale point cloud on a web terminal according to claim 5, characterized in that: The operation of updating the bounding box takes the bounding box of the child node as the input of the program.

8. The method for rendering a large-scale point cloud on a web terminal as claimed in claim 5, characterized in that: Step S4 includes the following sub-steps: S401: for the node with real point cloud data in the octree currently required to be rendered obtained in step S3, the Lod level of the low-level data is determined from the different Lod level data pre-generated based on step S2 according to the camera position, the frustum range and the maximum number of point clouds currently allowed to be rendered; S402: Rendering low-level data in real time and rendering high-level data in non-real time.

9. The method for rendering a large-scale point cloud on a web terminal according to claim 8, characterized in that: The low-level data range is the data with Lod level [6,12].

10. The method for rendering a large-scale point cloud on a web terminal according to claim 8, characterized in that: The two buffers of real-time rendering and non-real-time rendering are merged in a linear blending manner to generate the final rendering result.

Images