A point cloud data processing method, device, equipment and medium

Abstract

The application discloses a point cloud data processing method, device and equipment and a medium, and relates to the technical field of data processing. The method comprises the following steps: obtaining user point cloud data to be processed, and creating an octree for the point cloud data to be processed; determining an initial observation view angle of the point cloud data to be processed according to the octree, and generating an initial rendering view of the point cloud data to be processed according to the initial observation view angle; generating a current rendering view of the user, determining a leaf node to be loaded according to the current rendering view; determining a leaf node to be rendered, and determining a rendering priority; asynchronously acquiring data points and performing concurrent decoding, creating a geometric object, rendering the geometric object according to the rendering priority, and obtaining visualized point cloud data; generating a directional bounding box of the user according to a point cloud segmentation request, marking a segmentation object, and processing the visualized point cloud data according to the segmentation object. The application provides a comprehensive solution of point cloud data with asynchronous loading, real-time rendering and accurate segmentation.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method, apparatus, device, and medium for processing point cloud data. Background Technology

[0002] With the rapid development of technologies such as LiDAR and 3D scanning, point cloud data is widely used in fields such as autonomous driving, building surveying, and geographic information systems. Point cloud data typically reaches tens of millions or even hundreds of millions of points, placing extremely high performance demands on data processing and rendering.

[0003] Currently, when processing point cloud data, the entire point cloud data needs to be loaded into memory. This makes it impossible to process large-scale point cloud data that exceeds the system memory limit. Synchronously loading point cloud data from memory results in long waiting times when processing large-scale data, leading to high loading latency and noticeable delays in user interaction response. Simultaneously rendering a large amount of point cloud data also causes a significant drop in frame rate, resulting in a clear rendering efficiency bottleneck, poor user experience, and an inability to meet the needs of fine-grained processing.

[0004] Therefore, how to provide a more efficient, real-time rendering solution for large-scale point cloud data is an important issue that the industry urgently needs to address. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, apparatus, device and medium for processing point cloud data, thereby solving the problems of high loading latency, slow interactive response and rendering performance bottlenecks in the processing of large-scale point cloud data.

[0006] According to a first aspect, embodiments of the present invention provide a method for processing point cloud data, the method comprising:

[0007] Obtain the user's point cloud data to be processed, and create an octree for the point cloud data to be processed;

[0008] Based on the octree, determine the initial viewing angle of the point cloud data to be processed, and generate the initial rendering view of the point cloud data to be processed based on the initial viewing angle.

[0009] Obtain the user's view interaction commands, generate the user's current rendered view, and determine the leaf nodes to be loaded based on the current rendered view;

[0010] Identify the leaf nodes to be rendered among the leaf nodes to be loaded, and determine the rendering priority of the leaf nodes to be rendered.

[0011] Depending on whether the data points in the leaf node to be rendered are cached, the data points are asynchronously acquired and concurrently decoded to create a geometry object. The geometry object is then rendered according to the rendering priority to obtain the visualized point cloud data. The asynchronous reading includes local Blob reading and / or data chunk loading based on HTTP Range chunk requests.

[0012] Obtain the user's point cloud segmentation request, determine the polygonal region drawn by the user in the current rendering view based on the point cloud segmentation request, generate the user's oriented bounding box based on the polygonal region, mark the segmentation object based on the oriented bounding box, and process the visualized point cloud data based on the segmentation object.

[0013] In conjunction with the first aspect, in the first embodiment of the first aspect, the step of acquiring user point cloud data to be processed and creating an octree for the point cloud data to be processed specifically includes:

[0014] Acquire the point cloud data to be processed and extract the coordinate information of each data point in the point cloud data to be processed;

[0015] Traverse all coordinate information to determine the point cloud boundary of the point cloud data to be processed, and determine the virtual bounding box of the point cloud data to be processed in three-dimensional space based on the point cloud boundary.

[0016] Use the virtual bounding box as the root node of the octree, and place the data points into the root node in sequence according to their coordinate information.

[0017] The root node is divided into eight equal child nodes, and the data points in the root node are distributed to each child node according to the coordinate information of the data points.

[0018] Recursively partition the space for each child node until the preset termination condition is reached, generate the node structure and obtain the leaf node, and obtain the octree containing the point cloud data to be processed based on the root node, child nodes and leaf nodes.

[0019] Create proxy nodes for the nodes of the octree, determine the node information of the nodes, and associate the node information with the proxy nodes.

[0020] In conjunction with the first aspect, in the second embodiment of the first aspect, the step of determining the initial viewing angle of the point cloud data to be processed based on the octree, and generating an initial rendering view of the point cloud data to be processed based on the initial viewing angle, specifically includes:

[0021] Based on the center point and size of the virtual bounding box, and by analyzing the spatial characteristics of the point cloud data to be processed, the main extension direction and the optimal observation direction of the point cloud data to be processed are obtained.

[0022] Use the center point as the observation target point of the virtual camera, and adjust the position of the observation target point using the main extension direction;

[0023] The basic observation distance of the virtual camera is determined based on the size. The camera position of the virtual camera is generated based on the basic observation distance. The upward direction vector of the virtual camera is generated based on the camera position and the optimal observation direction. The basic observation distance is the distance within the field of view of all point cloud data to be processed.

[0024] Based on the optimal viewing direction and the upward direction vector, generate the projection matrix and view matrix of the virtual camera;

[0025] The initial viewing angle is determined based on the projection matrix and the view matrix, and the initial rendered view is generated based on the initial viewing angle.

[0026] In conjunction with the second embodiment of the first aspect, in the third embodiment of the first aspect, the step of obtaining the user's view interaction instructions, generating the user's current rendered view, and determining the leaf nodes to be loaded based on the current rendered view specifically includes:

[0027] Obtain the user's view interaction commands, parse the view interaction commands, and determine the user's view interaction operations;

[0028] Adjust the camera parameters of the virtual camera based on the view interaction operation, adjust the initial rendering view based on the adjusted camera parameters, and generate the user's current rendering view;

[0029] The camera frustum of the virtual camera is determined based on the current rendered view. Starting from the root node of the octree, a recursive traversal is performed to determine the spatial relationship between the sub-boundary boxes corresponding to the leaf nodes and the camera frustum. Based on the spatial relationship, the leaf nodes to be loaded are determined. The sub-boundary boxes are jointly determined by the virtual bounding box and the node information of the leaf nodes, and the sub-boundary boxes are part of the virtual bounding box.

[0030] In conjunction with the first aspect, in the fourth embodiment of the first aspect, the step of asynchronously acquiring data points and concurrently decoding them based on whether the data points in the leaf node to be rendered are cached, creating a geometry object, rendering the geometry object according to the rendering priority, and obtaining visualized point cloud data specifically includes:

[0031] Determine whether the data points of the leaf nodes to be rendered are cached in memory based on the node information;

[0032] If a memory cache hit is determined, the cached data is retrieved, and a geometry object is created based on the cached data.

[0033] If a memory cache miss is detected, determine whether the data points of the leaf node to be rendered should be persistently cached based on the node information.

[0034] If a persistent cache hit is determined, the hit persistent cache data is retrieved, and a geometry object is created based on the persistent cache data;

[0035] If the persistent cache is not hit, the data to be loaded is read asynchronously based on the node information of the leaf node to be rendered, all binary slices are obtained, and the binary slices are decoded concurrently to obtain the geometry object.

[0036] Extend the minimum and maximum coordinates of the child bounding boxes of the leaf nodes to be rendered into the visible boundary;

[0037] The geometric objects are rendered according to the rendering priority to obtain visualized point cloud data.

[0038] In conjunction with the first aspect, in the fifth embodiment of the first aspect, determining the leaf nodes to be rendered among the leaf nodes to be loaded, and determining the rendering priority of the leaf nodes to be rendered, specifically includes:

[0039] Initialize the visible boundary by setting the minimum value of the visible boundary to positive infinity and the maximum value to negative infinity;

[0040] Determine the center point of the child bounding box of all leaf nodes to be loaded, and determine the Euclidean distance between each center point and the camera position of the virtual camera. Then, determine the rendering priority of each leaf node based on the Euclidean distance. The rendering priority has a certain mapping relationship with the Euclidean distance.

[0041] Determine the child bounding boxes of all leaf nodes to be loaded, and determine the bounding sphere of each leaf node to be loaded and the diameter of the bounding sphere;

[0042] Determine the camera field of view and screen height of the virtual camera. Based on the diameter of the bounding sphere, the camera field of view, and the screen height, determine the projected pixel size of each leaf node on the display screen. If the projected pixel size exceeds the preset minimum pixel threshold, the corresponding leaf node is used as the leaf node to be rendered.

[0043] In conjunction with the first aspect, in the sixth embodiment of the first aspect, the steps of obtaining the user's point cloud segmentation request, generating the user's oriented bounding box based on the point cloud segmentation request, marking the segmentation object based on the oriented bounding box, and processing the visualized point cloud data based on the segmentation object specifically include:

[0044] Obtain the point cloud segmentation request, determine the polygonal region drawn by the user in the current rendering view based on the point cloud segmentation request, and set a session-unique identifier for the point cloud segmentation request;

[0045] Determine the polygon vertices of the polygon region, and construct a minimum oriented bounding box that completely contains the polygon region based on the polygon vertices;

[0046] Determine the spatial relationship between the child bounding box and the oriented bounding box, and determine the leaf nodes to be segmented based on the spatial relationship;

[0047] Depending on whether the data points in the leaf nodes to be segmented are cached, asynchronously retrieve the data points and perform concurrent decoding to create geometry objects;

[0048] Based on preset filtering conditions, the data points in the geometric object are filtered point by point;

[0049] The filtered geometry objects are allocated according to the request order indicated by the session unique identifier, resulting in segmented objects;

[0050] Determine the rendering priority of the segmented objects, render the segmented objects according to the rendering priority, and obtain visualized point cloud data.

[0051] According to a second aspect, embodiments of the present invention also provide a point cloud data processing apparatus, the apparatus comprising:

[0052] The point cloud organization module is used to acquire user point cloud data to be processed and create an octree for the point cloud data to be processed.

[0053] The initial generation module is used to determine the initial viewing angle of the point cloud data to be processed based on the octree, and to generate the initial rendering view of the point cloud data to be processed based on the initial viewing angle.

[0054] The perspective adjustment module is used to obtain the user's view interaction commands, generate the user's current rendered view, and determine the leaf nodes to be loaded based on the current rendered view;

[0055] The rendering preparation module is used to determine the leaf nodes to be rendered among the leaf nodes to be loaded, and to determine the rendering priority of the leaf nodes to be rendered.

[0056] The asynchronous loading module is used to asynchronously acquire data points and perform concurrent decoding based on whether the data points in the leaf node to be rendered are cached, create geometry objects, render the geometry objects according to the rendering priority, and obtain visualized point cloud data; the asynchronous reading includes local Blob reading and / or data chunk loading based on HTTP Range chunk requests;

[0057] The point cloud segmentation module is used to obtain the user's point cloud segmentation request, determine the polygonal region drawn by the user in the current rendering view based on the point cloud segmentation request, generate the user's oriented bounding box based on the polygonal region, mark the segmentation object based on the oriented bounding box, and process the visualized point cloud data based on the segmentation object.

[0058] According to a third aspect, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the point cloud data processing method described above.

[0059] According to a fourth aspect, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the point cloud data processing method as described above.

[0060] The point cloud data processing method, apparatus, device, and medium of the present invention create an octree for the point cloud data to be processed, and organize the point cloud data using the octree, thereby enabling efficient and multi-level organization and management of large-scale data in three-dimensional space. The nodes in the octree can adopt a proxy node mechanism to achieve lazy loading. By asynchronously acquiring data points and performing concurrent decoding based on whether the data points in the leaf nodes to be rendered are cached, geometric objects are created. Compared with the traditional synchronous loading method, this significantly improves the data loading efficiency and also provides a resource release function. By using rendering judgment logic based on Euclidean distance and screen projection size, the leaf nodes to be rendered in the leaf nodes to be loaded are determined, and the rendering priority of the leaf nodes to be rendered is determined, thereby optimizing the rendering accuracy and achieving real-time rendering. The user's oriented bounding box is generated according to the point cloud segmentation request, the segmentation object is marked according to the oriented bounding box, and the visualized point cloud data is processed according to the segmentation object, which improves the user's segmentation interaction, supports pixel-level accurate segmentation, and improves the segmentation accuracy. This invention provides a comprehensive solution for point cloud data that integrates asynchronous loading, real-time rendering, and precise segmentation. It significantly improves point cloud data processing efficiency, reduces labor costs, supports large-scale data processing, expands the scope of applications, has low deployment costs, and is easy to promote. Attached Figure Description

[0061] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings:

[0062] Figure 1 A flowchart illustrating the point cloud data processing method provided by the present invention is shown.

[0063] Figure 2 This invention illustrates the overall logic diagram of the point cloud data processing method provided by the present invention.

[0064] Figure 3 This diagram illustrates point cloud segmentation based on an octree in the point cloud data processing method provided by the present invention.

[0065] Figure 4A schematic diagram of the point cloud data processing device provided by the present invention is shown.

[0066] Figure 5 A schematic diagram of the hardware structure of the electronic device provided by the present invention is shown. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] With the rapid development of technologies such as LiDAR and 3D scanning, point cloud data is widely used in fields such as autonomous driving, building surveying, and geographic information systems. Point cloud data typically reaches tens of millions or even hundreds of millions of points, placing extremely high performance demands on data processing and rendering.

[0069] Currently, when processing point cloud data, the entire point cloud data needs to be loaded into memory. This makes it impossible to process large-scale point cloud data that exceeds the system memory limit. Synchronously loading point cloud data from memory results in long waiting times when processing large-scale data, leading to high loading latency and noticeable delays in user interaction response. Simultaneously rendering a large amount of point cloud data also causes a significant drop in frame rate, resulting in a clear rendering efficiency bottleneck, poor user experience, and an inability to meet the needs of fine-grained processing.

[0070] In conclusion, how to provide a more efficient, real-time rendering solution for large-scale point cloud data is an important issue that the industry urgently needs to address.

[0071] To address the aforementioned issues, this specification provides a point cloud data processing method aimed at overcoming the performance bottlenecks in large-scale point cloud data processing in existing technologies. It offers a comprehensive solution for efficient asynchronous loading, real-time rendering, and precise segmentation of point cloud data. The point cloud data processing method provided in this specification can be applied to electronic devices, including laptops, desktop computers, smartphones, smart wearable devices, and tablets. Furthermore, the point cloud data processing method provided in this specification can also be applied to applications running on the aforementioned electronic devices. Figure 1 This is a flowchart illustrating a point cloud data processing method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method may include the following steps:

[0072] S101. Obtain the user's point cloud data to be processed and create an octree for the point cloud data to be processed.

[0073] In this embodiment of the invention, an octree structure is used to organize the point cloud data to be processed, resulting in an octree containing the point cloud data. An octree structure is a data structure used to describe three-dimensional space. For massive and unordered point cloud data to be processed, each data point in the point cloud data has its coordinate information in three-dimensional space. Using an octree structure, the point cloud data to be processed can be organized, thereby enabling efficient and multi-level organization and management of the point cloud data in three-dimensional space.

[0074] The point cloud data to be processed can be pre-stored in the electronic device or acquired by the electronic device from an external source. For example, the electronic device may acquire the data from an external data acquisition device.

[0075] There are no restrictions on the specific acquisition method of the point cloud data to be processed, as long as the electronic device can acquire the point cloud data to be processed.

[0076] S102. Based on the octree, determine the initial viewing angle of the point cloud data to be processed, and generate the initial rendering view of the point cloud data to be processed based on the initial viewing angle.

[0077] In this embodiment of the invention, an initial viewing angle is automatically determined. This initial viewing angle loads a simplified representation of the point cloud data to be processed, displays the overall outline and spatial structure of the point cloud data to be processed to the user, and avoids obscuring important parts of the point cloud data, so that the user can obtain a good sense of three-dimensionality and spatial perception, and provides the user with basic navigation control (such as rotation, translation, zoom, etc.) in the future.

[0078] The initial viewing perspective allows the user to see the entire point cloud data to be processed, ensuring that the virtual bounding box of the octree is precisely within the view frustum. This virtual bounding box determines the position and viewing orientation of the virtual camera. By adjusting these parameters, the entire virtual bounding box is made to fit within the view frustum and fill the view as much as possible, while also considering perspective projection distortion. If the user renders the point cloud data using this initial rendering view, the complete data will be displayed.

[0079] S103. Obtain the user's view interaction instructions, generate the user's current rendering view, and determine the leaf nodes to be loaded based on the current rendering view.

[0080] In this embodiment of the invention, based on the hierarchical structure of the octree, starting from the root node, it recursively checks which leaf nodes are within the view frustum of the virtual camera and marks these leaf nodes.

[0081] The virtual camera's view frustum is a hexahedral truncated pyramid, bounded by six planes, specifically including:

[0082] Near clipping plane: The visible plane closest to the virtual camera; Far clipping plane: The visible plane farthest from the virtual camera; Left and right clipping planes: Define the horizontal field of view; Top and bottom clipping planes: Define the vertical field of view.

[0083] User view interaction commands are used to change the rendering perspective / view. View interaction commands may change the camera parameters of the virtual camera. When any camera parameter of the virtual camera changes, the view frustum of the virtual camera may also change. Therefore, when the camera parameters change, the view frustum will be updated synchronously.

[0084] S104. Determine the leaf nodes to be rendered among the leaf nodes to be loaded, and determine the rendering priority of the leaf nodes to be rendered.

[0085] In this embodiment of the invention, Level of Detail (LOD) control is performed before rendering, dynamically adjusting the level of detail of the rendered object based on its importance on the screen. For objects that are far away and occupy a small portion of the screen, a simplified model is used for rendering; for objects that are close and occupy a large portion of the screen, a detailed model is used. Specifically, the rendering priority of a leaf node is determined based on the distance between the leaf node and the virtual camera, as well as the bounding sphere of the leaf node, and an appropriate level of detail is selected for the rendering data of the leaf node. For leaf nodes with low rendering priority, a simplified representation (such as the bounding box of a proxy node or a low-resolution point cloud) may be used.

[0086] S105. Depending on whether the data points in the leaf node to be rendered are cached, asynchronously obtain the data points and perform concurrent decoding, create a geometry object, render the geometry object according to the rendering priority, and obtain the visualized point cloud data.

[0087] Please see Figure 2 In this embodiment of the invention, multi-level caching and intelligent memory management are employed for various types of point cloud data and geometric objects generated from point cloud data. The memory manager used for caching data adopts a multi-level caching architecture, maintaining two storage layers: a Least Recently Used (LRU) memory cache and the browser's IndexedDB persistent storage. Therefore, when loading point cloud data, an asynchronous work pool is used, and corresponding asynchronous interval requests are generated to load cached data from the memory cache and the browser database, respectively.

[0088] Specifically, the system first searches for the corresponding geometry object in the LRU memory cache based on node information (e.g., looking up a specified node key). If the cache is hit, the object is returned directly for the fastest data access. If the memory cache is not hit, the corresponding geometry object is asynchronously searched for in the IndexedDB persistent storage and returned directly. If neither of the two caches is hit, a null value is returned, triggering the data loading process.

[0089] In this embodiment of the invention, to accelerate subsequent access, if the persistent cache is hit, the data will be synchronized to the memory cache and returned after being found.

[0090] Through a multi-level caching architecture, geometric data can be stored in an LRU memory cache to provide fast access capabilities. When the cache is full, the least recently used data is automatically evicted, and the data is asynchronously stored in IndexedDB to ensure data persistence across browser sessions and avoid duplicate network requests. The dual-write mechanism ensures data reliability and access performance.

[0091] The multi-level caching architecture also provides complete resource release capabilities, including clearing the LRU memory cache and calling GPU resource cleanup functions to release graphics resources such as WebGL buffers and textures, preventing memory leaks and supporting the processing of extremely large-scale data. Specifically, an LRU strategy is used to manage the memory cache. When the cache exceeds its limit, the least recently used data is automatically removed, and frequently used data is persisted to IndexedDB for faster access next time. When a node is no longer visible, its data is not immediately removed but retained in the cache. If memory pressure is high, the geometric data of these nodes, including WebGL buffers, is then released.

[0092] The data loading process employs asynchronous data acquisition, which can utilize local Blob reading and / or data chunk loading based on HTTP Range requests. Subsequently, concurrent decoding is performed using a Web Worker thread pool. Once decoding is complete, the data is stored in the global point cloud storage, and corresponding geometry objects are created. The asynchronous worker pool is responsible for maintaining the Web Worker thread pool.

[0093] In this embodiment of the invention, during the rendering of the geometric object, a rendering engine is used for rendering. The rendering engine uses a WebGL shader to render the point cloud. The shader performs visibility checks, color calculations, point size settings, etc.

[0094] S106. Obtain the user's point cloud segmentation request, generate the user's oriented bounding box based on the point cloud segmentation request, mark the segmentation objects based on the oriented bounding box, and process the visualized point cloud data based on the segmentation objects.

[0095] Please see Figure 3 Users select point cloud regions by drawing polygons, perform point cloud segmentation operations, and generate user point cloud segmentation requests. Based on the user-drawn polygons, an Oriented Bounding Box (OBB) is constructed, and the octree is traversed to quickly locate nodes that intersect with the OBB. Points within the intersecting nodes undergo precise polygon interior testing, and points that meet the criteria are marked as segmentation objects.

[0096] This process employs a point-grabbing mechanism; if a point is already occupied by another object, it is reallocated according to preset rules.

[0097] The point cloud data processing method of this invention creates an octree for the point cloud data to be processed, and uses the octree to organize the point cloud data, thereby enabling efficient and multi-level organization and management of large-scale data in three-dimensional space. The nodes in the octree can adopt a proxy node mechanism to achieve lazy loading. By asynchronously acquiring data points and performing concurrent decoding based on whether the data points in the leaf nodes to be rendered are cached, geometric objects are created. Compared with the traditional synchronous loading method, this significantly improves the data loading efficiency and also provides a resource release function. By using rendering judgment logic based on Euclidean distance and screen projection size, the leaf nodes to be rendered in the leaf nodes to be loaded are determined, and the rendering priority of the leaf nodes to be rendered is determined, thereby optimizing the rendering accuracy and achieving real-time rendering. The method generates a user's oriented bounding box based on the point cloud segmentation request, marks the segmentation objects based on the oriented bounding box, and processes the visualized point cloud data based on the segmentation objects, improving the user's segmentation interaction, supporting pixel-level accurate segmentation, and improving segmentation accuracy. This invention provides a comprehensive solution for point cloud data that integrates asynchronous loading, real-time rendering, and precise segmentation. It significantly improves point cloud data processing efficiency, reduces labor costs, supports large-scale data processing, expands the scope of applications, has low deployment costs, and is easy to promote.

[0098] In this embodiment of the invention, step S101 specifically includes:

[0099] S1011. Obtain the point cloud data to be processed and extract the coordinate information of each data point in the point cloud data. For example, by parsing the header information of the user's point cloud data to be processed, the coordinate information of each data point can be obtained.

[0100] S1012. By traversing all the coordinate information, the maximum and minimum values ​​of the point cloud data to be processed on the three coordinate axes in the three-dimensional space can be obtained. Based on the maximum and minimum values ​​of the point cloud data to be processed on the three coordinate axes, the point cloud boundary of the point cloud data to be processed can be determined. Based on the point cloud boundary, the virtual bounding box of the point cloud data to be processed in the three-dimensional space can be determined. This virtual bounding box is a cube that can contain all data points, that is, all data points in the point cloud data to be processed are located inside the virtual bounding box (including the edges).

[0101] It should be noted that the virtual bounding box mentioned in this application refers to the bounding box or bounding box in computer graphics.

[0102] S1013. Use the virtual bounding box as the root node of the octree, and place the data points into the root node in sequence according to their coordinate information. Starting from the root node, determine whether the root node meets the preset termination condition of the octree. The preset termination condition is also the recursive termination condition, which is the key to controlling the depth and fineness of the octree.

[0103] In this embodiment of the invention, the preset termination condition can be: the number of data points in the node does not exceed a preset value; the preset maximum depth is reached, that is, the number of times the octree is split reaches a preset depth; the physical size of the node does not exceed a preset size, for example, when the side length of the node does not exceed the preset side length, it means that the physical size of the node does not exceed the preset size.

[0104] The preset termination conditions can be configured by the user, that is, the various conditions can be configured.

[0105] S1014. Divide the root node into eight equal child nodes, and distribute the data points in the root node to each child node according to the coordinate information of the data points.

[0106] Typically, the root node does not meet the preset termination condition. In this case, the root node is cut in each of the three dimensions of 3D space, dividing it into eight equal child nodes. These eight child nodes are also children of the root node. Then, each data point in the root node is traversed. For the root node, the data points it contains are the point cloud data to be processed. Based on the coordinate information of the data points, the spatial range to which each data point belongs is determined. Based on the spatial range, the child nodes to which each data point belongs can be determined. This allows each data point to be redistributed among the various child nodes.

[0107] S1015. Perform recursive spatial partitioning on each child node until the preset termination condition is reached, generate the node structure and obtain the leaf node, and obtain the octree containing the point cloud data to be processed based on the root node, child nodes and leaf nodes.

[0108] Then, starting from the root node's child nodes, it is determined whether each child node satisfies the preset termination condition of the octree. When a child node meets the preset termination condition, it is marked as a leaf node, and all data points within that leaf node are stored or associated with it, forming the node information of that leaf node. This process is repeated to perform hierarchical recursive spatial partitioning on child nodes and their child nodes. When the recursive process of all branches is completed, a complete octree is generated. By using an octree to store the point cloud data to be processed, massive, unordered point cloud data can be transformed into an ordered, efficiently queried, and manageable data structure.

[0109] Whenever new point cloud data to be processed generates a new octree, the octree can be stored in a preset octree manager for easy identification of subsequent point cloud data to be loaded.

[0110] S1016. Create proxy nodes for the nodes of the octree, determine the node information of the nodes, and associate the node information with the proxy nodes.

[0111] In an octree, a proxy node represents a node that has not yet been loaded with detailed data. It typically contains only the node's metadata, not the actual data points (point cloud data). By creating proxy nodes for nodes in an octree, the overall structure of the octree can be maintained, while avoiding storing all node data points in memory, thus saving memory. During octree initialization, only the hierarchical relationship of the tree can be loaded, instead of immediately loading the data points of each node. When a user needs to access the data points of a specific node in the octree, the node's data points are dynamically loaded based on the node information, and the proxy node is replaced with a complete node containing the actual point cloud data, thereby achieving lazy loading.

[0112] Each node in the octree containing the point cloud data to be processed has its corresponding node information. In this embodiment of the invention, the node information includes, but is not limited to, the following: unique node identifier, spatial bounding box, number of data points contained, byte offset in the data file, data block size, node type (including ordinary nodes and proxy nodes), tree level, child node array, and parent node reference.

[0113] In an octree, each node represents a three-dimensional spatial region, and each node can contain data points, i.e., point cloud data. By serializing the nodes and their data points into a file in a specific format, the point cloud data to be processed can be efficiently stored and accessed. This process generates the byte offset and data block size of the data file in the node information.

[0114] In this embodiment of the invention, users can create proxy nodes or ordinary nodes for each node in the octree according to their needs. For example, proxy nodes can be created for all non-leaf nodes, while leaf nodes are managed using ordinary nodes. Alternatively, proxy nodes can be created for all nodes. When a user needs to access the data points of a node in the octree, for example, when the user's view moves to a certain area and the point cloud data of that area needs to be rendered, the system will check whether the nodes in that area are proxy nodes. If it is a proxy node, the loading process is triggered, and data points are loaded from the file according to the node information. After loading is complete, the proxy node is replaced with a node containing the actual point cloud data, and the rendering is updated.

[0115] Specifically, when it is necessary to load a node and its data points, the data points of that node are read directly from a specific position in the file based on information such as the byte offset and data block size in the data file in the node information, without having to read the entire file. This allows for direct location of the specified position in the file, enabling on-demand loading, avoiding the parsing of the entire file, and greatly reducing loading overhead.

[0116] The point cloud data to be processed is organized based on an octree structure, enabling each node of each octree to support a proxy node mechanism, realize lazy loading, and perform dynamic loading and unloading of nodes, thereby optimizing memory usage.

[0117] In this embodiment of the invention, step S102 specifically includes:

[0118] S1021. Based on the center point and size of the virtual bounding box, and by analyzing the spatial characteristics of the point cloud data to be processed, the main extension direction and the optimal observation direction of the point cloud data to be processed are obtained.

[0119] In this embodiment of the invention, the main extension direction of the point cloud data to be processed can be determined by methods such as Principal Components Analysis (PCA). For example, the distribution pattern of the point cloud data to be processed in three-dimensional space is first analyzed, the covariance matrix is ​​calculated, and then the covariance matrix is ​​decomposed into eigenvalues ​​and sorted according to the eigenvalues ​​to find the direction of maximum change of the point cloud (first principal component), the second largest direction (second principal component), and the direction of minimum change (third principal component). The first principal component usually represents the most important extension direction of the point cloud, and the third principal component often represents the height direction or the direction of minimum change.

[0120] Typically, the camera's orientation, or viewing direction, can be a panoramic view, a top-down view, a front view, etc. The spatial features of the point cloud data to be processed include the shape features of the point cloud data, which contain the proportions of the point cloud data in various directions. Therefore, based on the spatial features, the optimal view type, or the optimal viewing direction mentioned above, can be determined.

[0121] S1022. Use the center point as the observation target point of the virtual camera, and adjust the position of the observation target point using the main extension direction.

[0122] The center point of the virtual bounding box can serve as the observation target point for the virtual camera. The observation target point is then slightly offset along the main extension direction to highlight the main structural features of the point cloud data being processed. Specifically, if there are high-density regions in the point cloud data, the target point may be appropriately shifted towards these important regions.

[0123] S1023. Determine the basic observation distance of the virtual camera based on the size, generate the camera position of the virtual camera based on the basic observation distance, and generate the upward direction vector of the virtual camera based on the camera position and the optimal observation direction. The basic observation distance is the distance between all the point cloud data to be processed within the field of view of the virtual camera.

[0124] Based on the dimensions mentioned above, the basic observation distance is calculated to ensure that the entire point cloud data to be processed is within the field of view of the virtual camera, and then the camera position can be obtained.

[0125] The basic observation distance is the distance from the virtual camera to the observation target point. The basic observation distance can be calculated by the diagonal length of the virtual bounding box and the field of view, or by multiplying the diagonal length by the scaling factor.

[0126] S1024. Generate the projection matrix and view matrix of the virtual camera based on the optimal viewing direction and the upward direction vector.

[0127] The upward direction vector is the virtual camera's upward orientation. The upward orientation ensures a more natural initial viewing angle. The upward direction vector for the panoramic view is the upward direction of the world coordinate system. The upward direction vector for the top view needs to be adjusted to avoid gimbal lock. Other views use the height direction of the point cloud data to be processed as the upward direction vector.

[0128] In this embodiment of the invention, before generating the projection matrix and view matrix of the virtual camera, the viewing angle of the virtual camera can be optimized. This can be achieved by checking whether the line of sight from the optimal viewing distance to the target point is obstructed, further optimizing the optimal viewing distance. If visibility is poor, different viewing angles can be tried to adjust the optimal viewing distance. For example, it can be checked whether there is dense point cloud obstruction along the line of sight (path) from the optimal viewing distance to the target point, and the point cloud density along the line of sight can be evaluated to ensure that the main features are clearly visible.

[0129] To improve the user's initial viewing experience and avoid overly extreme or unnatural viewing angles, it is possible to determine whether the virtual camera is tilted too much and adjust the upward direction vector accordingly. This ensures that the virtual camera is not tilted too much. For example, the angle between the upward direction vector and the horizontal plane can be calculated and the upward direction vector can be adjusted based on this angle.

[0130] S1025. Determine the initial viewing angle based on the projection matrix and the view matrix, and generate the initial rendering view based on the initial viewing angle.

[0131] The vision matrix determines the virtual camera's position, orientation, and viewing direction, transforming the world coordinate system to the camera coordinate system. The projection matrix determines the virtual field of view, projection method, and depth range, projecting the 3D scene onto a 2D screen. After determining these two parameters, the initial viewing angle can be determined, and an initial rendered view can be generated based on this initial viewing angle.

[0132] In this embodiment of the invention, step S103 specifically includes:

[0133] S1031. Obtain the user's view interaction instructions, parse the view interaction instructions, and determine the user's view interaction operation.

[0134] In this embodiment of the invention, after obtaining the user's view interaction instructions, the view interaction instructions are parsed to determine the user's view interaction operation. The view interaction operation includes, but is not limited to: remain unchanged (i.e., render with the initial rendered view), rotate (e.g., generated by the user dragging the mouse or rotating by touch), translate (e.g., generated by the user dragging the right mouse button or translating with the middle mouse button), zoom (e.g., generated by the user zooming with the mouse wheel or with two fingers), and reset.

[0135] S1032. Adjust the camera parameters of the virtual camera according to the view interaction operation, adjust the initial rendering view according to the adjusted camera parameters, and generate the user's current rendering view.

[0136] After obtaining the view interaction operations, the camera parameters of the virtual camera can be adjusted according to the view interaction operations, thereby adjusting the initial rendered view and generating the user's current rendered view.

[0137] S1033. Determine the camera frustum of the virtual camera based on the current rendered view. Starting from the root node of the octree, perform a recursive traversal to determine the spatial relationship between the child bounding boxes of the leaf nodes and the camera frustum. Based on the spatial relationship, determine the leaf nodes to be loaded, and use the data points located within the camera frustum as the point cloud data to be loaded. The child bounding boxes are jointly determined by the virtual bounding box and the node information of the leaf nodes. The child bounding box is a part of the virtual bounding box. The information contained in the node information, such as the tree level, child node array, and parent node reference, can determine the specific part to which it belongs, thus obtaining the child bounding boxes of the leaf nodes.

[0138] When camera parameters change, the projection matrix and / or view matrix will also be adjusted accordingly, and a new view frustum will be generated based on the adjusted projection matrix and / or view matrix.

[0139] For each node, the point cloud data to be loaded is filtered based on whether its child bounding boxes intersect with the camera's view frustum. Specifically: if the node is completely within the camera's view frustum (all data points of the virtual bounding box / child bounding boxes are within the view frustum), then the node and its child nodes need to be loaded; if the node is partially within the camera's view frustum (the child bounding boxes intersect with the camera's view frustum, but not all data points are within it), then the child nodes need to be recursively checked; if the node is completely outside the camera's view frustum (all data points of the virtual bounding box / child bounding boxes are outside the camera's view frustum), then the node and its child nodes are skipped. This method allows data points within the camera's view frustum to be processed as point cloud data to be loaded, while also marking all leaf nodes to be loaded.

[0140] In this embodiment of the invention, step S104 specifically includes:

[0141] S1041. Initialize the visible boundary by setting the minimum value of the visible boundary to positive infinity and the maximum value to negative infinity.

[0142] At the start of each frame's rendering, visible boundaries are initialized, creating an initial empty boundary state. Simultaneously, the visible node array and visible geometry array from the previous frame are cleared to prepare for the new rendering cycle. The current state of the camera parameters is also captured; these parameters will be used for subsequent Euclidean distance and projection calculations.

[0143] S1042. Determine the center points of the child bounding boxes of all leaf nodes to be loaded, and determine the Euclidean distance between each center point and the camera position of the virtual camera. Then, determine the rendering priority of each leaf node based on the Euclidean distance. Rendering priorities can include: highest rendering level, medium rendering level, and lowest rendering level. The smaller the Euclidean distance, the higher the rendering priority; the larger the Euclidean distance, the lower the rendering priority. In other words, there is a certain mapping relationship between rendering priority and Euclidean distance.

[0144] S1043. Determine the child bounding boxes of all leaf nodes to be loaded, and determine the bounding sphere of each leaf node to be loaded and the diameter of the bounding sphere.

[0145] S1044. Determine the camera's field of view and screen height. Based on the diameter of the bounding sphere, the camera's field of view, and the screen height, determine the projected pixel size of each leaf node on the display screen (screen plane). If the projected pixel size exceeds a preset minimum pixel threshold, mark the leaf nodes; these marked leaf nodes are then designated as the leaf nodes to be rendered. Conversely, if the projected pixel size does not exceed the preset minimum pixel threshold, skip rendering the corresponding leaf node's point cloud data to optimize performance.

[0146] Nodes far from the virtual camera occupy only a few pixels on the real display screen. Rendering them with high detail would waste performance. By skipping nodes with small projected sizes on the screen, the number of rendered points can be reduced, thus improving the frame rate. This setting avoids rendering minute details that users cannot see, concentrating computational resources on visually important areas. It provides smooth interaction while maintaining visual quality, ensuring a good visual experience for users.

[0147] In this embodiment of the invention, step S105 specifically includes:

[0148] S1051. Determine whether the data points of the leaf nodes to be rendered are cached in memory based on the node information.

[0149] S1052. If a memory cache hit is determined, the hit cached data is retrieved, and a geometry object is created based on the cached data.

[0150] If the memory cache is not hit, S1053 determines whether the data points of the leaf node to be rendered should be persistently cached based on the node information.

[0151] In this embodiment of the invention, the browser's IndexedDB is used for persistent storage.

[0152] S1054. If it is determined that the persistent cache has been hit, the hit persistent cache data is retrieved, and a geometry object is created based on the persistent cache data.

[0153] S1055. If it is determined that the persistent cache is not hit, the data to be loaded is read asynchronously according to the node information of the leaf node to be rendered, all binary slices are obtained, and the binary slices are decoded concurrently to obtain the geometry object.

[0154] The sub-byte offset and data block size in the node information can determine the range of data to be acquired, as well as the start and end byte positions.

[0155] If the data to be loaded is already cached as a Blob object, binary slice reading can be performed directly; alternatively, a slice request based on HTTP Range can be generated and sent to the server by setting the HTTP Range header, and a slice request with the Range header can be sent to obtain a specified byte range of the point cloud data file. This method supports breakpoint resumption and concurrent download.

[0156] In this embodiment of the invention, all binary slices can be distributed to the Web Worker thread pool for concurrent decoding, thereby creating geometry objects and obtaining point cloud data to be rendered.

[0157] S1056. Extend the minimum and maximum coordinates of the child bounding boxes of the leaf nodes to be rendered into the visible boundary, and finally obtain the minimum bounding box containing all visible content. This allows the geometry object to be rendered to be added to the rendering list.

[0158] This process uses a progressive expansion mechanism to ensure that the final minimum bounding box can tightly enclose all visible content. As more visible nodes are processed, the minimum bounding box gradually expands to accurately reflect the current field of view.

[0159] S1057. Render the geometric object according to the rendering priority to obtain visualized point cloud data, such as using WebGL shaders to render the geometric object.

[0160] To avoid the rendered visualization point cloud data having too low a resolution, resulting in substandard visualization effects, this embodiment of the invention further includes determining the current frame rate of the visualization point cloud data after obtaining it. If the current frame rate does not exceed a preset target frame rate, the minimum pixel threshold is increased, and rendering is re-performed based on the increased minimum pixel threshold until the current frame rate exceeds the preset target frame rate. That is, steps S104 to S105 are re-executed based on the increased minimum pixel threshold to reduce the number of rendering nodes.

[0161] The geometry object to be rendered contains information such as the coordinates, color, intensity, object number, semantic tags, visibility tags, and node information for the leaf nodes. Once the user's point cloud data to be processed is obtained and an octree is created from the point cloud data, the geometry object can be obtained.

[0162] The WebGL shader process includes:

[0163] Uniform variables: These variables are set by JavaScript before drawing, for example, through the gl.uniform family of functions. They are the same for all points. For example, when a user selects an object, the selected object number is set to the object's ID, and the selection color is set to the highlight color (such as red).

[0164] Visibility checking: Allows users to determine the visibility of each point on the GPU. Visibility flags may be set by the CPU-side octree traversal and LOD (Level of Detail) policy process. For example, when constructing geometry objects, points not within the current view frustum or that do not meet LOD conditions may be marked as invisible (setting the visibility flag to 0.0). This allows the WebGL shader to quickly cull these points, preventing them from being rendered.

[0165] Coordinate transformation: Transforms points from local coordinates to screen coordinates using matrix transformations. The model matrix typically contains transformations for nodes (e.g., if an octree node has its own transformation, that transformation is applied). Both the view matrix and the projection matrix are determined by the camera parameters.

[0166] Color calculation: Used to implement point cloud selection and highlighting effects. For example, when a user selects an object, all points of that object will be displayed in the selected color. Similarly, when the mouse hovers over an object, the points of that object will be displayed in the hover color, thus enhancing interactivity.

[0167] Point size setting: In WebGL shaders, users can set the point size using the gl_PointSize variable built into the vertex shader. Setting it to a fixed value ensures that points do not shrink as distance increases, but may cause nearby points to be too large.

[0168] In this embodiment of the invention, step S106 specifically includes:

[0169] S1061. Obtain the point cloud segmentation request, determine the polygonal region drawn by the user in the current rendering view based on the point cloud segmentation request, and set a session unique identifier for the point cloud segmentation request.

[0170] When a user draws a polygonal region on the current rendering view using a mouse or touchscreen to select the point cloud segmentation region of interest, the process also generates a point cloud segmentation request for the user. By drawing the required polygonal region in the current rendering view, the user can achieve precise segmentation of any polygonal region.

[0171] In this embodiment of the invention, a session-unique identifier is also set for each point cloud segmentation request. This session-unique identifier is used for subsequent point-grabbing mechanisms.

[0172] S1062. Determine the polygon vertices of the polygonal region (usually a series of consecutive points forming a closed polygon), and construct a bounding box that completely contains the smallest polygonal region based on the polygon vertices. It is understandable that if the coordinate system of the polygonal region has not yet been transformed to a unified coordinate system, the polygon vertices can be projected onto the world coordinate system using camera parameters, and then a bounding box that completely contains the smallest polygonal region can be constructed based on the projected polygon vertices.

[0173] An OBB (Object-Oriented Bounding Box) includes a center point, three mutually perpendicular axial vectors (unit vectors), and half-lengths along each axis. Unlike traditional axis-aligned bounding boxes, OBB axes can be aligned with the object's main direction, providing a tighter bounding effect and enabling fast spatial queries and collision detection. In this embodiment of the invention, the minimum oriented bounding box can be calculated using methods such as PCA or the minimum area bounding box algorithm.

[0174] S1063. Determine the spatial relationship between the sub-bounding box and the oriented bounding box, and determine the leaf nodes to be segmented based on the spatial relationship.

[0175] Similarly, for each node, the point cloud data to be loaded is filtered based on whether its child bounding boxes intersect with the oriented bounding box. Whether they intersect is determined using the Separating Axis Theorem (SAT). That is, if the data points are completely within the oriented bounding box (all data points of the virtual bounding box / child bounding box are within the oriented bounding box), then the node and its child nodes need to be loaded; if the data points are partially within the oriented bounding box (the child bounding box intersects with the oriented bounding box, but not all data points are within it), then the child nodes need to be recursively checked; if the data points are completely outside the camera's view frustum (all data points of the virtual bounding box / child bounding box are outside the oriented bounding box), then the node and its child nodes are skipped.

[0176] That is, starting from the root node of the octree and recursively traversing it, the spatial relationship between the child bounding box and the oriented bounding box corresponding to the leaf node is determined. Based on the spatial relationship, the leaf node to be segmented is determined, and the data points located within the oriented bounding box are taken as the point cloud data to be segmented.

[0177] S1064. Depending on whether the data points in the leaf nodes to be segmented are cached, asynchronously retrieve the data points and perform concurrent decoding to create a geometry object. The geometry object is retrieved using asynchronous loading and concurrent decoding, as detailed in step S105.

[0178] S1065, and filter the data points in the geometric object, i.e., the point cloud data, point by point.

[0179] In this embodiment of the invention, each data point is traversed, and the following multi-layer filtering is performed based on preset filtering conditions:

[0180] Visibility filtering: Check whether the data point is within the current camera's view frustum and is not occluded. If it is occluded, filter it out to avoid segmenting invisible points. See step S1033 for details.

[0181] OBB containment detection: Determines whether a data point is inside the OBB. Since the OBB is a convex polyhedron, it checks whether each coordinate component is within its half-length.

[0182] Polygon Interior Detection: Determines whether a data point is within the area of ​​a user-drawn polygon. This is typically achieved using ray casting or winding number methods. Since the user-drawn polygon is projected onto a plane, points are projected onto the same plane, and then 2D polygon inclusion detection is performed.

[0183] That is, the preset filtering conditions are: filtering out data points that are occluded by the current camera's view frustum, filtering out data points that are not inside the OBB, and filtering out data points that are not within the polygon area.

[0184] In some special cases, the bounding boxes and polygon regions are not unified to the coordinate system of the octree. In these cases, to better perform point-by-point filtering, the bounding boxes and polygon regions can be transformed to the local coordinate system of the nodes for comparison with the local coordinates of the points. If the node has a corresponding transformation matrix, the transformation matrix is ​​applied. For example, the root transformation matrix can be used for the above transformation process. The root transformation matrix is ​​the transformation matrix of the octree root node in the world coordinate system, usually provided by the data provider. The root transformation matrix is ​​used to synchronize the bounding boxes, polygon regions, and octree to ensure that their offsets are consistent within the same coordinate system. This ensures that subsequent processes perform correct spatial queries and collision detection, avoiding spatial query errors caused by coordinate system inconsistencies.

[0185] S1066. Allocate filtered geometry objects according to the request order indicated by the session unique identifier to obtain segmented objects. That is, allocate filtered geometry objects to each object (execution object) according to the request order indicated by the session unique identifier to obtain segmented objects.

[0186] Check whether the filtered geometry objects have been marked by other objects. If they have been marked, record the original object information and reallocate them according to the request order indicated by the session unique identifier. This process is a point-grabbing mechanism, which realizes the reallocation of point clouds among multiple objects, thereby improving the overall processing efficiency.

[0187] It should be noted that if some geometric objects are marked, the marked data will be reallocated.

[0188] It is understandable that if the filtered geometry objects are not marked, and the corresponding filtered geometry objects are not locked by any execution object, then it is necessary to use the point preemption mechanism to redistribute the point clouds among multiple objects only when it is determined that the filtered geometry objects have been marked.

[0189] S1067. Determine the rendering priority of the segmented object, and render the segmented object according to the rendering priority to obtain visualized point cloud data. The method for determining the rendering priority and the specific details of the rendering method are shown in steps S104 and S105, respectively.

[0190] Set the filtered geometry object numbers to the target object numbers. Simultaneously, update the point color attributes based on the color corresponding to the target object number (so that they are displayed as different colors during rendering). Additionally, update the object number, color array, and other attributes of the nodes in the octree, and store the selected point set in the segmentation data controller.

[0191] The following examples illustrate several typical application scenarios:

[0192] Scenario 1: Autonomous driving point cloud data annotation (data scale: LiDAR data containing 2 million point clouds per frame; processing requirements: real-time segmentation of objects such as vehicles, pedestrians, and roads):

[0193] The system loads the user's point cloud data to be processed and organizes the data based on an octree structure. The user draws a polygonal region through the interactive interface and specifies the object to be segmented (such as a vehicle). The segmentation processing module constructs an OBB, recursively traverses the octree, and asynchronously loads unloaded geometric objects. It performs visibility filtering, polygon interior detection, and OBB containment detection point by point, and processes marked points through a point preemption mechanism. The rendering engine module renders the segmented objects with specified colors based on the LOD strategy and WebGL shaders, maintaining a frame rate of 60 FPS. The entire process takes less than 2 seconds to load and has a segmentation accuracy of >99%.

[0194] Scenario 2: Building Survey Point Cloud Processing (Data Scale: Building scan data containing 50 million point clouds; Processing Requirements: Accurate segmentation of building components, support for multi-user collaboration):

[0195] Multiple users access the system via the network, employing an asynchronous, piecewise loading mechanism to load the building's point cloud data as needed. The first user segments the building's column components, quickly locating the column area using OBB and completing point marking and coloring. When the second user segments the beam components, a point-grabbing mechanism automatically handles the point cloud of overlapping areas between columns and beams, reassigning ownership. All user operations are synchronized in real time, and the rendering engine dynamically updates the scene to ensure smooth rendering during multi-user collaboration. The system reduces redundant loading through multi-level caching, keeping memory usage within a reasonable range and supporting real-time collaborative editing.

[0196] Scenario 3: Geographic Information System Application (Data Scale: 100 million point cloud data covering an urban area; Processing Requirements: terrain analysis, vegetation classification, building recognition):

[0197] The system organizes urban point cloud data hierarchically using an octree structure, loading only node data within the current field of view. When the user selects the vegetation classification function, the system automatically constructs an OBB bounding box for the vegetation area and traverses the octree nodes to filter the point cloud. The segmented vegetation point cloud is marked in green, terrain in gray, and buildings in blue, with the colors dynamically adjusted by shaders. When the user adjusts the viewpoint, the rendering engine updates the visible boundaries in real time and optimizes the rendering accuracy of distant point clouds through LOD control, reducing resource consumption. The entire process uses less than 4GB of memory, supports hierarchical loading and on-demand rendering, and meets the needs of geographic information analysis.

[0198] The point cloud data processing apparatus provided in the embodiments of the present invention will be described below. The point cloud data processing apparatus described below can be referred to in correspondence with the point cloud data processing method described above.

[0199] To address the aforementioned issues, this specification provides a point cloud data processing device designed to overcome the performance bottleneck in large-scale point cloud data processing in the prior art, offering a comprehensive solution for efficient asynchronous loading, real-time rendering, and precise segmentation of point cloud data. Figure 4 This is a schematic diagram of the structure of a point cloud data processing device according to an embodiment of the present invention, as shown below. Figure 4 As shown, the device may include:

[0200] Point cloud organization module 10 is used to acquire user point cloud data to be processed and create an octree for the point cloud data to be processed.

[0201] The initial generation module 20 is used to determine the initial viewing angle of the point cloud data to be processed based on the octree, and to generate the initial rendering view of the point cloud data to be processed based on the initial viewing angle.

[0202] The perspective adjustment module 30 is used to obtain the user's view interaction commands, generate the user's current rendered view, and determine the leaf nodes to be loaded based on the current rendered view.

[0203] The rendering preparation module 40 is used to determine the leaf nodes to be rendered among the leaf nodes to be loaded, and to determine the rendering priority of the leaf nodes to be rendered.

[0204] The asynchronous loading module 50 is used to asynchronously acquire data points and perform concurrent decoding based on whether the data points in the leaf node to be rendered are cached, create geometry objects, render the geometry objects according to the rendering priority, and obtain visualized point cloud data.

[0205] In this embodiment of the invention, to accelerate subsequent access, if the persistent cache is hit, the data will be synchronized to the memory cache and returned after being found.

[0206] Through a multi-level caching architecture, geometric data can be stored in an LRU memory cache to provide fast access capabilities. When the cache is full, the least recently used data is automatically evicted, and the data is asynchronously stored in IndexedDB to ensure data persistence across browser sessions and avoid duplicate network requests. The dual-write mechanism ensures data reliability and access performance.

[0207] The multi-level caching architecture also provides complete resource release capabilities, including clearing the LRU memory cache and calling GPU resource cleanup functions to release graphics resources such as WebGL buffers and textures, preventing memory leaks and supporting the processing of extremely large-scale data. Specifically, an LRU strategy is used to manage the memory cache. When the cache exceeds its limit, the least recently used data is automatically removed, and frequently used data is persisted to IndexedDB for faster access next time. When a node is no longer visible, its data is not immediately removed but retained in the cache. If memory pressure is high, the geometric data of these nodes, including WebGL buffers, is then released.

[0208] The data loading process employs asynchronous data acquisition, which can utilize local Blob reading and / or data chunk loading based on HTTP Range requests. Subsequently, concurrent decoding is performed using a Web Worker thread pool. Once decoding is complete, the data is stored in the global point cloud storage, and corresponding geometry objects are created. The asynchronous worker pool is responsible for maintaining the Web Worker thread pool.

[0209] The point cloud segmentation module 60 is used to obtain the user's point cloud segmentation request, generate the user's oriented bounding box based on the point cloud segmentation request, mark the segmentation object based on the oriented bounding box, and process the visualized point cloud data based on the segmentation object.

[0210] The point cloud data processing device of this invention creates an octree for the point cloud data to be processed, and uses the octree to organize the point cloud data, thereby enabling efficient and multi-level organization and management of large-scale data in three-dimensional space. The nodes in the octree can adopt a proxy node mechanism to achieve lazy loading. By asynchronously acquiring data points and performing concurrent decoding based on whether the data points in the leaf nodes to be rendered are cached, geometric objects are created. Compared with the traditional synchronous loading method, this significantly improves the data loading efficiency and also provides a resource release function. Through rendering judgment logic based on Euclidean distance and screen projection size, the leaf nodes to be rendered in the leaf nodes to be loaded are determined, and the rendering priority of the leaf nodes to be rendered is determined, thereby optimizing the rendering accuracy and achieving real-time rendering. The device generates a user's oriented bounding box based on the point cloud segmentation request, marks the segmentation objects based on the oriented bounding box, and processes the visualized point cloud data based on the segmentation objects, improving the user's segmentation interaction, supporting pixel-level accurate segmentation, and improving the segmentation accuracy. This invention provides a comprehensive solution for point cloud data that integrates asynchronous loading, real-time rendering, and precise segmentation. It significantly improves point cloud data processing efficiency, reduces labor costs, supports large-scale data processing, expands the scope of applications, has low deployment costs, and is easy to promote.

[0211] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical commands in the memory 530 to execute a point cloud data processing method, which includes:

[0212] Obtain the user's point cloud data to be processed, and create an octree for the point cloud data to be processed;

[0213] Based on the octree, determine the initial viewing angle of the point cloud data to be processed, and generate the initial rendering view of the point cloud data to be processed based on the initial viewing angle.

[0214] Obtain the user's view interaction commands, generate the user's current rendered view, and determine the leaf nodes to be loaded based on the current rendered view;

[0215] Identify the leaf nodes to be rendered among the leaf nodes to be loaded, and determine the rendering priority of the leaf nodes to be rendered.

[0216] Depending on whether the data points in the leaf node to be rendered are cached, the data points are asynchronously obtained and concurrently decoded, a geometry object is created, and the geometry object is rendered according to the rendering priority to obtain the visualized point cloud data.

[0217] Obtain the user's point cloud segmentation request, generate the user's oriented bounding box based on the point cloud segmentation request, mark the segmentation objects based on the oriented bounding box, and process the visualized point cloud data based on the segmentation objects.

[0218] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0219] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0220] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of processing point cloud data, characterized by, The method comprises: acquiring user point cloud data to be processed, creating an octree for the point cloud data to be processed; determining an initial observation perspective of the point cloud data to be processed according to the octree, and generating an initial rendering view of the point cloud data to be processed according to the initial observation perspective; acquiring a view interaction instruction of the user, generating a current rendering view of the user, and determining a leaf node to be loaded according to the current rendering view; determining a leaf node to be rendered in the leaf node to be loaded, and determining a rendering priority of the leaf node to be rendered; asynchronously acquiring data points and performing concurrent decoding according to whether the data points in the leaf node to be rendered are cached, creating a geometry object, rendering the geometry object according to the rendering priority, and obtaining visualized point cloud data; the asynchronous acquisition comprises local Blob reading and / or data slice loading based on HTTP Range slice request; acquiring a point cloud segmentation request of the user, determining a polygon region drawn by the user in the current rendering view according to the point cloud segmentation request, generating a directional bounding box of the user according to the polygon region, marking a segmentation object according to the directional bounding box, and processing the visualized point cloud data according to the segmentation object; the acquiring of the view interaction instruction of the user, the generating of the current rendering view of the user, and the determining of the leaf node to be loaded according to the current rendering view specifically comprises: acquiring a view interaction instruction of the user, analyzing the view interaction instruction, and determining a view interaction operation of the user; adjusting camera parameters of a virtual camera according to the view interaction operation, adjusting the initial rendering view according to the adjusted camera parameters, and generating the current rendering view of the user; determining a camera frustum of the virtual camera according to the current rendering view, taking a root node of the octree as a starting point and performing recursive traversal, determining a spatial relationship between a child bounding box corresponding to the leaf node and the camera frustum, and determining the leaf node to be loaded according to the spatial relationship; the child bounding box is determined by the virtual bounding box and node information of the leaf node, and the child bounding box is part of the virtual bounding box; the acquiring of the point cloud segmentation request of the user, the generating of the directional bounding box of the user according to the point cloud segmentation request, the marking of the segmentation object according to the directional bounding box, and the processing of the visualized point cloud data according to the segmentation object specifically comprises: acquiring a point cloud segmentation request, determining a polygon region drawn by the user in the current rendering view according to the point cloud segmentation request, and setting a session unique identifier for the point cloud segmentation request; determining polygon vertices of the polygon region, and constructing a smallest directional bounding box containing the polygon region according to the polygon vertices; determining a spatial relationship between the child bounding box and the directional bounding box, and determining a leaf node to be segmented according to the spatial relationship; asynchronously acquiring data points and performing concurrent decoding according to whether the data points in the leaf node to be segmented are cached, and creating a geometry object; performing point-by-point filtering on the data points in the geometry object according to a preset filtering condition; allocating the filtered geometry object according to a request order indicated by the session unique identifier, and obtaining the segmentation object; checking whether the filtered geometry object has been marked by other objects, recording original object information if the filtered geometry object has been marked, and reallocating according to the request order indicated by the session unique identifier; Determine the rendering priority of the segmented object, render the segmented object according to the rendering priority, and obtain the visual point cloud data.

2. The method of processing point cloud data according to claim 1, wherein, The method comprises the following steps: Obtain the to-be-processed point cloud data, and extract the coordinate information of each data point in the to-be-processed point cloud data; Traverse all the coordinate information, determine the point cloud boundary of the to-be-processed point cloud data, and determine the virtual bounding box of the to-be-processed point cloud data in the three-dimensional space according to the point cloud boundary; Take the virtual bounding box as the root node of the octree, and sequentially put the data points into the root node according to the coordinate information of the data points; Divide the root node into eight equal child nodes, and distribute the data points in the root node to the child nodes according to the coordinate information of the data points; Recursively divide each child node until a preset termination condition is reached, generate a node structure and obtain leaf nodes, and obtain the octree of the to-be-processed point cloud data according to the root node, the child nodes and the leaf nodes; Create a proxy node for each node of the octree, determine the node information of the node, and associate the node information with the proxy node.

3. The method of processing point cloud data according to claim 1, wherein, According to the octree, determine the initial observation view angle of the to-be-processed point cloud data, and generate an initial rendering view of the to-be-processed point cloud data according to the initial observation view angle, which comprises the following steps: According to the center point and the size of the virtual bounding box, and by analyzing the spatial characteristics of the to-be-processed point cloud data, obtain the main extension direction and the best observation direction of the to-be-processed point cloud data; Take the center point as the observation target point of the virtual camera, and adjust the position of the observation target point by using the main extension direction; Determine the basic observation distance of the virtual camera according to the size, generate the camera position of the virtual camera according to the basic observation distance, and generate the up direction vector of the virtual camera according to the camera position and the best observation direction; the basic observation distance is the distance at which all the to-be-processed point cloud data are within the field of view of the virtual camera; Generate the projection matrix and the view matrix of the virtual camera according to the best observation direction and the up direction vector; Determine the initial observation view angle according to the projection matrix and the view matrix, and generate the initial rendering view according to the initial observation view angle.

4. The method of processing point cloud data according to claim 1, wherein, According to whether the data points in the to-be-rendered leaf nodes are cached, asynchronously obtain the data points and perform concurrent decoding, create a geometric object, render the geometric object according to the rendering priority, and obtain the visual point cloud data, which comprises the following steps: Determine whether the data points in the to-be-rendered leaf nodes are cached in the memory according to the node information; In the case of determining that the memory cache hits, call the hit cache data, and create a geometric object according to the cache data; In the case of determining that the memory cache misses, determine whether the data points in the to-be-rendered leaf nodes are persistently cached according to the node information; In the case of determining that the persistent cache hits, call the hit persistent cache data, and create a geometric object according to the persistent cache data; In the case of determining that the persistent cache misses, asynchronously read the to-be-loaded data according to the node information of the to-be-rendered leaf nodes, obtain all the binary slices, and concurrently decode the binary slices to obtain a geometric object; The minimum and maximum coordinate points of the child bounding box of the leaf node to be rendered are extended into the visible boundary; The geometric object is rendered according to the rendering priority, and visual point cloud data is obtained.

5. The method of processing point cloud data according to claim 1, wherein, The leaf node to be rendered is determined from the leaf nodes to be loaded, and the rendering priority of the leaf node to be rendered is determined, specifically including: The visible boundary is initialized, and the minimum value of the visible boundary is set to positive infinity and the maximum value is set to negative infinity; The center points of the child bounding boxes of all the leaf nodes to be loaded are determined, and the Euclidean distances between the center points and the camera position of the virtual camera are determined, and the rendering priority of each leaf node is determined according to the Euclidean distances; the rendering priority has a certain mapping relationship with the Euclidean distance; The child bounding boxes of all the leaf nodes to be loaded are determined, the enclosing sphere of each leaf node to be loaded and the diameter of the enclosing sphere are determined; The camera field of view angle of the virtual camera and the screen height are determined, the projection pixel size of each leaf node on the display screen is determined according to the diameter of the enclosing sphere, the camera field of view angle and the screen height, and in the case that the projection pixel size exceeds a preset minimum pixel threshold, the corresponding leaf node is taken as the leaf node to be rendered.

6. An apparatus for processing point cloud data, characterized by comprising: The device comprises: A point cloud organization module configured to obtain user point cloud data to be processed, and create an octree for the point cloud data to be processed; An initial generation module configured to determine an initial observation view angle of the point cloud data to be processed according to the octree, and generate an initial rendering view of the point cloud data to be processed according to the initial observation view angle; A view angle adjustment module configured to obtain a view interaction instruction of a user, generate a current rendering view of the user, and determine leaf nodes to be loaded according to the current rendering view; A rendering preparation module configured to determine a leaf node to be rendered from the leaf nodes to be loaded, and determine a rendering priority of the leaf node to be rendered; An asynchronous loading module configured to asynchronously obtain data points and perform concurrent decoding according to whether the data points in the leaf node to be rendered are cached, create a geometric object, render the geometric object according to the rendering priority, and obtain visual point cloud data; the asynchronous obtaining includes local Blob reading and / or data segment loading based on HTTP Range-based segment request; A point cloud segmentation module configured to obtain a point cloud segmentation request of a user, determine a polygon region drawn by the user in the current rendering view according to the point cloud segmentation request, generate a directional bounding box of the user according to the polygon region, mark a segmentation object according to the directional bounding box, and process the visual point cloud data according to the segmentation object; The view angle adjustment module specifically comprises: Obtaining a view interaction instruction of a user, analyzing the view interaction instruction, and determining a view interaction operation of the user; Adjusting camera parameters of a virtual camera according to the view interaction operation, adjusting the initial rendering view according to the adjusted camera parameters, and generating a current rendering view of the user. According to the current rendering view, a camera frustum of a virtual camera is determined, an octree root node is taken as a starting point and recursively traversed, a spatial relationship between a child bounding box corresponding to a leaf node and the camera frustum is determined, and a leaf node to be loaded is determined according to the spatial relationship; the child bounding box is determined by a virtual bounding box and node information of the leaf node, and the child bounding box is part of the virtual bounding box; The point cloud segmentation module specifically includes: A point cloud segmentation request is acquired, a polygon region drawn by a user in a current rendering view is determined according to the point cloud segmentation request, and a session unique identifier is set for the point cloud segmentation request; Polygon vertices of the polygon region are determined, and a minimum directional bounding box completely containing the polygon region is constructed according to the polygon vertices; A spatial relationship between the child bounding box and the directional bounding box is determined, and a leaf node to be segmented is determined according to the spatial relationship; Data points in the leaf node to be segmented are asynchronously acquired and concurrently decoded according to whether the data points are cached, and a geometry object is created; Data points in the geometry object are filtered point by point according to a preset filtering condition; The filtered geometry object is distributed according to a request order indicated by the session unique identifier, and a segmented object is obtained; whether the filtered geometry object has been marked by other objects is checked, and if so, original object information is recorded, and the filtered geometry object is redistributed according to the request order indicated by the session unique identifier; A rendering priority of the segmented object is determined, the segmented object is rendered according to the rendering priority, and visualized point cloud data is obtained.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the point cloud data processing method of any one of claims 1 to 5 when executing the program.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the steps of the point cloud data processing method of any one of claims 1 to 5 when executed by the processor.

Images