Spatial search method, device and equipment supporting mass point cloud data and medium

Abstract

The application provides a kind of spatial retrieval method, device, equipment and medium supporting mass point cloud data, it is related to point cloud processing technical field, comprising: the space that the point cloud data to be handled occupies is carried out stereoscopic division and obtains multiple layer space grid set;According to the orientation information corresponding in the space grid to which the space grid in last layer space grid set belongs, the unique grid index code of space grid is constructed;The mapping relationship between point cloud data and grid index code is established;Point cloud data is rearranged and stored in target storage area based on mapping relationship;Receive retrieval request and determine to be rendered grid index code, retrieve to be rendered point cloud data from target storage area according to to be rendered grid index code, realize the retrieval and real-time rendering of to be rendered point cloud data.The application can support unlimited data volume point cloud data fast retrieval and real-time rendering, greatly improve point cloud data retrieval and rendering efficiency.

Description

Technical Field

[0001] This invention relates to the field of point cloud processing technology, and in particular to a spatial retrieval method, apparatus, device, and medium that supports massive point cloud data. Background Technology

[0002] 3D laser scanning data can reach GB (gigabytes) or even TB (terabytes), posing significant challenges to data storage, retrieval, and real-time rendering. Without real-time point cloud updates, it's difficult to meet the visualization needs of massive point clouds. Memory consumption easily becomes a bottleneck when processing massive point clouds. Ordinary processing methods may lead to excessive memory usage, impacting system performance, and index query efficiency may be low, slowing down data retrieval during rendering and affecting real-time rendering results. Techniques based on multithreading and complex data structures for processing massive data place high demands on hardware performance. On devices with low hardware configurations, the expected fast rendering, display, and interactive effects may not be achievable. Utilizing multithreading techniques such as computation threads, I / O (Input / Output) threads, rendering threads, and data unloading threads to manage and schedule internal and external memory data presents significant challenges in coordinating and synchronizing threads. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a spatial retrieval method, apparatus, device and medium that supports massive point cloud data, which can support fast retrieval and real-time rendering of point cloud data with unlimited data volume, and greatly improve the efficiency of point cloud data retrieval and rendering.

[0004] In a first aspect, the present invention provides a spatial retrieval method supporting massive point cloud data, comprising:

[0005] The space occupied by the point cloud data to be processed is divided into three-dimensional segments to obtain a multi-layer spatial grid set;

[0006] For any spatial grid in a multi-layer spatial grid set, construct a unique grid index code for the spatial grid according to the orientation information of the spatial grid in the spatial grid set of the previous layer;

[0007] Establish a mapping relationship between point cloud data and the grid index code corresponding to each spatial grid in a multi-layer spatial grid set;

[0008] The point cloud data is rearranged and stored in the target storage area based on the mapping relationship;

[0009] If a retrieval request for point cloud data is received, the index code of the grid to be rendered is determined, and the point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered.

[0010] In one implementation, the space occupied by the point cloud data to be processed is divided into multiple spatial mesh sets in a three-dimensional manner, including:

[0011] The space occupied by the point cloud data to be processed is divided into multiple subspaces, and the subspaces with more than a preset threshold are further subdivided until the number of points in each subspace is less than the preset threshold, thus obtaining the point cloud layering depth corresponding to the point cloud data.

[0012] Determine the bounding box of the point cloud corresponding to the space occupied by the point cloud data;

[0013] The point cloud bounding box is used as the first-layer spatial mesh set. The spatial mesh set is then divided into two layers for each spatial mesh in the first-layer spatial mesh set. The process continues until the point cloud layer depth is satisfied, at which point the spatial mesh set is stopped, resulting in a multi-layer spatial mesh set.

[0014] In one implementation, for any spatial grid in a multi-layer spatial grid set, a unique grid index code for the spatial grid is constructed according to the orientation information of the spatial grid to which it belongs in the previous layer spatial grid set, including:

[0015] For any spatial grid in a multi-layer spatial grid set, the spatial grid to which the spatial grid belongs in the previous layer spatial grid set is denoted as the parent spatial grid, the spatial grid is denoted as the first child spatial grid, and the other spatial grids obtained by dividing the parent spatial grid into three dimensions are denoted as the second child spatial grid.

[0016] Based on the relative positional relationship between the first sub-level spatial grid and each second sub-level spatial grid, the orientation information of the first sub-level spatial grid in the parent spatial grid is determined, and each orientation information is pre-configured with a corresponding orientation code.

[0017] The grid index code of the parent spatial grid is combined with the azimuth code corresponding to the first child spatial grid to obtain the grid index code of the child spatial grid.

[0018] In one implementation, establishing a mapping relationship between point cloud data and the grid index code corresponding to each spatial grid within a multi-layer spatial grid set includes:

[0019] Based on the total number of points contained in the point cloud data, determine the number of points to be assigned for each layer of the spatial grid set in the multi-layer spatial grid set;

[0020] The point cloud data is sampled according to the number of points to be assigned in order to determine the layer number information corresponding to each point in the point cloud data. The layer number information is used to describe the layer of the spatial grid set to which the point belongs.

[0021] For any target point in the point cloud data, based on the coordinate information and the layer number information corresponding to the target point, determine the target spatial grid in its own spatial grid set, so as to establish a mapping relationship between the grid index code of the target point and the target spatial grid.

[0022] In one implementation, determining the target spatial grid into which the target point falls within its respective spatial grid set, based on the coordinate information and layer number information corresponding to the target point, includes:

[0023] Based on the layer number information corresponding to the target point, determine the logical edge length of the spatial grid corresponding to the spatial grid set to which the target point belongs;

[0024] For each spatial grid set between the first spatial grid set and the spatial grid set to which the target point belongs, the coordinate information of the bounding box of the point cloud data and the logical side length of the spatial grid are normalized according to the coordinate information of the bounding box of the point cloud data and the spatial grid, so as to determine the orientation information of the target point in the spatial grid set based on the normalized coordinate information.

[0025] Based on the orientation information of the target point in each layer of the spatial grid set, determine the target spatial grid in which the target point falls within its respective spatial grid set.

[0026] In one implementation, rearranging and storing point cloud data in a target storage area based on a mapping relationship includes:

[0027] The arrangement order of point cloud data is determined based on the mapping relationship and the hierarchical and block order of the spatial grid. The point cloud data is then rearranged and stored in the target storage area according to the arrangement order.

[0028] In one implementation, if a retrieval request for point cloud data is received, the index encoding of the mesh to be rendered is determined, including:

[0029] If a retrieval request for point cloud data is received, the coverage area of ​​the visual cone is determined based on the anterior and posterior plane information of the visual cone and the camera information carried in the retrieval request.

[0030] Candidate grid index codes are determined based on the coordinates of multiple vertices of the visual cone coverage area, and spatial grid coverage area is generated based on the candidate grid index codes;

[0031] Based on the intersection relationship between the view frustum coverage area and the spatial mesh coverage area, the index code of the mesh to be rendered within the view frustum coverage area is determined.

[0032] Secondly, the present invention also provides a spatial retrieval device supporting massive point cloud data, comprising:

[0033] The 3D partitioning module is used to partition the space occupied by the point cloud data to be processed into a multi-layer spatial mesh set.

[0034] The encoding construction module is used to construct a unique grid index code for any spatial grid in a multi-layer spatial grid set, based on the orientation information of the spatial grid in the spatial grid set of the previous layer.

[0035] The mapping establishment module is used to establish the mapping relationship between point cloud data and the grid index code corresponding to each spatial grid in the multi-layer spatial grid set;

[0036] The rearrangement and storage module is used to rearrange point cloud data according to the mapping relationship and store it in the target storage area;

[0037] The point cloud retrieval module is used to determine the index code of the grid to be rendered when a retrieval request for point cloud data is received. The module then retrieves the point cloud data to be rendered from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, thereby realizing the retrieval and real-time rendering of the point cloud data to be rendered.

[0038] Thirdly, the present invention also provides an electronic device including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement any of the methods provided in the first aspect.

[0039] Fourthly, the present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement any of the methods provided in the first aspect.

[0040] This invention provides a spatial retrieval method, apparatus, device, and medium supporting massive point cloud data. First, the space occupied by the point cloud data to be processed is three-dimensionally divided into a multi-layered spatial grid set. Then, for any spatial grid in the multi-layered spatial grid set, a unique grid index code is constructed based on the orientation information of the spatial grid corresponding to its corresponding spatial grid in the previous layer. Next, a mapping relationship is established between the point cloud data and the grid index code corresponding to each spatial grid in the multi-layered spatial grid set. Based on the mapping relationship, the point cloud data is rearranged and stored in a target storage area. If a retrieval request for point cloud data is received, the grid index code to be rendered is determined, and the point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the grid index code to be rendered, thus realizing the retrieval and real-time rendering of the point cloud data to be rendered. The above method does not rely on hardware configuration. Through three-dimensional spatial division, grid index code construction, establishment of the mapping relationship between point cloud and grid index code, and point cloud rearrangement and storage, it can support fast retrieval and real-time rendering of point cloud data with unlimited data volume, greatly improving the efficiency of point cloud data retrieval and rendering.

[0041] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0043] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 A flowchart illustrating a spatial retrieval method supporting massive point cloud data provided in an embodiment of the present invention;

[0045] Figure 2 This is an overall flowchart of a spatial retrieval method supporting massive point cloud data provided in an embodiment of the present invention;

[0046] Figure 3 An example diagram of grid index encoding provided in an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the structure of a spatial retrieval device supporting massive point cloud data provided in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Currently, existing technologies are insufficient to meet the visualization needs of massive point clouds. Based on this, the present invention provides a spatial retrieval method, device, equipment, and medium that supports massive point cloud data. It does not rely on hardware configuration, performs spatial three-dimensional partitioning of massive point cloud data, and achieves rapid spatial retrieval and real-time rendering of massive point cloud data with unlimited data volume through steps such as spatial index encoding of spatial grids and determining the index encoding of the grid to be rendered based on window spatial data.

[0051] To facilitate understanding of this embodiment, a spatial retrieval method supporting massive point cloud data disclosed in this embodiment of the invention will first be described in detail. See [link to relevant documentation]. Figure 1 The diagram shows a spatial retrieval method supporting massive point cloud data. The method mainly includes the following steps S102 to S110:

[0052] Step S102: The space occupied by the point cloud data to be processed is divided into three dimensions to obtain a multi-layer spatial grid set.

[0053] Each layer of the multi-layer spatial grid set comprises multiple spatial grids. The first layer contains point cloud bounding boxes. The second layer contains multiple spatial grids obtained by stereo subdividing the point cloud bounding boxes within the first layer. The third layer contains multiple spatial grids obtained by stereo subdividing each spatial grid within the second layer, and so on. Furthermore, the number of layers in the multi-layer spatial grid set is positively correlated with the size of the point cloud data.

[0054] In one example, the point cloud layering depth is first determined based on the scale of the point cloud data. Then, the bounding box of the point cloud data is divided into layers in a stereoscopic manner until the point cloud layering depth is reached, at which point the stereoscopic division can be stopped, resulting in a multi-layer spatial mesh set.

[0055] Step S104: For any spatial grid in the multi-layer spatial grid set, construct a unique grid index code for the spatial grid according to the orientation information of the spatial grid in the spatial grid set of the previous layer.

[0056] The grid index code consists of multiple layers of orientation codes. For any spatial grid, the first layer of orientation code is used to represent the orientation information of the spatial grid in the first layer of spatial grid set, the second layer of orientation code is used to represent the orientation information of the spatial grid in the second layer of spatial grid set, and so on. The orientation information of the spatial grid in different layers of spatial grid set is recorded through the grid index code.

[0057] In one example, the azimuth codes of multiple spatial grids belonging to the same upper-level spatial grid can be determined based on the azimuth information. By combining the grid index codes of the upper-level spatial grid, the grid index codes of the multiple spatial grids can be obtained separately. Here, multiple spatial grids belonging to the same upper-level spatial grid are those obtained by three-dimensionally subdividing the same spatial grid within the upper-level spatial grid set.

[0058] Step S106: Establish the mapping relationship between point cloud data and the grid index code corresponding to each spatial grid in the multi-layer spatial grid set.

[0059] In one example, the number of points to be assigned to each spatial grid set is first determined (i.e., the number of points to be assigned). Then, for each spatial grid set, point cloud data is collected according to the number of points to be assigned to determine the spatial grid set to which each point in the point cloud data is assigned. Finally, based on the coordinate information of the point, the target spatial grid that each point will eventually fall into in its assigned spatial grid set is determined, and a mapping relationship between the grid index code of the point and the target spatial grid it falls into is established.

[0060] Step S108: The point cloud data is rearranged and stored in the target storage area based on the mapping relationship.

[0061] The target storage area can be a virtual storage area or a local storage area, such as a specified disk. In one example, the arrangement order of the point cloud data can be determined based on the mapping relationship and the hierarchical block order of the spatial grid, and then the point cloud data can be rearranged and stored in the target storage area according to the arrangement order.

[0062] Step S110: If a retrieval request for point cloud data is received, the index code of the grid to be rendered is determined, and the point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered.

[0063] The retrieval request can carry information about the anterior and posterior planes of the view frustum and camera information. Using this information, the coverage area of ​​the view frustum can be determined, which is also the display area of ​​the retrieval request. Candidate grid index codes can be roughly selected based on the vertex coordinates of the view frustum coverage area. By intersecting the view frustum coverage area with the spatial grid coverage areas corresponding to the candidate grid index codes, the grid index code to be rendered within the display area can be determined. The point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the grid index code to be rendered, thus realizing the retrieval and real-time rendering of the point cloud data to be rendered.

[0064] The spatial retrieval method for massive point cloud data provided in this invention does not rely on hardware configuration. Through spatial three-dimensional partitioning, grid index encoding construction, establishment of the mapping relationship between point cloud and grid index encoding, and point cloud rearrangement and storage, it can support fast retrieval and real-time rendering of point cloud data with unlimited data volume, greatly improving the efficiency of point cloud data retrieval and rendering.

[0065] For ease of understanding, this embodiment of the invention provides a specific implementation of a spatial retrieval method supporting massive point cloud data, see [link to relevant documentation]. Figure 2 The diagram illustrates the overall flowchart of a spatial retrieval method supporting massive point cloud data. This method involves steps such as octree-based spatial 3D partitioning, 3D mesh spatial index encoding, establishing a mapping relationship between point cloud data and the spatial network, rearranging point cloud data according to the spatial mesh mapping relationship, determining the grid index encoding to be rendered based on the viewport space, and retrieving and rendering point cloud data in real time based on the grid index encoding. Specifically:

[0066] (1) Spatial 3D partitioning based on octree principle: The space occupied by point cloud data is divided into layers / blocks according to the octree principle. This mainly includes the following (1.1) to (1.3):

[0067] (1.1) Divide the space occupied by the point cloud data to be processed into multiple subspaces, and further subdivide the subspaces with a number of points greater than a preset threshold until the number of points in each subspace is less than the preset threshold, thereby obtaining the point cloud layering depth corresponding to the point cloud data.

[0068] In one example, starting from the root node, the space containing the point cloud data is divided into eight subspaces. For each subspace, the number of points it contains is calculated. If the number of points in a subspace exceeds a preset threshold, the subspace is further subdivided, increasing the layer depth; conversely, if the number of points in a subspace is below the preset threshold, subdivision of that subspace is stopped. This process is repeated until all subspaces meet the stop-subdivision condition. The final octree depth is the point cloud layer depth. The stop-subdivision condition is that the number of points in a subspace is below the preset threshold.

[0069] (1.2) Determine the bounding box of the point cloud corresponding to the space occupied by the point cloud data.

[0070] In one example, the edges of the axial bounding box are parallel to the coordinate axes, which is a simple and direct method of bounding box representation. The bounding box and its coordinate information of the point cloud are determined by finding the minimum and maximum values ​​of the point cloud data on each coordinate axis.

[0071] (1.3) Take the point cloud bounding box as the first layer of spatial mesh set, and continue to perform stereo division on each spatial mesh in the first layer of spatial mesh set to obtain the second layer of spatial mesh set until the point cloud layer depth is satisfied, and stop the stereo division to obtain a multi-layer spatial mesh set.

[0072] The spatial grid can be a cube. In one example, the cube whose bounding box coordinates are identified is marked as the first-layer cube (i.e., the first-layer spatial grid set); the first-layer cube is divided into 8 smaller cubes, which are the second-layer cubes (i.e., the second-layer spatial grid set), and so on. Each layer's smaller cube is further divided into 8 smaller cubes, which are the cubes of the next layer, until the point cloud layer depth is satisfied, and all the cube coordinates are obtained.

[0073] (2) Three-dimensional grid spatial index coding: Construct a unique grid index coding for a multi-level spatial grid set divided by an octree.

[0074] The encoding rules for spatial index coding are as follows: The spatial grid is generated by dividing it according to the properties of an octree. Each spatial grid is further refined into eight sub-level spatial grids, and each spatial grid is numbered 0 to 8 according to its orientation. The eight spatial grids are divided into upper and lower levels. The upper level is numbered clockwise, corresponding to 0 to 3, and the lower level is numbered clockwise, corresponding to 4 to 8. The maximum encoding value is 8, corresponding to the binary code 111, which can be represented using 3 bits. Following this rule, three bits are used to represent the orientation code, and the numbers 0 to 8 are represented in binary, resulting in eight numbers: 000, 001, 010, 011, 100, 101, 110, and 111, each uniquely corresponding to an orientation.

[0075] This invention uses a 32-bit integer as the carrier, storing a directional code in groups of 3 bits from left to right. The first-layer directional code represents the directional information of the spatial grid within the first-layer spatial grid set; the second-layer directional code represents the directional information of the spatial grid within the second-layer spatial grid set, and so on, recording the directional information of the spatial grid in different layers to generate a unique grid index code. This grid index code ensures a one-to-one mapping from spatial grids to grid index codes; each spatial grid corresponds to a unique grid index code. Furthermore, the spatial location of the grid can be quickly calculated based on the grid index code. The characteristic that spatially adjacent grid index codes also have adjacent numerical values ​​provides a guarantee for efficient querying and locating of subsequent batch data.

[0076] Based on the above encoding rules, this embodiment of the invention provides a specific implementation method for constructing a unique grid index encoding for a spatial grid, mainly including the following (2.1) to (2.3):

[0077] (2.1) For any spatial grid in a multi-layer spatial grid set, the spatial grid to which the spatial grid belongs in the previous layer spatial grid set is called the parent spatial grid, the spatial grid is called the first child spatial grid, and the other spatial grids obtained by dividing the parent spatial grid into three dimensions are called the second child spatial grids.

[0078] For example, see Figure 3 The diagram shows an example of grid index encoding. Taking the third-level spatial grid set as an example, it is assumed that a certain spatial grid (denoted as the parent spatial grid) in the second-level spatial grid set is divided into 3D to obtain 8 spatial grids in the last level. Any one of the 8 spatial grids is denoted as the first child spatial grid, and the remaining 7 spatial grids in the 8 spatial grids are all denoted as the second child spatial grids.

[0079] (2.2) Based on the relative positional relationship between the first sub-level spatial grid and each second sub-level spatial grid, determine the azimuth information of the first sub-level spatial grid in the parent spatial grid. Each azimuth information is pre-configured with a corresponding azimuth code.

[0080] Please continue reading Figure 3 By determining the relative positional relationship between the first sub-level spatial grid and each second sub-level spatial grid, the orientation information of the first sub-level spatial grid is determined, and then the orientation code 110 is uniquely corresponding to this orientation information.

[0081] (2.3) Combine the grid index code of the parent spatial grid with the azimuth code corresponding to the first child spatial grid to obtain the grid index code of the child spatial grid.

[0082] To reduce the computational cost of encoding, the grid index code of the parent spatial grid can be directly combined with the azimuth code corresponding to the first child spatial grid. Please continue reading... Figure 3 By combining the orientation code 110 of the first-level spatial grid with the grid index code 001-010 of the parent spatial grid, the grid index code 001-010-110 corresponding to the first-level spatial grid can be obtained. This process can be repeated to gradually determine the grid index code corresponding to each spatial grid in subsequent levels.

[0083] In this embodiment of the invention, every three bits represent a directional code, and a 32-bit integer can represent a spatial grid set of up to 10 layers. This can be used to encode a total of 8^10 spatial grids, exceeding 1 billion spatial grids, which is sufficient to support business needs.

[0084] (3) Establishing a mapping relationship between point cloud data and spatial networks mainly includes the following (3.1) to (3.3):

[0085] (3.1) Based on the total number of points contained in the point cloud data, determine the number of points to be assigned for each layer of the spatial grid set in the multi-layer spatial grid set.

[0086] In one example, a hierarchical index E is selected based on the data size and distribution characteristics. Let totalNum be the total number of points contained in the point cloud data. The number of points to be assigned corresponding to each layer of the spatial grid set can be obtained according to the following equation. :

[0087] * totalNum.

[0088] (3.2) The point cloud data is sampled according to the number of points to be assigned in order to determine the layer number information corresponding to each point in the point cloud data. The layer number information is used to describe the layer of the spatial grid set to which the point belongs.

[0089] In one example, for each spatial grid set, a sampling method is used to uniformly extract point cloud data, ensuring that the number of points contained in each spatial grid set matches the number of points to be assigned in that layer, and that the point cloud data is uniformly distributed, preserving details. In this embodiment of the invention, by uniformly extracting point cloud data for each spatial grid set, the spatial grid set to which each point in the point cloud data belongs can be determined, and the layer number of the spatial grid set to which it belongs can then be used as the layer number information.

[0090] Furthermore, the sampling method employs random sampling, and the random sampling process uses a random number generator based on the Mason twitch algorithm to improve the quality of random number generation and solve the problem of insufficient randomness provided by traditional pseudo-random sequences, which leads to overly dense or sparse data distribution. The random numbers are used to represent the identifiers of points within the point cloud data. For example, for the first-layer spatial grid set, if the number of points to be assigned is X, then the random number generator generates X random numbers. The points represented by these X random numbers are then identified as belonging to the first-layer spatial grid set; that is, the layer number information corresponding to the points represented by these X random numbers is "1".

[0091] (3.3) For any target point in the point cloud data, based on the coordinate information and the layer number information corresponding to the target point, determine the target spatial grid in its own spatial grid set, so as to establish the mapping relationship between the grid index encoding of the target point and the target spatial grid. See (I) to (III) below for details:

[0092] (I) Determine the logical edge length of the spatial grid corresponding to the spatial grid set to which the target point belongs based on the layer information of the target point.

[0093] For example, if a multi-layer spatial grid set includes 4 layers and the target point belongs to a layer of 3, then it is necessary to determine the logical edge length of the spatial grid corresponding to the 3rd layer spatial grid set.

[0094] This invention provides a specific implementation method for determining the logical side length of a spatial grid:

[0095] The spatial grid logical edge length interval is calculated based on the target's assigned layer information. The maximum number of layers to be assigned (i.e., the point cloud layer depth, 4 layers) is denoted as maxLevel. If the target point is sampled and assigned to a layer level, it is calculated according to the following bitwise operation formula:

[0096] .

[0097] (II) For each spatial grid set between the first spatial grid set and the spatial grid set to which the target point belongs, the coordinate information of the bounding box of the point cloud corresponding to the space occupied by the point cloud data and the logical side length of the spatial grid are normalized to determine the orientation information of the target point in the spatial grid set based on the normalized coordinate information.

[0098] Specifically, the coordinate information of the target point is represented as Pointer, the bounding box of the point cloud is called boundary, and the logical side length of the spatial grid is interval. The point coordinates are normalized according to the following formula.

[0099] ;

[0100] ;

[0101] .

[0102] For example, calculating interval0 in layer 1 yields the normalized coordinates (ux, uy, uz) of the target point in layer 2. After normalization, the azimuth code of the target point in layer 1 is obtained. Assuming the normalized coordinates of the target point in layer 1 are ux=1.0, uy=0.2, uz=0.8, rounding the coordinates according to the order zyz gives an approximate value of 101, which is the azimuth code of the target point in layer 1. Continuing to calculate the azimuth code of the target point in layer 2, assuming the normalized coordinates are ux=0.1, uy=0.2, uz=0.1, rounding the coordinates according to the order zyz gives an approximate value of 000, which is the azimuth code of the target point in layer 2. Similarly, assuming the normalized coordinates of the target point in layer 3 are ux=0.1, uy=0.2, uz=1.2, the approximate azimuth code is 001.

[0103] (III) Based on the orientation information of the target point in each layer of the spatial grid set, determine the target spatial grid in which the target point falls within its respective spatial grid set. In the specific implementation, the orientation code of the target point in each layer is obtained, and finally the grid index code is obtained by the three-dimensional grid spatial index encoding method, for example, "101-000-001", which determines the target spatial grid in which the target point falls within its respective spatial grid set.

[0104] (4) Rearrange point cloud data according to spatial grid mapping relationship: Determine the arrangement order of point cloud data based on mapping relationship and spatial grid hierarchical block order, and rearrange and store point cloud data in target storage area according to arrangement order. This mainly includes the following (4.1) to (4.2):

[0105] (4.1) Point cloud LAS file rearrangement: The original point cloud LAS file point data is usually stored in the scanning order of devices such as LiDAR, without special design for layered and block-based reading and writing. In order to make full use of the sequential reading characteristics of mechanical hard disks, the point cloud data is stored sequentially in spatial layers and blocks. This can reduce the movement of the mechanical hard disk head, facilitate batch reading of data in layers and blocks, and speed up the extraction and parsing of point cloud data.

[0106] (4.2) Disk index file generation: Record the hard disk index corresponding to the grid index code and write it to the file index file. For example, the index file format is: |Grid index code|File offset|Data unit length|.

[0107] (5) Determine the grid index encoding to be rendered based on the viewport space, mainly including the following (5.1) to (5.3):

[0108] (5.1) If a retrieval request for point cloud data is received, the coverage area of ​​the visual cone is determined based on the anterior and posterior plane information of the visual cone and the camera information carried in the retrieval request.

[0109] The viewing cone is the region encompassing all objects that the camera may see. It is a truncated pyramid shape, where the near plane corresponds to the smaller face of the pyramid, and the far plane corresponds to the larger face. The extent of the viewing cone can be calculated using the following (a) through (e):

[0110] (a) Determining the planes of the view frustum: The view frustum has six planes, including four side planes (left, right, top, and bottom) and two end planes (near plane and far plane). These planes can be calculated using the projection matrix and the view matrix.

[0111] (b) Calculate the plane equations: The equation for each plane can be calculated using the elements of the Model-View-Projection Matrix (MVP matrix). For each plane, the equation is of the form Ax + By + Cz + D = 0, where A, B, C, and D are the coefficients of the plane equation.

[0112] (c) Determining the near and far planes: The near and far planes are the two end faces of the viewing cone, defining the closest and farthest distances that the camera can see. The near plane is usually closer to the camera, while the far plane is farther away from the camera.

[0113] (d) Calculate the initial cone coverage area: The cone coverage area can be calculated by determining the vertices of the cone, which are the intersections of the near and far planes with the four sides of the cone. These vertices determine the shape and size of the cone in three-dimensional space, i.e., the cone coverage area.

[0114] (e) View frustum culling: During rendering, objects outside the view frustum can be culled to improve rendering efficiency. This involves determining whether an object intersects with the view frustum and whether it is completely outside the view frustum.

[0115] (5.2) Determine the candidate grid index code based on the coordinates of multiple vertices of the vertebral body coverage area, and generate the spatial grid coverage area based on the candidate grid index code.

[0116] In one example, a hexahedral model is established based on the coverage area of ​​the visual cone, and the coordinates of eight vertices in the front and back planes are obtained. The grid index code of the spatial grid to which the vertex falls is determined based on the vertex coordinates and the layer information (see (3) above for details, which will not be repeated in this embodiment of the invention). The maximum and minimum values ​​of the grid index codes corresponding to the vertices are used to delineate the coverage area of ​​the point spatial grid, so as to reduce the amount of calculation and improve the search efficiency.

[0117] (5.3) Based on the intersection relationship between the view frustum coverage area and the spatial grid coverage area, determine the grid index code to be rendered within the view frustum coverage area.

[0118] (6) The retrieval and real-time rendering of point cloud data are achieved based on the index encoding of the grid to be rendered, mainly through the following (6.1) to (6.4):

[0119] (6.1) Load the disk index file.

[0120] (6.2) Based on the grid index to be rendered, find the required data offset and data block length in the disk index file.

[0121] (6.3) Extract binary data blocks from the rearranged point cloud data based on the data offset and data block length, and use the point cloud processing driver module to process and restore the coordinates and other attribute information of the point cloud data.

[0122] (6.4) Driven by real-time frustum data, point cloud data is extracted from the disk on demand, freeing it from the limitations of computer memory and enabling the retrieval and real-time rendering of massive point cloud data.

[0123] In summary, this invention proposes a novel spatial retrieval coding technique for 3D networks based on octree spatial partitioning. It establishes a mapping relationship between point cloud data and the spatial network, and then rearranges the point cloud data according to this mapping relationship. The index code of the grid to be rendered is calculated based on the viewport space, and point cloud data retrieval and real-time rendering are achieved based on this grid index code. This method supports rapid retrieval and real-time rendering of point cloud data with unlimited data volume, greatly improving the efficiency of point cloud data retrieval and rendering.

[0124] To verify the effectiveness of this invention, over 700 million laser point cloud data points from the Xiaolangdi area, totaling 23GB, were used as test data. The test computer's performance was: Intel(R) Core(TM) I7-10700 CPU @2.9GHz, 64GB of RAM, and an NVIDIA Quadro P1000 graphics card. The first test involved directly reading the over 700 million laser point cloud data points into memory without any block or layering processing. After 30 minutes, the point cloud data still could not be rendered. The second test used the proposed method. The results showed that the Xiaolangdi laser point cloud data, after being preprocessed using this method (blocking, layering, indexing, etc.), took 20 minutes, and the software rendering took 18 seconds. Editing the building point cloud allowed for rapid retrieval of building point cloud IDs, real-time updating of point cloud data attributes, and rendering.

[0125] Through experimentation, the method for rapid spatial retrieval and real-time rendering of massive point cloud data proposed in this embodiment of the invention can realize rapid retrieval and real-time rendering of large-scale point cloud data, which is of great significance for subsequent fine classification and manual editing of point cloud data and 3D modeling of point cloud data.

[0126] Based on the foregoing embodiments, this invention provides a spatial retrieval device that supports massive point cloud data. See [link to previous embodiment]. Figure 4 The diagram shows a spatial retrieval device that supports massive point cloud data. The device mainly includes the following parts:

[0127] The 3D partitioning module 402 is used to 3D partition the space occupied by the point cloud data to be processed to obtain a multi-layer spatial grid set.

[0128] The encoding construction module 404 is used to construct a unique grid index code for any spatial grid in a multi-layer spatial grid set, based on the orientation information of the spatial grid in the spatial grid set of the previous layer.

[0129] The mapping establishment module 406 is used to establish the mapping relationship between point cloud data and the grid index code corresponding to each spatial grid in the multi-layer spatial grid set;

[0130] The rearrangement and storage module 408 is used to rearrange point cloud data according to the mapping relationship and store it in the target storage area.

[0131] The point cloud retrieval module 410 is used to determine the index code of the grid to be rendered if a retrieval request for point cloud data is received, and retrieve the point cloud data to be rendered from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered.

[0132] The spatial retrieval device for massive point cloud data provided in this invention does not rely on hardware configuration. Through spatial three-dimensional division, grid index encoding construction, establishment of mapping relationship between point cloud and grid index encoding, and point cloud rearrangement and storage, it can support fast retrieval and real-time rendering of point cloud data with unlimited data volume, greatly improving the efficiency of point cloud data retrieval and rendering.

[0133] In one embodiment, the three-dimensional division module 402 is specifically used for:

[0134] The space occupied by the point cloud data to be processed is divided into multiple subspaces, and the subspaces with more than a preset threshold are further subdivided until the number of points in each subspace is less than the preset threshold, thus obtaining the point cloud layering depth corresponding to the point cloud data.

[0135] Determine the bounding box of the point cloud corresponding to the space occupied by the point cloud data;

[0136] The point cloud bounding box is used as the first-layer spatial mesh set. The spatial mesh set is then divided into two layers for each spatial mesh in the first-layer spatial mesh set. The process continues until the point cloud layer depth is satisfied, at which point the spatial mesh set is stopped, resulting in a multi-layer spatial mesh set.

[0137] In one implementation, the encoding construction module 404 is specifically used for:

[0138] For any spatial grid in a multi-layer spatial grid set, the spatial grid to which the spatial grid belongs in the previous layer spatial grid set is denoted as the parent spatial grid, the spatial grid is denoted as the first child spatial grid, and the other spatial grids obtained by dividing the parent spatial grid into three dimensions are denoted as the second child spatial grid.

[0139] Based on the relative positional relationship between the first sub-level spatial grid and each second sub-level spatial grid, the orientation information of the first sub-level spatial grid in the parent spatial grid is determined, and each orientation information is pre-configured with a corresponding orientation code.

[0140] The grid index code of the parent spatial grid is combined with the azimuth code corresponding to the first child spatial grid to obtain the grid index code of the child spatial grid.

[0141] In one implementation, the mapping establishment module 406 is specifically used for:

[0142] Based on the total number of points contained in the point cloud data, determine the number of points to be assigned for each layer of the spatial grid set in the multi-layer spatial grid set;

[0143] The point cloud data is sampled according to the number of points to be assigned in order to determine the layer number information corresponding to each point in the point cloud data. The layer number information is used to describe the layer of the spatial grid set to which the point belongs.

[0144] For any target point in the point cloud data, based on the coordinate information and the layer number information corresponding to the target point, determine the target spatial grid in its own spatial grid set, so as to establish a mapping relationship between the grid index code of the target point and the target spatial grid.

[0145] In one implementation, the mapping establishment module 406 is specifically used for:

[0146] Based on the layer number information corresponding to the target point, determine the logical edge length of the spatial grid corresponding to the spatial grid set to which the target point belongs;

[0147] For each spatial grid set between the first spatial grid set and the spatial grid set to which the target point belongs, the coordinate information of the bounding box of the point cloud data and the logical side length of the spatial grid are normalized according to the coordinate information of the bounding box of the point cloud data and the spatial grid, so as to determine the orientation information of the target point in the spatial grid set based on the normalized coordinate information.

[0148] Based on the orientation information of the target point in each layer of the spatial grid set, determine the target spatial grid in which the target point falls within its respective spatial grid set.

[0149] In one embodiment, the rearrangement and storage module 408 is specifically used for:

[0150] The arrangement order of point cloud data is determined based on the mapping relationship and the hierarchical and block order of the spatial grid. The point cloud data is then rearranged and stored in the target storage area according to the arrangement order.

[0151] In one implementation, the point cloud retrieval module 410 is specifically used for:

[0152] If a retrieval request for point cloud data is received, the coverage area of ​​the visual cone is determined based on the anterior and posterior plane information of the visual cone and the camera information carried in the retrieval request.

[0153] Candidate grid index codes are determined based on the coordinates of multiple vertices of the visual cone coverage area, and spatial grid coverage area is generated based on the candidate grid index codes;

[0154] Based on the intersection relationship between the view frustum coverage area and the spatial mesh coverage area, the index code of the mesh to be rendered within the view frustum coverage area is determined.

[0155] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0156] This invention provides an electronic device, specifically, the electronic device includes a processor and a storage device; the storage device stores a computer program, and the computer program, when run by the processor, executes the method described in any of the above embodiments.

[0157] Figure 5 The present invention provides a schematic diagram of the structure of an electronic device 100, which includes a processor 50, a memory 51, a bus 52 and a communication interface 53. The processor 50, the communication interface 53 and the memory 51 are connected through the bus 52. The processor 50 is used to execute executable modules, such as computer programs, stored in the memory 51.

[0158] The memory 51 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 53 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0159] Bus 52 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0160] The memory 51 is used to store programs. After receiving an execution instruction, the processor 50 executes the programs. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 50 or implemented by the processor 50.

[0161] Processor 50 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 50 or by instructions in software form. Processor 50 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 51. The processor 50 reads the information in memory 51 and, in conjunction with its hardware, completes the steps of the above method.

[0162] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0163] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0164] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A spatial retrieval method supporting massive point cloud data, characterized in that, include: The space occupied by the point cloud data to be processed is divided into three-dimensional segments to obtain a multi-layer spatial grid set; For any spatial grid in the multi-layer spatial grid set, a unique grid index code for the spatial grid is constructed according to the orientation information of the spatial grid to which the spatial grid belongs in the previous layer spatial grid set. Establish a mapping relationship between the point cloud data and the grid index code corresponding to each spatial grid in the multi-layer spatial grid set; The point cloud data is rearranged and stored in the target storage area based on the mapping relationship; If a retrieval request for the point cloud data is received, the index code of the grid to be rendered is determined, and the point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered. Establishing a mapping relationship between the point cloud data and the grid index code corresponding to each spatial grid in the multi-layer spatial grid set includes: determining the number of points to be assigned corresponding to each layer of the multi-layer spatial grid set based on the total number of points contained in the point cloud data; sampling the point cloud data according to the number of points to be assigned to determine the layer number information corresponding to each point in the point cloud data, wherein the layer number information is used to describe the layer number of the spatial grid set to which the point belongs; for any target point in the point cloud data, determining the target spatial grid in its spatial grid set according to the coordinate information corresponding to the target point and the layer number information, so as to establish a mapping relationship between the target point and the grid index code of the target spatial grid. Based on the coordinate information corresponding to the target point and the belonging layer information, determining the target spatial grid in which the target point falls within its own spatial grid set includes: determining the logical side length of the spatial grid corresponding to the spatial grid set to which the target point belongs based on the belonging layer information corresponding to the target point; for each spatial grid set from the first spatial grid set to the spatial grid set to which the target point belongs, normalizing the coordinate information corresponding to the target point based on the coordinate information of the bounding box of the point cloud data and the logical side length of the spatial grid, so as to determine the orientation information of the target point in that spatial grid set based on the normalized coordinate information; and determining the target spatial grid in which the target point falls within its own spatial grid set based on the orientation information corresponding to the target point in each spatial grid set. If a retrieval request for the point cloud data is received, the grid index code to be rendered is determined, including: if a retrieval request for the point cloud data is received, the view frustum coverage area is determined based on the view frustum anterior and posterior plane information and camera information carried in the retrieval request; candidate grid index codes are determined based on the coordinates of multiple vertices of the view frustum coverage area, and a spatial grid coverage area is generated based on the candidate grid index codes; and the grid index code to be rendered within the view frustum coverage area is determined based on the intersection relationship between the view frustum coverage area and the spatial grid coverage area.

2. The spatial retrieval method supporting massive point cloud data according to claim 1, characterized in that, The space occupied by the point cloud data to be processed is divided into three dimensions to obtain a multi-layer spatial mesh set, including: The space occupied by the point cloud data to be processed is divided into multiple subspaces, and the subspaces with more than a preset threshold are further subdivided until the number of points in each subspace is less than the preset threshold, so as to obtain the point cloud layering depth corresponding to the point cloud data. Determine the bounding box of the point cloud corresponding to the space occupied by the point cloud data; The point cloud bounding box is used as the first layer of spatial mesh set. The spatial mesh set is then divided into two layers for each spatial mesh in the first layer to obtain the second layer of spatial mesh set. The division is stopped when the point cloud layer depth is satisfied, resulting in a multi-layer spatial mesh set.

3. The spatial retrieval method supporting massive point cloud data according to claim 1, characterized in that, For any spatial grid in the multi-layer spatial grid set, a unique grid index code for the spatial grid is constructed according to the orientation information of the spatial grid to which the spatial grid belongs in the previous layer spatial grid set, including: For any spatial grid in the multi-layer spatial grid set, the spatial grid to which the spatial grid belongs in the previous layer spatial grid set is denoted as the parent spatial grid, the spatial grid is denoted as the first child spatial grid, and other spatial grids obtained by three-dimensionally dividing the parent spatial grid are denoted as the second child spatial grid. Based on the relative positional relationship between the first sub-level spatial grid and each second sub-level spatial grid, the orientation information of the first sub-level spatial grid in the parent spatial grid is determined, and each orientation is pre-configured with a corresponding orientation code; The grid index code of the parent spatial grid is combined with the azimuth code corresponding to the first child spatial grid to obtain the grid index code of the child spatial grid.

4. The spatial retrieval method supporting massive point cloud data according to claim 1, characterized in that, The point cloud data is rearranged and stored in the target storage area based on the mapping relationship, including: The arrangement order of the point cloud data is determined based on the mapping relationship and the hierarchical and block order of the spatial grid. The point cloud data is then rearranged and stored in the target storage area according to the arrangement order.

5. A spatial retrieval device supporting massive point cloud data, characterized in that, include: The 3D partitioning module is used to partition the space occupied by the point cloud data to be processed into a multi-layer spatial mesh set. The encoding construction module is used to construct a unique grid index code for any spatial grid in the multi-layer spatial grid set, according to the orientation information of the spatial grid to which the spatial grid belongs in the previous layer spatial grid set. The mapping establishment module is used to establish a mapping relationship between the point cloud data and the grid index code corresponding to each spatial grid in the multi-layer spatial grid set; The rearrangement and storage module is used to rearrange the point cloud data according to the mapping relationship and store it in the target storage area. The point cloud retrieval module is used to determine the grid index code to be rendered if a retrieval request for the point cloud data is received, and retrieve the point cloud data to be rendered from the rearranged point cloud data stored in the target storage area according to the grid index code to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered. The mapping establishment module is specifically used for: determining the number of points to be assigned corresponding to each layer of the spatial grid set in the multi-layer spatial grid set based on the total number of points contained in the point cloud data; sampling the point cloud data according to the number of points to be assigned to determine the layer number information corresponding to each point in the point cloud data, wherein the layer number information is used to describe the layer number of the spatial grid set to which the point belongs; for any target point in the point cloud data, determining the target spatial grid in its spatial grid set according to the coordinate information corresponding to the target point and the layer number information, so as to establish a mapping relationship between the grid index code of the target point and the target spatial grid. The mapping establishment module is specifically used for: determining the logical side length of the spatial grid corresponding to the spatial grid set to which the target point belongs, based on the belonging layer information corresponding to the target point; for each spatial grid set between the first spatial grid set and the spatial grid set to which the target point belongs, normalizing the coordinate information corresponding to the target point based on the coordinate information of the bounding box of the point cloud data and the logical side length of the spatial grid, so as to determine the orientation information of the target point in the spatial grid set of that layer based on the normalized coordinate information; Based on the orientation information of the target point in each layer of the spatial grid set, determine the target spatial grid in which the target point falls within its respective spatial grid set; The point cloud retrieval module is specifically used for: if a retrieval request for the point cloud data is received, determining the coverage area of ​​the visual cone based on the anterior and posterior plane information and camera information carried in the retrieval request; determining candidate grid index codes based on the coordinates of multiple vertices of the coverage area of ​​the visual cone; and generating a spatial grid coverage area based on the candidate grid index codes. Based on the intersection relationship between the view frustum coverage area and the spatial grid coverage area, the index code of the grid to be rendered within the view frustum coverage area is determined.

6. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method described in any one of claims 1 to 4.

Images