Space retrieval method and device supporting massive point cloud data, equipment and medium

Abstract

The invention provides a space retrieval method and device supporting mass point cloud data, equipment and a medium, and relates to the technical field of point cloud processing, and the method comprises the steps: carrying out the three-dimensional division of a space occupied by to-be-processed point cloud data, and obtaining a multi-layer space grid set; constructing a unique grid index code of the space grid according to the corresponding azimuth information of the space grid in the space grid to which the space grid belongs in the upper-layer space grid set; establishing a mapping relation between the point cloud data and the grid index codes; rearranging the point cloud data based on the mapping relation and storing the point cloud data in a target storage area; and receiving a retrieval request, determining the index code of the to-be-rendered grid, and retrieving the to-be-rendered point cloud data from the target storage area according to the index code of the to-be-rendered grid, thereby realizing retrieval and real-time rendering of the to-be-rendered point cloud data. According to the method, quick retrieval and real-time rendering of the point cloud data with unlimited data volume can be supported, and the retrieval and rendering efficiency of the point cloud data is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of point cloud processing technology, and in particular to a spatial retrieval method, device, equipment and medium supporting massive point cloud data. Background Art

[0002] The amount of 3D laser scanning data can reach GB (Gigabyte) or even TB (Terabyte), which poses great challenges to data storage, retrieval and real-time rendering. If the point cloud cannot be refreshed in real time, it will be difficult to meet the visualization needs of massive point clouds. When processing massive point clouds, memory consumption can easily become a bottleneck. Ordinary processing methods may cause a large amount of memory to be occupied, affecting system performance. At the same time, the index query efficiency may also be low, resulting in a slower speed of acquiring data during the rendering process, affecting the real-time rendering effect. The technology based on multi-threading and complex data structures to process massive data has high requirements for hardware performance. On devices with low hardware configuration, the expected fast rendering display and interactive operation effects may not be achieved. The use of multi-threading technology such as computing threads, IO (Input / Output Thread) threads, rendering threads, and data unloading threads to achieve the scheduling and management of internal and external memory data makes it difficult to coordinate and synchronize threads. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a spatial retrieval method, device, equipment and medium that supports massive point cloud data, which can support rapid retrieval and real-time rendering of point cloud data of unlimited data volume, greatly improving the efficiency of point cloud data retrieval and rendering.

[0004] In a first aspect, the present invention provides a spatial retrieval method supporting massive point cloud data, comprising: The space occupied by the point cloud data to be processed is divided into three dimensions to obtain a multi-layer spatial grid set; For any spatial grid in the multi-layer spatial grid set, a unique grid index code of the spatial grid is constructed according to the corresponding position information of the spatial grid in the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set; Establishing a mapping relationship between point cloud data and a grid index code corresponding to each spatial grid in a multi-layer spatial grid set; Rearrange the point cloud data based on the mapping relationship and store it in the target storage area; If a retrieval request for point cloud data is received, the index code of the grid to be rendered is determined, and the point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, thereby realizing the retrieval and real-time rendering of the point cloud data to be rendered.

[0005] In one embodiment, the space occupied by the point cloud data to be processed is divided into three dimensions to obtain a multi-layer space grid set, including: The space occupied by the point cloud data to be processed is divided into multiple subspaces, and the subspaces with the number of points greater than a preset threshold are subdivided until the number of points contained in each subspace is less than the preset threshold, and the point cloud layer depth corresponding to the point cloud data is obtained; Determine the point cloud bounding box corresponding to the space occupied by the point cloud data; The point cloud bounding box is used as the first layer of spatial grid set, and each spatial grid in the first layer of spatial grid set is continuously divided into three dimensions to obtain the second layer of spatial grid set, until the point cloud layering depth is met and the three-dimensional division is stopped to obtain a multi-layer spatial grid set.

[0006] In one implementation, for any spatial grid in the multi-layer spatial grid set, a unique grid index code of the spatial grid is constructed according to the corresponding position information of the spatial grid in the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set, including: For any spatial grid in the multi-layer spatial grid set, the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set is recorded as the parent spatial grid, the spatial grid is recorded as the first child spatial grid, and the other spatial grids obtained by three-dimensionally dividing the parent spatial grid are recorded as the second child spatial grids; Determine the position information corresponding to the first sub-level spatial grid in the parent-level spatial grid according to the relative position relationship between the first sub-level spatial grid and each second sub-level spatial grid, each position information is pre-configured with a corresponding position code; The grid index code of the parent spatial grid is combined with the orientation code corresponding to the first child spatial grid to obtain the grid index code of the child spatial grid.

[0007] In one embodiment, establishing a mapping relationship between point cloud data and a grid index code corresponding to each spatial grid in a multi-layer spatial grid set includes: According to the total number of points contained in the point cloud data, determine the number of points to be allocated corresponding to each layer of the spatial grid set in the multi-layer spatial grid set; The point cloud data is sampled according to the number of points to be assigned, so as to determine the belonging layer number information corresponding to each point in the point cloud data, and the belonging layer number information is used to describe the number of layers of the spatial grid set to which the point belongs; For any target point in the point cloud data, the target spatial grid into which the target point falls in the set of spatial grids to which it belongs is determined according to the coordinate information and the number of layers corresponding to the target point, so as to establish a mapping relationship between the target point and the grid index code of the target spatial grid.

[0008] In one implementation, determining the target spatial grid in which the target point falls in the set of spatial grids to which it belongs, according to the coordinate information and the belonging layer number information corresponding to the target point, includes: According to the belonging layer information corresponding to the target point, determine the logical side length of the spatial grid corresponding to the spatial grid set to which the target point belongs; For each layer of spatial grid sets between the first layer of spatial grid sets and the spatial grid sets to which the target point belongs, the coordinate information corresponding to the target point is normalized according to the coordinate information of the point cloud bounding box corresponding to the space occupied by the point cloud data and the logical side length of the spatial grid, so as to determine the orientation information corresponding to the target point in the spatial grid sets of this layer based on the normalized coordinate information; According to the corresponding position information of the target point in each layer of the spatial grid set, the target spatial grid into which the target point falls in the spatial grid set to which it belongs is determined.

[0009] In one implementation, rearranging the point cloud data based on the mapping relationship and storing it in a target storage area includes: The arrangement order of the point cloud data is determined based on the mapping relationship and the hierarchical and blocking order of the spatial grid, and the point cloud data is rearranged and stored in the target storage area according to the arrangement order.

[0010] In one implementation, if a search request for point cloud data is received, determining the index code of the mesh to be rendered includes: If a search request for point cloud data is received, the coverage range of the viewing frustum is determined based on the front and rear plane information of the viewing frustum and the camera information carried in the search request; Determine candidate grid index codes according to multiple vertex coordinates of a viewing frustum coverage range, and generate a spatial grid coverage range based on the candidate grid index codes; Based on the intersection relationship between the viewing frustum coverage range and the spatial grid coverage range, the index code of the grid to be rendered within the viewing frustum coverage range is determined.

[0011] In a second aspect, the present invention further provides a spatial retrieval device supporting massive point cloud data, comprising: A three-dimensional division module is used to perform three-dimensional division on the space occupied by the point cloud data to be processed to obtain a multi-layer spatial grid set; A coding construction module, for constructing a unique grid index code of any spatial grid in the multi-layer spatial grid set according to the corresponding position information of the spatial grid in the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set; A mapping establishment module is used to establish a mapping relationship between point cloud data and a grid index code corresponding to each spatial grid in a multi-layer spatial grid set; A rearrangement and storage module is used to rearrange the point cloud data according to the mapping relationship and store it in the target storage area; The point cloud retrieval module is used to determine the index code of the grid to be rendered if a retrieval request for point cloud data is received, and retrieve the point cloud data to be rendered from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered.

[0012] In a third aspect, the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement any one of the methods provided in the first aspect.

[0013] In a fourth aspect, the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.

[0014] The present invention provides a spatial retrieval method, device, equipment and medium that supports massive point cloud data. First, the space occupied by the point cloud data to be processed is three-dimensionally divided to obtain a multi-layer spatial grid set; then, for any spatial grid in the multi-layer spatial grid set, a unique grid index code of the spatial grid is constructed according to the corresponding orientation information of the spatial grid in the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set; then a mapping relationship between the point cloud data and the grid index code corresponding to each spatial grid in the multi-layer spatial grid set is established, and the point cloud data is rearranged and stored in a target storage area based on the mapping relationship; if a retrieval request for point cloud data is received, the grid index code to be rendered is determined, and the point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the grid index code to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered. The above method does not rely on hardware configuration, and through three-dimensional spatial division, grid index code construction, establishment of a mapping relationship between point cloud and grid index code, and rearrangement and storage of point cloud, it can support rapid retrieval and real-time rendering of point cloud data with unlimited data volume, greatly improving the efficiency of point cloud data retrieval and rendering.

[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a spatial retrieval method supporting massive point cloud data provided by an embodiment of the present invention; Figure 2 An overall flow chart of a spatial retrieval method supporting massive point cloud data provided by an embodiment of the present invention; Figure 3 An example diagram of a grid index encoding provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a spatial retrieval device supporting massive point cloud data provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] At present, the existing technology is difficult to meet the needs of massive point cloud visualization. Based on this, the present invention implements a spatial retrieval method, device, equipment and medium that supports massive point cloud data. It does not rely on hardware configuration, and performs spatial stereoscopic division of massive point cloud data. It also realizes rapid spatial retrieval and real-time rendering of massive point cloud data of unlimited data volume through steps such as spatial index encoding of spatial grids and determining the index encoding of grids to be rendered based on window spatial data.

[0021] To facilitate understanding of this embodiment, firstly, a spatial retrieval method supporting massive point cloud data disclosed in an embodiment of the present invention is described in detail. Figure 1 The flowchart of a spatial retrieval method supporting massive point cloud data is shown in FIG. 1 , and the method mainly includes the following steps S102 to S110: Step S102, three-dimensionally divide the space occupied by the point cloud data to be processed into a multi-layer spatial grid set.

[0022] Each layer of the multi-layer spatial grid set includes multiple spatial grids, the first layer of the spatial grid set includes a point cloud bounding box, the second layer of the spatial grid set includes multiple spatial grids obtained by three-dimensional division of the point cloud bounding box in the first layer of the spatial grid set, the third layer of the spatial grid set includes multiple spatial grids obtained by three-dimensional division of each spatial grid in the second layer of the spatial grid set, and so on. Furthermore, the number of layers of the multi-layer spatial grid set is positively correlated with the scale of the point cloud data.

[0023] In one example, the point cloud layering depth is first determined according to the scale of the point cloud data, and then the point cloud bounding box of the space occupied by the point cloud data is divided layer by layer until the point cloud layering depth is reached, and the three-dimensional division can be stopped to obtain a multi-layer spatial grid set.

[0024] Step S104: for any spatial grid in the multi-layer spatial grid set, a unique grid index code of the spatial grid is constructed according to the corresponding position information of the spatial grid in the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set.

[0025] Among them, the grid index code is composed of multiple layers of orientation codes. For the grid index code of any spatial grid, the first layer orientation code is used to represent the orientation information of the spatial grid in the first layer spatial grid set, and the second layer orientation code is used to represent the orientation information of the spatial grid in the second layer spatial grid set, and so on. The orientation information of the spatial grid in different layers of spatial grid sets is recorded through grid index coding.

[0026] In one example, the azimuth codes of multiple spatial grids belonging to the same upper spatial grid can be determined according to the azimuth information, and the grid index codes of the upper spatial grid can be combined to obtain the grid index codes of the multiple spatial grids. Among them, the multiple spatial grids belonging to the same upper spatial grid, that is, these spatial grids are obtained by three-dimensional division of the same spatial grid in the upper spatial grid set.

[0027] Step S106, establishing a mapping relationship between the point cloud data and the grid index code corresponding to each spatial grid in the multi-layer spatial grid set.

[0028] In one example, the number of points allocated to each layer of spatial grid sets (that is, the number of points to be allocated) is first determined, and then for each layer of spatial grid sets, point cloud data is collected according to the number of points to be allocated to determine the spatial grid set to which each point in the point cloud data is allocated. Finally, the target spatial grid into which each point in the allocated spatial grid set ultimately falls is determined based on the coordinate information of the point, and a mapping relationship between the point and the grid index code of the target spatial grid into which it falls is established.

[0029] Step S108: rearrange the point cloud data based on the mapping relationship and store it in the target storage area.

[0030] The target storage area may be a virtual storage area or a local storage area, such as a designated disk as the target storage area. In one example, the arrangement order of the point cloud data may be determined based on the mapping relationship and the hierarchical block order of the spatial grid, and then the point cloud data may be rearranged and stored in the target storage area according to the arrangement order.

[0031] Step S110, if a search request for point cloud data is received, the index code of the grid to be rendered is determined, and the point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, so as to realize the search and real-time rendering of the point cloud data to be rendered.

[0032] Among them, the retrieval request can carry the front and back plane information of the viewing frustum and the camera information. The front and back plane information of the viewing frustum and the camera information can be used to determine the coverage range of the viewing frustum, and the coverage range of the viewing frustum is also the display range of the retrieval request; the alternative grid index codes can be roughly screened out according to the vertex coordinates of the coverage range of the viewing frustum; the spatial grid coverage range corresponding to the viewing frustum and the alternative grid index code is intersected to determine the grid index code to be rendered within the display range; the point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the grid index code to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered.

[0033] The spatial retrieval method supporting massive point cloud data provided by the embodiment of the present invention does not rely on hardware configuration. Through spatial stereo division, grid index coding construction, establishment of a mapping relationship between point cloud and grid index coding, and point cloud rearrangement and storage, it can support rapid retrieval and real-time rendering of point cloud data of unlimited data volume, thereby greatly improving the efficiency of point cloud data retrieval and rendering.

[0034] For ease of understanding, the present invention provides a specific implementation of a spatial retrieval method supporting massive point cloud data, see Figure 2The overall flow chart of a spatial retrieval method supporting massive point cloud data is shown, which involves the steps of spatial stereo division based on octree, spatial index coding of stereo grid, establishing a mapping relationship between point cloud data and spatial network, rearranging point cloud data according to the spatial grid mapping relationship, determining the index coding of the grid to be rendered according to the window space, and realizing the retrieval and real-time rendering of point cloud data based on the index coding of the grid to be rendered. Specifically: (1) Spatial stereoscopic division based on the octree principle: The space occupied by the point cloud data is divided into layers / blocks based on the octree principle. This mainly includes the following (1.1) to (1.3): (1.1) The space occupied by the point cloud data to be processed is divided into multiple subspaces, and the subspaces with the number of points greater than a preset threshold are subdivided until the number of points contained in each subspace is less than the preset threshold, thereby obtaining the point cloud layering depth corresponding to the point cloud data.

[0035] In one example, starting from the root node, the space where the point cloud data is located is divided into eight subspaces. For each subspace, the number of points contained therein is calculated. If the number of points contained in a subspace exceeds the preset threshold, the subspace is continued to be subdivided to increase the layer depth; conversely, if the number of points contained in a subspace is lower than the preset threshold, the subdivision of the subspace can be stopped. Repeat the above steps until all subspaces meet the conditions for stopping subdivision, and the final octree depth is the point cloud layer depth. Among them, the condition for stopping subdivision is that the number of points contained in the subspace is lower than the preset threshold.

[0036] (1.2) Determine the point cloud bounding box corresponding to the space occupied by the point cloud data.

[0037] In one example, the edges of the axial bounding box are parallel to the coordinate axes, which is a simple and direct bounding box representation method. The point cloud bounding box and its coordinate information are determined by finding the minimum and maximum values ​​of the point cloud data on each coordinate axis.

[0038] (1.3) Take the point cloud bounding box as the first-layer spatial grid set, and continue to perform three-dimensional division on each spatial grid in the first-layer spatial grid set to obtain the second-layer spatial grid set, until the point cloud layering depth is met and the three-dimensional division is stopped to obtain a multi-layer spatial grid set.

[0039] The spatial grid can be a cube. In one example, the cube marked by the point cloud bounding box coordinates is marked as the first-layer cube (i.e., the first-layer spatial grid set); the first-layer cube is divided into 8 small cubes as the second-layer cube (i.e., the second-layer spatial grid set), and so on, each layer of small cubes is further divided into 8 small cubes as the next layer of cubes, until the point cloud layering depth is met, and all cube coordinates are obtained.

[0040] (2) Three-dimensional grid spatial index coding: Construct a unique grid index coding for the multi-layer spatial grid set divided by the octree.

[0041] The encoding rules of spatial index encoding are as follows: The generation of spatial grids is divided according to the properties of the octree. Each spatial grid will be refined into 8 spatial grids of the next level, and each spatial grid will be numbered 0 to 8 according to the direction. The 8 spatial grids are divided into two layers, the upper layer is numbered clockwise, corresponding to 0 to 3, and the lower layer is numbered clockwise, corresponding to 4 to 8. The maximum encoding value is 8, corresponding to the binary code 111, which can be represented by 3 bits. According to this rule, three bits are used to represent the direction code, and the numbers 0 to 8 are represented in binary, resulting in 8 numbers: 000, 001, 010, 011, 100, 101, 110, 111, each uniquely corresponding to a direction.

[0042] The embodiment of the present invention uses a 32-bit integer as a carrier, and stores an orientation code in groups of 3 bits from left to right. The first-layer orientation code is used to characterize the orientation information of the spatial grid in the first-layer spatial grid set, and the second-layer orientation code is used to characterize the orientation information of the spatial grid in the second-layer spatial grid set, and so on. The orientation information of the spatial grid in different layers is recorded to generate a unique grid index code. The grid index code ensures a one-to-one mapping from the spatial grid to the grid index code. A spatial grid must correspond to a unique grid index code. At the same time, the spatial position of the grid can be quickly calculated based on the grid index code. The numerical values ​​of spatially adjacent grid index codes are also adjacent, which provides a guarantee for efficient query and positioning of subsequent batch data.

[0043] Based on the above encoding rules, the embodiment of the present invention provides a specific implementation method for constructing a unique grid index encoding of a spatial grid, which mainly includes the following (2.1) to (2.3): (2.1) For any spatial grid in a multi-layer spatial grid set, the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set is recorded as the parent spatial grid, the spatial grid is recorded as the first child spatial grid, and the other spatial grids obtained by three-dimensionally dividing the parent spatial grid are recorded as the second child spatial grids.

[0044] For example, see Figure 3 In the example diagram of a grid index encoding shown in FIG, taking the third-layer spatial grid set as an example, it is assumed that a spatial grid in the second-layer spatial grid set (recorded as the parent spatial grid) is three-dimensionally divided to obtain 8 spatial grids in the last level, and any spatial grid among the 8 spatial grids is recorded as the first-child spatial grid, and the remaining 7 spatial grids among the 8 spatial grids are all recorded as the second-child spatial grids.

[0045] (2.2) According to the relative position relationship between the first sub-level spatial grid and each second sub-level spatial grid, the corresponding position information of the first sub-level spatial grid in the parent-level spatial grid is determined, and each position information is pre-configured with a corresponding position code.

[0046] Please continue to see Figure 3 By determining the relative position relationship between the first sub-level space grid and each second sub-level space grid, the orientation information of the first sub-level space grid is determined, and then the orientation code 110 uniquely corresponding to the orientation information is determined.

[0047] (2.3) The grid index code of the parent spatial grid is combined with the orientation code corresponding to the first child spatial grid to obtain the grid index code of the child spatial grid.

[0048] To reduce the amount of computation required for encoding, the grid index code of the parent spatial grid can be directly combined with the orientation code corresponding to the first child spatial grid. Figure 3 , the position code 110 of the first sub-level spatial grid is combined with the grid index code 001-010 of the parent spatial grid to obtain the grid index code 001-010-110 corresponding to the first sub-level spatial grid. Similarly, the grid index code corresponding to each spatial grid in the subsequent levels can be gradually determined.

[0049] In the embodiment of the present invention, every three bits express an orientation code, and a 32-bit integer can express a set of 10 spatial grids at most, which can be used to encode a total of 8 to the power of 10 spatial grids, more than 1 billion spatial grids, which is sufficient to support business needs.

[0050] (3) Establishing a mapping relationship between point cloud data and spatial network, mainly including the following (3.1) to (3.3): (3.1) According to the total number of points contained in the point cloud data, determine the number of points to be allocated corresponding to each layer of the multi-layer spatial grid set.

[0051] In one example, the hierarchical index E is selected according to the data size and distribution characteristics, and totalNum is the total number of points contained in the point cloud data. The number of points to be allocated corresponding to each layer of the spatial grid set can be obtained according to the following equation: : * totalNum.

[0052] (3.2) The point cloud data is sampled according to the number of points to be assigned to determine the layer number information corresponding to each point in the point cloud data. The layer number information is used to describe the layer number of the spatial grid set to which the point belongs.

[0053] In one example, for each layer of spatial grid sets, a sampling method is used to uniformly extract point cloud data, ensuring that the number of points contained in each layer of spatial grid sets matches the number of points to be allocated in that layer, and that the point cloud data is evenly distributed, and details are preserved. In an embodiment of the present invention, by uniformly extracting point cloud data for each layer of spatial grid sets, the spatial grid set to which each point in the point cloud data belongs can be determined, and then the number of layers of the spatial grid set to which it belongs can be used as the belonging layer number information.

[0054] Furthermore, the sampling method adopts random sampling, and the random sampling process uses a random generator based on the Mersenne Twister algorithm to improve the quality of random number generation and solve the problem of over-dense or over-sparse data distribution caused by insufficient randomness provided by traditional pseudo-random sequences. Among them, random numbers are used to represent the identification of points in point cloud data. For example, for the first-layer spatial grid set, the corresponding number of points to be assigned is X, and then a random generator is used to generate X random numbers, and it is determined that the points corresponding to the identification represented by these X random numbers belong to the first-layer spatial grid set, that is, the belonging layer information corresponding to the points corresponding to the identification represented by these X random numbers is not "1".

[0055] (3.3) For any target point in the point cloud data, determine the target spatial grid in which the target point falls in the set of spatial grids to which it belongs, based on the coordinate information and the number of layers corresponding to the target point, so as to establish a mapping relationship between the target point and the grid index code of the target spatial grid. For details, see (I) to (III) below: (I) Determine the logical side length of the spatial grid corresponding to the spatial grid set to which the target point belongs according to the information of the number of layers corresponding to the target point.

[0056] For example, if the multi-layer spatial grid set includes 4 layers in total, and the belonging layer number information corresponding to the target point is 3, then it is necessary to determine the logical side length of the spatial grid corresponding to the third layer of the spatial grid set.

[0057] The embodiment of the present invention provides a specific implementation method for determining the logical side length of a spatial grid: The logical side length interval of the spatial grid is calculated according to the target's belonging layer information. The maximum number of layers to be allocated (that is, the point cloud layer depth, 4 layers) is recorded as maxLevel. If the target point is sampled and assigned to the layer level, it is calculated according to the following bit operation formula: .

[0058] (II) For each layer of spatial grid sets between the first layer of spatial grid sets and the spatial grid set to which the target point belongs, the coordinate information corresponding to the target point is normalized according to the coordinate information of the point cloud bounding box corresponding to the space occupied by the point cloud data and the logical side length of the spatial grid, so as to determine the orientation information corresponding to the target point in the spatial grid set of this layer based on the normalized coordinate information.

[0059] Specifically, the coordinate information of the target point is represented as Pointer, the point cloud bounding box is boundary, and the logical side length of the spatial grid is interval. The point coordinates are normalized according to the following formula.

[0060] ; ; .

[0061] For example, calculate the interval0 of the first layer and then obtain the normalized coordinate information (ux, uy, uz) of the target point in the second layer. After normalization, obtain the azimuth code of the target point in the first layer. Assume that the normalized coordinate information of the target point in the first layer is: ux=1.0, uy=0.2, uz=0.8, and round off the coordinate information in the order of zyz to approximately obtain the value 101, which is the azimuth code of the target point in the first layer; continue to calculate the azimuth code of the target point in the second layer. Assume that the normalized coordinate information of the target point in the second layer is: ux=0.1, uy=0.2, uz=0.1, and round off the coordinate information in the order of zyz to approximately obtain the value 000, which is the azimuth code of the target point in the second layer; similarly, assume that the normalized coordinate information of the target point in the third layer is: ux=0.1, uy=0.2, uz=1.2, and the approximate azimuth code is 001.

[0062] (III) According to the corresponding orientation information of the target point in each layer of the spatial grid set, determine the target spatial grid in which the target point falls in the spatial grid set to which it belongs. In the specific implementation, the orientation code of the target point in each layer is obtained respectively, and finally the grid index code is obtained according to the method of three-dimensional grid spatial index coding, for example, "101-000-001", which means that the target spatial grid in which the target point falls in the spatial grid set to which it belongs is determined.

[0063] (4) Rearrange the point cloud data according to the spatial grid mapping relationship: Determine the arrangement order of the point cloud data based on the mapping relationship and the hierarchical block order of the spatial grid, and rearrange the point cloud data according to the arrangement order and store it in the target storage area. This mainly includes the following (4.1) to (4.2): (4.1) Rearrangement of point cloud Las files: The original point cloud Las file point data is usually stored in the order of scanning by laser radar and other equipment, without special design for reading and writing layers and blocks. In order to make full use of the sequential reading characteristics of mechanical hard disks, the point cloud data is stored in order according to spatial layered and block grids, which can reduce the movement of the mechanical hard disk head, facilitate batch reading of data according to layers and blocks, and speed up the extraction and analysis of point cloud data.

[0064] (4.2) Disk index file generation: record the disk index of the data corresponding to the grid index code and write it into the file index file. For example, the index file format is: |grid index code|file offset|data unit length|.

[0065] (5) Determine the index code of the mesh to be rendered according to the viewport space, which mainly includes the following (5.1) to (5.3): (5.1) If a search request for point cloud data is received, the viewing frustum coverage range is determined based on the front and rear plane information of the viewing frustum and the camera information carried in the search request.

[0066] The viewing frustum is the area where all objects that the camera may see are located. It is a four-sided pyramid shape, where the near plane corresponds to the smaller side of the pyramid and the far plane corresponds to the larger side of the pyramid. The coverage of the viewing frustum can be calculated by the following (a) to (e): (a) Determine the planes of the viewing frustum: The viewing frustum has six faces, including four side faces (left, right, top, and bottom) and two end faces (near and far planes). These planes can be calculated using the projection matrix and the view matrix.

[0067] (b) Calculate the plane equation: The equation of each plane can be calculated from the elements of the Model-View-Projection Matrix (MVP matrix). For each plane, the equation is of the form Ax + By + Cz + D = 0, where A, B, C, and D are the coefficients of the plane equation.

[0068] (c) Determine the near and far planes: The near and far planes are the two end surfaces of the viewing frustum, which define the closest and farthest distances that the camera can see. The near plane is usually closer to the camera, while the far plane is farther away from the camera.

[0069] (d) Calculate the initial frustum footprint: The frustum footprint can be calculated by determining the vertices of the frustum, which are the intersection points of the near plane and the far plane with the four sides of the frustum. Through these vertices, the shape and size of the frustum in three-dimensional space can be determined, that is, the frustum footprint.

[0070] (e) View frustum culling: During rendering, objects that are not within the view frustum can be culled to improve rendering efficiency. This involves determining whether the object intersects the view frustum and whether it is completely outside the view frustum.

[0071] (5.2) Determine an alternative grid index code according to the coordinates of multiple vertices in the coverage range of the viewing frustum, and generate a spatial grid coverage range based on the alternative grid index code.

[0072] In one example, a hexahedral model is established according to the coverage range of the viewing frustum, and the coordinates of a total of 8 vertices on the front and back planes are obtained. The grid index code of the spatial grid to which the vertex falls is determined according to the vertex coordinates and the information of the number of layers (for details, please refer to the above (3), which will not be described in detail in the embodiment of the present invention). The maximum and minimum values ​​of the grid index codes corresponding to the vertices are used to define the coverage range of the point space grid, so as to reduce the amount of calculation and improve the search efficiency.

[0073] (5.3) Based on the intersection relationship between the viewing frustum coverage range and the spatial grid coverage range, determine the index code of the grid to be rendered within the viewing frustum coverage range.

[0074] (6) Retrieval and real-time rendering of point cloud data are achieved based on the index encoding of the grid to be rendered, mainly through the following (6.1) to (6.4): (6.1) Load the disk index file.

[0075] (6.2) Based on the mesh index to be rendered, find the required data offset and data block length in the disk index file.

[0076] (6.3) Extract binary data blocks from the rearranged point cloud data based on the data offset and data block length, and use the point cloud processing driver module to process and restore the coordinates and other attribute information of the point cloud data.

[0077] (6.4) Use real-time viewing frustum data to drive and extract point cloud data from disk on demand, get rid of the limitation of computer memory, and realize the retrieval and real-time rendering of massive point cloud data.

[0078] In summary, the embodiment of the present invention proposes a new three-dimensional network space retrieval coding technology based on the spatial stereoscopic division of the octree, and establishes a mapping relationship between point cloud data and the spatial network according to the characteristics of the new technology, and then rearranges the point cloud data according to the mapping relationship, calculates the index code of the grid to be rendered according to the window space, and realizes the retrieval and real-time rendering of the point cloud data based on the index code of the grid to be rendered. This method can support the rapid retrieval and real-time rendering of point cloud data with unlimited data volume, greatly improving the efficiency of point cloud data retrieval and rendering.

[0079] In order to verify the effect of the embodiment of the present invention, more than 700 million laser point cloud data of Xiaolangdi area, with a data volume of 23GB, were used as test data. The performance of the test computer is: Intel (R) core (TM) I7-10700 CPU @2.9Ghz2.9Ghz, memory is 64GB, and graphics card is NVIDIA Quadro P1000. The first group of tests directly read more than 700 million laser point cloud data into memory without processing such as block layering of the point cloud. After 30 minutes, the point cloud still could not render the point cloud data. The second group of tests adopted this method. The test showed that the series of pre-processing such as block layering, indexing, etc. of the more than 700 million laser point cloud data of Xiaolangdi took 20 minutes according to this method, and it took 18 seconds to import the software for rendering. The building point cloud editing can realize the rapid retrieval of the building point cloud ID, and the real-time update of the point cloud data attributes and rendering.

[0080] After experiments, it was found that the method for rapid spatial retrieval and real-time rendering of massive point cloud data proposed in the embodiment of the present invention can realize rapid retrieval and real-time rendering of point cloud data of big data, which is of great significance in the subsequent fine classification and manual editing of point cloud data and three-dimensional modeling of point cloud data.

[0081] Based on the above embodiments, the present invention provides a spatial retrieval device that supports massive point cloud data. Figure 4 The schematic diagram of the structure of a spatial retrieval device supporting massive point cloud data is shown in FIG. 1 , and the device mainly includes the following parts: A three-dimensional division module 402 is used to perform three-dimensional division on the space occupied by the point cloud data to be processed to obtain a multi-layer space grid set; The coding construction module 404 is used to construct a unique grid index code of any spatial grid in the multi-layer spatial grid set according to the corresponding position information of the spatial grid in the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set; A mapping establishment module 406 is used to establish a mapping relationship between the point cloud data and the grid index code corresponding to each spatial grid in the multi-layer spatial grid set; A rearrangement and storage module 408 is used to rearrange the point cloud data according to the mapping relationship and store it in a target storage area; The point cloud retrieval module 410 is used to determine the index code of the grid to be rendered if a retrieval request for point cloud data is received, and retrieve the point cloud data to be rendered from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered.

[0082] The spatial retrieval device supporting massive point cloud data provided by the embodiment of the present invention does not rely on hardware configuration. Through spatial stereo division, grid index coding construction, establishment of a mapping relationship between point cloud and grid index coding, and point cloud rearrangement and storage, it can support rapid retrieval and real-time rendering of point cloud data of unlimited data volume, thereby greatly improving the efficiency of point cloud data retrieval and rendering.

[0083] In one implementation, the three-dimensional division module 402 is specifically used to: The space occupied by the point cloud data to be processed is divided into multiple subspaces, and the subspaces with the number of points greater than a preset threshold are subdivided until the number of points contained in each subspace is less than the preset threshold, and the point cloud layer depth corresponding to the point cloud data is obtained; Determine the point cloud bounding box corresponding to the space occupied by the point cloud data; The point cloud bounding box is used as the first layer of spatial grid set, and each spatial grid in the first layer of spatial grid set is continuously divided into three dimensions to obtain the second layer of spatial grid set, until the point cloud layering depth is met and the three-dimensional division is stopped to obtain a multi-layer spatial grid set.

[0084] In one implementation, the encoding construction module 404 is specifically used to: For any spatial grid in the multi-layer spatial grid set, the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set is recorded as the parent spatial grid, the spatial grid is recorded as the first child spatial grid, and the other spatial grids obtained by three-dimensionally dividing the parent spatial grid are recorded as the second child spatial grids; Determine the position information corresponding to the first sub-level spatial grid in the parent-level spatial grid according to the relative position relationship between the first sub-level spatial grid and each second sub-level spatial grid, each position information is pre-configured with a corresponding position code; The grid index code of the parent spatial grid is combined with the orientation code corresponding to the first child spatial grid to obtain the grid index code of the child spatial grid.

[0085] In one implementation, the mapping establishment module 406 is specifically configured to: According to the total number of points contained in the point cloud data, determine the number of points to be allocated corresponding to each layer of the spatial grid set in the multi-layer spatial grid set; The point cloud data is sampled according to the number of points to be assigned, so as to determine the belonging layer number information corresponding to each point in the point cloud data, and the belonging layer number information is used to describe the number of layers of the spatial grid set to which the point belongs; For any target point in the point cloud data, the target spatial grid into which the target point falls in the set of spatial grids to which it belongs is determined according to the coordinate information and the number of layers corresponding to the target point, so as to establish a mapping relationship between the target point and the grid index code of the target spatial grid.

[0086] In one implementation, the mapping establishment module 406 is specifically configured to: According to the belonging layer information corresponding to the target point, determine the logical side length of the spatial grid corresponding to the spatial grid set to which the target point belongs; For each layer of spatial grid sets between the first layer of spatial grid sets and the spatial grid sets to which the target point belongs, the coordinate information corresponding to the target point is normalized according to the coordinate information of the point cloud bounding box corresponding to the space occupied by the point cloud data and the logical side length of the spatial grid, so as to determine the orientation information corresponding to the target point in the spatial grid sets of this layer based on the normalized coordinate information; According to the corresponding position information of the target point in each layer of the spatial grid set, the target spatial grid into which the target point falls in the spatial grid set to which it belongs is determined.

[0087] In one implementation, the rearrangement and storage module 408 is specifically configured to: The arrangement order of the point cloud data is determined based on the mapping relationship and the hierarchical and blocking order of the spatial grid, and the point cloud data is rearranged and stored in the target storage area according to the arrangement order.

[0088] In one implementation, the point cloud retrieval module 410 is specifically used to: If a search request for point cloud data is received, the coverage range of the viewing frustum is determined based on the front and rear plane information of the viewing frustum and the camera information carried in the search request; Determine candidate grid index codes according to multiple vertex coordinates of a viewing frustum coverage range, and generate a spatial grid coverage range based on the candidate grid index codes; Based on the intersection relationship between the viewing frustum coverage range and the spatial grid coverage range, the index code of the grid to be rendered within the viewing frustum coverage range is determined.

[0089] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0090] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.

[0091] Figure 5A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 50, a memory 51, a bus 52 and a communication interface 53, wherein the processor 50, the communication interface 53 and the memory 51 are connected via the bus 52; the processor 50 is used to execute an executable module stored in the memory 51, such as a computer program.

[0092] The memory 51 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0093] The bus 52 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0094] Among them, the memory 51 is used to store programs, and the processor 50 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 50 or implemented by the processor 50.

[0095] The processor 50 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 50. The above processor 50 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 51, and the processor 50 reads the information in the memory 51 and completes the steps of the above method in combination with its hardware.

[0096] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0097] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0098] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A spatial retrieval method supporting massive point cloud data, characterized in that: include: The space occupied by the point cloud data to be processed is divided into three dimensions to obtain a multi-layer spatial grid set; For any of the spatial grids in the multi-layer spatial grid set, constructing a unique grid index code of the spatial grid according to the corresponding position information of the spatial grid in the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set; Establishing a mapping relationship between the point cloud data and a grid index code corresponding to each of the spatial grids in the multi-layer spatial grid set; Rearranging the point cloud data based on the mapping relationship and storing it in a target storage area; If a retrieval request for the point cloud data is received, the index code of the grid to be rendered is determined, and the point cloud data to be rendered is retrieved from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered.

2. The spatial retrieval method supporting massive point cloud data according to claim 1, characterized in that: The space occupied by the point cloud data to be processed is divided into three dimensions to obtain a multi-layer spatial grid set, including: Divide the space occupied by the point cloud data to be processed into multiple subspaces, and subdivide the subspaces whose number of points is greater than a preset threshold until the number of points contained in each subspace is less than the preset threshold, thereby obtaining the point cloud layering depth corresponding to the point cloud data; Determine a point cloud bounding box corresponding to the space occupied by the point cloud data; The point cloud bounding box is used as the first layer spatial grid set, and each spatial grid in the first layer spatial grid set is continuously divided into three dimensions to obtain a second layer spatial grid set, and the three-dimensional division is stopped when the point cloud layering depth is met to obtain a multi-layer spatial grid set.

3. The spatial retrieval method supporting massive point cloud data according to claim 1, characterized in that: For any of the spatial grids in the multi-layer spatial grid set, constructing a unique grid index code of the spatial grid according to the corresponding position information of the spatial grid in the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set, including: For any of the spatial grids in the multi-layer spatial grid set, the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set is recorded as a parent spatial grid, the spatial grid is recorded as a first child spatial grid, and other spatial grids obtained by three-dimensionally dividing the parent spatial grid are recorded as second child spatial grids; Determine, according to the relative position relationship between the first sub-level spatial grid and each of the second sub-level spatial grids, the position information corresponding to the first sub-level spatial grid in the parent spatial grid, each of the positions being pre-configured with a corresponding position code; The grid index code of the parent spatial grid is combined with the orientation code corresponding to the first child spatial grid to obtain the grid index code of the child spatial grid.

4. The spatial retrieval method supporting massive point cloud data according to claim 1, characterized in that: Establishing a mapping relationship between the point cloud data and a grid index code corresponding to each of the spatial grids in the multi-layer spatial grid set includes: Determine the number of points to be allocated corresponding to each layer of the spatial grid set in the multi-layer spatial grid set according to the total number of points included in the point cloud data; Sampling the point cloud data according to the number of points to be allocated to determine the belonging layer number information corresponding to each point in the point cloud data, wherein the belonging layer number information is used to describe the number of layers of the spatial grid set to which the point belongs; For any target point in the point cloud data, the target spatial grid into which the target point falls in the spatial grid set to which it belongs is determined based on the coordinate information corresponding to the target point and the belonging layer number information, so as to establish a mapping relationship between the target point and the grid index code of the target spatial grid.

5. The spatial retrieval method supporting massive point cloud data according to claim 4, characterized in that: Determining the target spatial grid into which the target point falls in the set of spatial grids to which the target point belongs according to the coordinate information corresponding to the target point and the belonging layer number information, including: Determine, according to the belonging layer number information corresponding to the target point, the logical side length of the spatial grid corresponding to the spatial grid set to which the target point belongs; For each layer of spatial grid sets between the first layer of spatial grid sets and the spatial grid set to which the target point belongs, normalizing the coordinate information corresponding to the target point according to the coordinate information of the point cloud bounding box corresponding to the space occupied by the point cloud data and the logical side length of the spatial grid, so as to determine the orientation information corresponding to the target point in the layer of spatial grid sets based on the normalized coordinate information; According to the position information corresponding to the target point in each layer of the spatial grid set, the target spatial grid into which the target point falls in the spatial grid set to which it belongs is determined.

6. The spatial retrieval method supporting massive point cloud data according to claim 1, characterized in that: Rearranging the point cloud data based on the mapping relationship and storing it in a target storage area includes: The arrangement order of the point cloud data is determined based on the mapping relationship and the hierarchical and blocking order of the spatial grid, and the point cloud data is rearranged and stored in a target storage area according to the arrangement order.

7. The spatial retrieval method supporting massive point cloud data according to claim 1, characterized in that: If a search request for the point cloud data is received, determining the index code of the grid to be rendered includes: If a search request for the point cloud data is received, determining a viewing frustum coverage range based on the viewing frustum front and rear plane information and camera information carried in the search request; Determine candidate grid index codes according to a plurality of vertex coordinates of the viewing frustum coverage range, and generate a spatial grid coverage range based on the candidate grid index codes; Based on the intersection relationship between the viewing frustum coverage range and the spatial grid coverage range, the index code of the grid to be rendered within the viewing frustum coverage range is determined.

8. A spatial retrieval device supporting massive point cloud data, characterized in that: include: A three-dimensional division module is used to perform three-dimensional division on the space occupied by the point cloud data to be processed to obtain a multi-layer spatial grid set; A coding construction module, configured to construct, for any of the spatial grids in the multi-layer spatial grid set, a unique grid index code of the spatial grid according to the corresponding position information of the spatial grid in the spatial grid to which the spatial grid belongs in the previous layer of spatial grid set; A mapping establishment module, used to establish a mapping relationship between the point cloud data and the grid index code corresponding to each of the spatial grids in the multi-layer spatial grid set; A rearrangement and storage module, used for rearranging the point cloud data according to the mapping relationship and storing it in a target storage area; The point cloud retrieval module is used to determine the index code of the grid to be rendered if a retrieval request for the point cloud data is received, and retrieve the point cloud data to be rendered from the rearranged point cloud data stored in the target storage area according to the index code of the grid to be rendered, so as to realize the retrieval and real-time rendering of the point cloud data to be rendered.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.

Images