A vectorization skeleton extraction method and system for indoor scene point cloud data

Abstract

The application discloses a kind of vectorization skeleton extraction method and system of indoor scene point cloud data, and the method steps include: S1. obtaining point cloud data, depth map data and different direction visible light image data under indoor environment;S2. respectively to point cloud data and visible light image data are carried out edge point extraction, and depth interval division is carried out to depth map data and constructs depth distribution mask;S3. according to depth distribution mask and point cloud edge point set constructs sparse graph structure to extract out point cloud sparse vectorization skeleton, and according to depth distribution mask and visible light image edge point set constructs dense graph structure to extract out dense vectorization skeleton;S4. point cloud sparse vectorization skeleton, dense vectorization skeleton fusion is formed to build final complete vectorization skeleton.The application has the advantages of simple realization method, high cost, high extraction accuracy and efficiency and strong robustness.

Description

Technical Field

[0001] This invention relates to, and more particularly to, a method and system for vectorized skeleton extraction of point cloud data of indoor scenes. Background Technology

[0002] In point cloud simplification tasks, it is necessary to extract the skeleton architecture of dense point clouds to serve as a lightweight intermediate modality for understanding, rendering, and other understanding and manipulation tasks. To achieve point cloud skeleton extraction, existing technologies typically employ the following two methods:

[0003] 1) Semantic-based point cloud skeletonization method: This method upsamples the point cloud, i.e., it takes an input point cloud and outputs a denser point cloud that lies on the implicit geometry (such as a surface) of the input point cloud. In other words, the semantic-based point cloud skeletonization method learns features at multiple granularities (from local to global) for each point, then reduces the point set in the feature space, and finally maps the reduced point set back to 3D to achieve point cloud skeletonization.

[0004] 2) Geometry-based point cloud skeletonization method: By extracting the edge features of points or using the k-nn method, points with similar features or located in the same region are aggregated into a single point to form a sparse point cloud skeleton.

[0005] However, both the semantic-based and geometry-based point cloud skeletonization methods mentioned above rely on complex object modeling processes and have high prior requirements for semantics or the geometry of the model itself. That is, they must obtain prior information such as semantics or the geometry of the model beforehand. In practical applications, point cloud acquisition time is usually short and the environment is limited, making it difficult to obtain complete and accurate prior information on semantics or geometry. This results in a lack of accuracy and robustness when densifying point clouds for modeling. Furthermore, the amount of point cloud data acquired in indoor scenes is usually large, while computing resources are often very limited, failing to meet the computational requirements of geometric downsampling. This makes it difficult to achieve accurate and fast point cloud skeleton extraction in indoor environments. Summary of the Invention

[0006] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a method and system for vectorized skeleton extraction of indoor scene point cloud data that is simple to implement, cost-effective, accurate and efficient, and robust. It can be applied to achieve fast and accurate vectorized skeleton extraction of scene point cloud data in indoor enclosed environments.

[0007] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0008] A method for vectorized skeleton extraction from indoor scene point cloud data, comprising the following steps:

[0009] Step S1. Acquire point cloud data, depth map data, and visible light image data from different directions at the same location in a specified indoor scene;

[0010] Step S2. Extract edge points from the point cloud data and the visible light image data respectively to obtain the point cloud edge point set and the visible light image edge point set, and divide the depth map data into depth intervals and construct a depth distribution mask according to the distribution of the depth map intervals;

[0011] Step S3. Construct a sparse graph structure based on the depth distribution mask and the point cloud edge point set to extract the sparse vectorized skeleton of the point cloud, and construct a dense graph structure based on the depth distribution mask and the visible light image edge point set to extract the dense vectorized skeleton.

[0012] Step S4. Construct the final complete vectorized skeleton based on the depth distribution mask of the depth range and the sparse vectorized skeleton and dense vectorized skeleton of the point cloud.

[0013] Furthermore, in step S2, edge point extraction of point cloud data includes: calculating the distance between each point in the point cloud data and the other points; when the minimum distance between the target point and the other points is greater than a first preset threshold, the target point is marked as an edge point, and all marked edge points constitute the edge point set of the point cloud; edge point extraction of visible light image data includes: calculating the distance between the color value of each pixel in the visible light image data and the surrounding pixels; when the minimum distance between the color value of the target pixel and the surrounding pixels is greater than a second preset threshold, the target pixel is marked as an edge point, and all marked edge points constitute the edge point set of the visible light image.

[0014] Furthermore, step S3 includes: uniformly dividing the maximum depth range corresponding to the depth map data into k intervals, where the pixels in each interval belong to the same category, and constructing the corresponding depth distribution mask based on the proportion of the number of pixels in each category to the total number of pixels.

[0015] Furthermore, in step S3, constructing a sparse graph structure based on the depth distribution mask and the point cloud edge point set includes: using the vectorization of the three-dimensional spatial coordinates of each edge point in the point cloud edge point set to construct vertices in the graph structure; setting the value of each edge in the graph structure according to the size relationship between the minimum distance between the first points of the corresponding edge points and the corresponding depth distribution mask to obtain the point cloud sparse graph structure, wherein the minimum distance between the first points is the minimum distance between the current edge point and the other edge points.

[0016] Furthermore, the value of each edge in the point cloud sparse graph structure is the ratio between the minimum distance between the first points of the corresponding edge points and the corresponding depth distribution mask.

[0017] Furthermore, in step S3, constructing a dense graph structure based on the depth distribution mask and the visible light image edge point set includes: vectorizing the position coordinates of each pixel in the visible light image edge point set to represent the vertices constituting the graph structure; setting the value of each edge in the graph structure according to the relationship between the minimum distance between the second points of the corresponding edge points and the corresponding depth distribution mask, thereby obtaining a dense vectorized graph structure of the image. The minimum distance between the second points is the minimum distance between the average color value of the current edge point and multiple adjacent points in the image and the remaining points.

[0018] Furthermore, in the image dense vectorized graph structure, vertices are formed by the vectorized representation of the position coordinates of each pixel in the visible light image edge point set and its adjacent multiple points in the image data, and the value of each edge is the ratio between the distance between the second point of the corresponding edge point and the corresponding depth distribution mask.

[0019] Furthermore, step S04 includes: sparsely vectorizing the point cloud into a skeleton Mask (G p ) and dense vectorized skeleton Mask (G i According to the formula G = (Mask(G) p )+Mask(G i The closed vectorization fusion is performed to obtain the final complete vectorized skeleton G, where Mask is the depth distribution mask.

[0020] A vectorized skeleton extraction system for indoor scene point cloud data includes:

[0021] The data acquisition module is used to acquire point cloud data, depth map data, and visible light image data from different directions collected at the same location in a specified indoor scene.

[0022] The data processing module is used to extract edge points from point cloud data and visible light image data to obtain point cloud edge point sets and visible light image edge point sets, respectively, and to divide depth map data into depth intervals and construct a depth distribution mask based on the distribution of the depth map intervals.

[0023] The skeleton extraction module is used to construct a sparse graph structure based on the depth distribution mask and the point cloud edge point set to extract the sparse vectorized skeleton of the point cloud, and to construct a dense graph structure based on the depth distribution mask and the visible light image edge point set to extract the dense vectorized skeleton.

[0024] The skeleton fusion module is used to construct the final complete vector skeleton by combining the sparse vectorized skeleton and the dense vectorized skeleton of the point cloud.

[0025] A vectorized skeleton extraction system for indoor scene point cloud data includes a processor and a memory, wherein the memory is used to store a computer program and the processor is used to execute the computer program to perform the method described above.

[0026] Compared with existing technologies, the advantages of this invention are as follows: This invention targets point cloud skeleton extraction in indoor scenes. It extracts three types of data: point cloud, visible light image, and depth map. Then, it extracts edge points from the point cloud and visible light image data respectively. For the depth map data, it constructs a depth distribution mask using depth distribution. It uses a graph structure to fuse the edge points of the depth distribution mask and the point cloud and visible light image data to extract sparse point cloud skeletons and dense vectorized skeletons of the image. Then, it combines a depth interval mask to fuse the sparse vectorized skeleton and dense vectorized skeleton of the point cloud to obtain a complete skeleton. It can achieve fast skeleton extraction of high-density 3D point clouds in enclosed indoor spaces based on visible light and depth maps. At the same time, since it adopts a graph structure form of vectorized representation of point cloud data, it can also effectively reduce the computational load of the method and achieve low-power point cloud skeleton extraction. Attached Figure Description

[0027] Figure 1 This is a schematic diagram illustrating the implementation process of the vectorized skeleton extraction method for indoor scene point cloud data in this embodiment.

[0028] Figure 2 This is a schematic diagram of the structural principle of the vectorized skeleton extraction device for indoor scene point cloud data in this embodiment. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0030] Because indoor environments are enclosed and relatively small, they are ideal for acquiring depth images. This invention focuses on skeleton extraction from point cloud data in indoor scenes. It fully utilizes depth map data and visible light image data to assist in skeleton extraction. Simultaneously, it uses depth map data to extract sparse point cloud and dense image vectorized skeletons based on graph structure. The sparse vectorized skeleton provides geometric structure and distribution guidance, while the dense vectorized skeleton provides semantic guidance. This results in a complete vectorized skeleton containing geometric structure, distribution, and semantic information, effectively improving the accuracy and efficiency of indoor scene acquisition. Figure 1 As shown, the detailed steps of the vectorized skeleton extraction method for indoor scene point cloud data in this embodiment include:

[0031] Step S1. Data Acquisition: Acquire point cloud data, depth map data, and visible light image data from different directions at the same location in a specified indoor scene.

[0032] In this embodiment, the indoor scene is a closed, non-open environment.

[0033] In this embodiment, three types of data are collected at the same location in an indoor scene: point cloud data, depth map data, and visible light image data from different directions. The collected point cloud data is denoted as P, and the collected multi-view visible light data is denoted as I1, I2, ..., I... n The acquired multi-view depth map data are denoted as D1, D2, ..., D n Further, the number of points in the collected point cloud data P of the enclosed indoor scene was counted, and the data size was denoted as N×3; the resolution of the collected visible light data I was measured and denoted as H. I ×W I The resolution of the acquired depth map data D is measured and denoted as H. D ×W D .

[0034] Step S2. Data processing: Extract edge points from the point cloud data and visible light image data to obtain the edge point set of the point cloud and the edge point set of the visible light image, respectively. Divide the depth map data into depth intervals and construct a depth distribution mask based on the distribution of the depth map intervals.

[0035] In this embodiment, the specific steps for edge point extraction of point cloud data include: calculating the distance between each point in the point cloud data and the other points; when the minimum distance between the target point and the other points is greater than a first preset threshold, the target point is marked as an edge point, and all marked edge points constitute the point cloud edge point set.

[0036] In this embodiment, the specific steps for edge point extraction of visible light image data include: calculating the distance between the color value of each pixel in the visible light image data and the surrounding pixels; when the minimum distance between the color value of the target pixel and the surrounding pixels is greater than a second preset threshold, the target pixel is marked as an edge point, and all marked edge points constitute the visible light image edge point set.

[0037] Specifically, when extracting edge points from the collected point cloud data P of size N×3, the Euclidean distance between each point's (x, y, z) coordinates and the remaining N-1 points is calculated. If the minimum distance between the target point and the remaining N-1 points is greater than a threshold δ, then... P When the target point is marked as an edge point, the point cloud consisting of all edge points is denoted as P. e Its dimensions are m×3.

[0038] When extracting edge points from a panoramic image I of size H×W, the Euclidean distance between the color value (r, g, b) of each pixel and its 24 surrounding pixels is calculated. If the minimum pixel distance between the target pixel and its 24 surrounding pixels exceeds a threshold δ, the edge point is extracted. I When the target pixel is marked as an edge point, the image point set consisting of all edge pixels is denoted as I. e Its size is i×j.

[0039] In this embodiment, when processing the depth map data, the maximum depth range corresponding to the depth map data is evenly divided into k intervals, and the pixels in each interval belong to the same category. A corresponding depth distribution mask is constructed based on the proportion of the number of pixels in each category to the total number of pixels.

[0040] Specifically, by classifying the depth value distribution in the depth map, the classification is based on the depth map value corresponding to each pixel in the depth map data. The maximum depth range corresponding to the depth map is evenly divided into k intervals, and the pixels in each interval belong to the same category. Based on the proportion of the number of pixels in different intervals to the total number of pixels, a depth distribution mask is constructed, denoted as Mask.

[0041] Step S3. Construct a sparse graph structure based on the depth distribution mask and the point cloud edge point set to extract the sparse vectorized skeleton of the point cloud, and construct a dense graph structure based on the depth distribution mask and the visible light image edge point set to extract the dense vectorized skeleton.

[0042] In this embodiment, the specific steps for constructing a sparse graph structure based on the depth distribution mask and the point cloud edge point set include: using the vectorized three-dimensional spatial coordinates of each edge point in the point cloud edge point set to construct vertices in the graph structure; setting the value of each edge in the graph structure according to the relationship between the minimum distance between the first points of the corresponding edge points and the corresponding depth distribution mask, thus obtaining the point cloud sparse graph structure, where the minimum distance between the first points is the minimum distance between the current edge point and the remaining edge points. Preferably, the value of each edge in the point cloud sparse graph structure is specifically the ratio between the minimum distance between the first points of the corresponding edge points and the corresponding depth distribution mask.

[0043] In this example, the specific steps for constructing a dense graph structure based on the depth distribution mask and the visible light image edge point set include: vectorizing the position coordinates of each pixel in the visible light image edge point set to represent the vertices constituting the graph structure; setting the value of each edge in the graph structure according to the relationship between the minimum distance between the second points of the corresponding edge points and the corresponding depth distribution mask; and obtaining the dense vectorized graph structure of the image. The minimum distance between the second points is the minimum distance between the average color value of the current edge point and multiple adjacent points in the image and the remaining points.

[0044] Preferably, in the image dense vectorized graph structure, vertices are formed by the vectorized representation of the position coordinates of each pixel in the visible light image edge point set and its 25 adjacent points in the image data, and the value of each edge is the ratio between the distance between the second point of the corresponding edge point and the corresponding depth distribution mask.

[0045] Specifically, based on the extracted edge point cloud P with a size of m×3, e A graph structure G = (V, e) is constructed, where V represents vertices, each vertex being a vectorized representation of the 3D spatial coordinates (x, y, z) of each edge point in the point cloud dataset, and E represents edges. The value (weight coefficient) of each edge is the ratio of the minimum Euclidean distance between that edge point and other points in the entire point cloud dataset P to the corresponding value in the Mask. A sparse graph structure Mask(G) is constructed in this way. p The image point set I, consisting of all edge pixels of size i×j. e For image point set I e For each pixel in the image data, a graph structure G = (V, E) is constructed, where V represents a vertex, and each vertex is represented by the vectorized coordinates (x, y) of its 25 neighboring points in the original image data I. E represents an edge, and the value (weight coefficient) of each edge is the ratio of the average color value (r, g, b) of that edge point in the original image data to the minimum Euclidean distance between that edge point and the other points to the corresponding value in the Mask. In this way, a dense graph structure Mask(G) is constructed. i The dense graph structure Mask(G) constructed through the above steps is obtained. i It can extract the edge structure of visible light images in indoor scenes, providing semantic-based guidance for skeleton extraction.

[0046] Step S4. Based on the depth distribution mask of the depth range and the sparse and dense vectorized skeletons of the point cloud, the final complete vectorized skeleton is constructed.

[0047] In this embodiment, the point cloud is sparsely vectorized into a skeleton Mask(G). p ) and dense vectorized skeleton Mask (G i According to the formula G = (Mask(G) p )+Mask(G i The closed vectorization fusion is performed to obtain the final complete vectorized skeleton G, which is composed of the point cloud sparse vectorized skeleton Mask(G). p ) provides geometric structure and distribution guidance, derived from a dense vectorized skeleton Mask (G i It provides semantic guidance, which enables the final result to be a complete vectorized skeleton that includes geometric structure, distribution, and semantic information guidance.

[0048] This embodiment focuses on point cloud skeleton extraction in indoor scenes. It extracts three types of data: point cloud, visible light image, and depth map. Then, it extracts edge points from the point cloud and visible light image data, and constructs a depth distribution mask using depth distribution for the depth map data. By fusing the depth distribution mask with the edge points of the point cloud and visible light image data using a graph structure, it achieves the extraction of sparse point cloud skeletons and dense vectorized skeletons from the image. Finally, it combines the sparse vectorized skeletons and dense vectorized skeletons of the point cloud to obtain a complete skeleton. This method can achieve fast skeleton extraction of high-density 3D point clouds in enclosed indoor spaces based on visible light and depth maps, effectively improving the extraction accuracy in indoor scenes. At the same time, due to the use of a graph structure form that represents the point cloud data in vector form, it can also effectively reduce the computational load of the method and achieve low-power point cloud skeleton extraction.

[0049] like Figure 2 As shown, the vectorized skeleton extraction system for indoor scene point cloud data in this embodiment includes:

[0050] The data acquisition module is used to acquire point cloud data, depth map data, and visible light image data from different directions collected at the same location in a specified indoor scene.

[0051] The data processing module is used to extract edge points from point cloud data and visible light image data to obtain point cloud edge point sets and visible light image edge point sets, respectively, and to divide depth map data into depth intervals and construct a depth distribution mask based on the distribution of depth map intervals.

[0052] The skeleton extraction module is used to construct a sparse graph structure based on the depth distribution mask and the edge point set of the point cloud to extract the sparse vectorized skeleton of the point cloud, and to construct a dense graph structure based on the depth distribution mask and the edge point set of the visible light image to extract the dense vectorized skeleton.

[0053] The skeleton fusion module is used to construct the final complete vectorized skeleton from the sparse vectorized skeleton and the dense vectorized skeleton of the point cloud.

[0054] In this embodiment, the data processing module includes a point cloud edge point extraction unit for extracting edge points from point cloud data, an image edge point extraction unit for extracting edge points from visible light image data, and a depth map processing unit for dividing depth image data into depth intervals and constructing a depth distribution mask based on the distribution of depth map intervals. The skeleton extraction module includes a point cloud skeleton extraction unit for constructing a sparse graph structure using the depth distribution mask and point cloud edge point sets to obtain a sparse skeleton of the point cloud, and an image skeleton extraction unit for constructing a dense graph structure using the depth distribution mask and image edge point sets to obtain a dense vectorized skeleton of the image.

[0055] The vectorized skeleton extraction system for indoor scene point cloud data in this embodiment corresponds one-to-one with the vectorized skeleton extraction method for indoor scene point cloud data described above, and will not be described in detail here.

[0056] In another embodiment, the vectorized skeleton extraction system for indoor scene point cloud data of the present invention may further include a processor and a memory, wherein the memory is used to store a computer program and the processor is used to execute the computer program to perform the method described above.

[0057] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for vectorized skeleton extraction from indoor scene point cloud data, characterized in that the steps include: include: Step S1. Acquire point cloud data, depth map data, and visible light image data from different directions at the same location in a specified indoor scene; Step S2. Extract edge points from the point cloud data and the visible light image data respectively to obtain the point cloud edge point set and the visible light image edge point set, and divide the depth map data into depth intervals and construct a depth distribution mask according to the distribution of the depth map intervals; Step S3. Construct a sparse graph structure based on the depth distribution mask and the point cloud edge point set to extract the sparse vectorized skeleton of the point cloud, and construct a dense graph structure based on the depth distribution mask and the visible light image edge point set to extract the dense vectorized skeleton. Step S4. The sparse vectorized skeleton and the dense vectorized skeleton of the point cloud are fused to form the final complete vectorized skeleton.

2. The vectorized skeleton extraction method for indoor scene point cloud data according to claim 1, characterized in that, In step S2, edge point extraction of point cloud data includes: calculating the distance between each point in the point cloud data and the other points; when the minimum distance between the target point and the other points is greater than a first preset threshold, the target point is marked as an edge point, and the edge points obtained by all marking are used to form the edge point set of the point cloud; edge point extraction of visible light image data includes: calculating the distance between the color value of each pixel in the visible light image data and the surrounding pixels; when the minimum distance between the color value of the target pixel and the surrounding pixels is greater than a second preset threshold, the target pixel is marked as an edge point, and the edge points obtained by all marking are used to form the edge point set of the visible light image.

3. The method for vectorized skeleton extraction of indoor scene point cloud data according to claim 1, characterized in that, In step S2, dividing the depth map data into depth intervals and constructing a depth distribution mask based on the distribution of the depth map intervals includes: uniformly dividing the maximum depth range corresponding to the depth map data into k intervals, where the pixels in each interval belong to the same category, and constructing the corresponding depth distribution mask based on the proportion of the number of pixels in each category to the total number of pixels.

4. The method for vectorized skeleton extraction of indoor scene point cloud data according to claim 1, characterized in that, In step S3, constructing a sparse graph structure based on the depth distribution mask and the point cloud edge point set includes: using the vectorization of the three-dimensional spatial coordinates of each edge point in the point cloud edge point set to construct the vertices in the graph structure; setting the value of each edge in the graph structure according to the size relationship between the minimum distance between the first points of the corresponding edge points and the corresponding depth distribution mask to obtain the point cloud sparse graph structure, wherein the minimum distance between the first points is the minimum distance between the current edge point and the other edge points.

5. The method for vectorized skeleton extraction of indoor scene point cloud data according to claim 4, characterized in that, In the point cloud sparse graph structure, the value of each edge is the ratio between the minimum distance between the first points of the corresponding edge points and the corresponding depth distribution mask.

6. The method for vectorized skeleton extraction of indoor scene point cloud data according to any one of claims 1 to 5, characterized in that, In step S3, constructing a dense graph structure based on the depth distribution mask and the visible light image edge point set includes: vectorizing the position coordinates of each pixel in the visible light image edge point set to represent the vertices constituting the graph structure; setting the value of each edge in the graph structure according to the proportional relationship between the minimum distance between the second points of the corresponding edge points and the corresponding depth distribution mask, thereby obtaining a dense vectorized graph structure of the image. The minimum distance between the second points is the minimum distance between the average color value of multiple adjacent points of the current edge point in the image and the color value of the remaining points.

7. The method for vectorized skeleton extraction of indoor scene point cloud data according to claim 6, characterized in that, In the image dense vectorized graph structure, vertices are formed by the vectorized representation of the position coordinates of each pixel in the visible light image edge point set and its adjacent multiple points in the image data. The value of each edge is the ratio between the distance between the second point of the corresponding edge point and the corresponding depth distribution mask.

8. The method for vectorized skeleton extraction of indoor scene point cloud data according to any one of claims 1 to 5, characterized in that, Step S4 includes: sparsely vectorizing the point cloud into a skeleton. and dense vectorized skeleton According to the formula Perform closed-loop vectorization fusion to obtain the final complete vectorized skeleton. ,in The depth distribution mask.

9. A vectorized skeleton extraction system for indoor scene point cloud data, characterized in that, include: The data acquisition module is used to acquire point cloud data, depth map data, and visible light image data from different directions collected at the same location in a specified indoor scene. The data processing module is used to extract edge points from point cloud data and visible light image data to obtain point cloud edge point sets and visible light image edge point sets, respectively, and to divide depth map data into depth intervals and construct a depth distribution mask based on the distribution of the depth map intervals. The skeleton extraction module is used to construct a sparse graph structure based on the depth distribution mask and the point cloud edge point set to extract the sparse vectorized skeleton of the point cloud, and to construct a dense graph structure based on the depth distribution mask and the visible light image edge point set to extract the dense vectorized skeleton. The skeleton fusion module is used to construct the final complete vector skeleton by combining the sparse vectorized skeleton and the dense vectorized skeleton of the point cloud.

10. A vectorized skeleton extraction system for indoor scene point cloud data, comprising a processor and a memory, wherein the memory is used to store computer programs, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 8.

Images