Three-dimensional model completion and restoration method and system based on point cloud vectorization skeleton matching

Abstract

The application discloses a kind of based on point cloud vectorization skeleton matching three-dimensional model completion repair method and system, the method steps include: S01. original point cloud data and panoramic image data are acquired and respectively carried out edge point extraction;S02. using point cloud edge point set and image edge point set respectively constructs graph structure to carry out skeleton extraction;S03. point cloud skeleton, image skeleton are matched and fused to form complete skeleton;S04. complete skeleton is detected, and the model defect area is searched;S05. according to the spatial position of model defect area, backtrack to the corresponding image area in original panoramic image data;S06. the image pixel set obtained by backtracking is converted into three-dimensional point cloud data, and then the original point cloud data is completed and repaired, to obtain the final repaired complete point cloud output.The application has the advantages of simple implementation method, high completion and repair efficiency, small amount of calculation and good completion and repair effect.

Description

Technical Field

[0001] This invention relates to the field of 3D model completion technology, and in particular to a 3D model completion and repair method and system based on point cloud vectorized skeleton matching. Background Technology

[0002] A point cloud is a dataset composed of multiple points. Each point in this dataset represents a set of X, Y, and Z geometric coordinates and an intensity value, which is recorded based on the reflectivity of the object's surface to indicate the strength of the returned signal. In short, a point cloud is a collection of data points in space that represent a 3D shape or object. Point clouds can describe 3D objects using relatively small amounts of data, and they can also be automatically colored for more realistic visualization.

[0003] During point cloud data acquisition, issues such as occlusion can lead to missing target point clouds. Point cloud completion and restoration involves repairing missing point clouds in the 3D model to estimate the complete point cloud, thereby obtaining a higher-quality point cloud. Existing technologies typically employ the following two methods for point cloud completion and restoration: 1) Semantic-based point cloud restoration method: By upsampling on the point cloud, that is, inputting a point cloud and outputting a denser point cloud, which falls on the implicit geometry (such as the surface) of the input point cloud, by learning the features of each point at multiple granularities (from local to global), expanding the point set in the feature space, and finally mapping the expanded point set back to the 3D model, point cloud restoration is achieved.

[0004] 2) Geometry-based point cloud repair method: Perform nearest neighbor detection or edge detection on the points in the 3D point cloud to find points that do not meet the threshold conditions, and then use interpolation methods to fill and repair the model.

[0005] However, both the semantic-based and geometry-based point cloud repair methods mentioned above rely on certain prior information to repair missing point clouds. This includes either prior information on semantic information or prior information on the geometric shape of the model itself. Furthermore, the repair process involves significant computational costs, is susceptible to environmental interference, and exhibits poor robustness. These methods are typically only suitable for processing missing point clouds with low missing rates and clear structural features. In practical applications, point cloud scanning time is usually short and the environment is limited. The initial point cloud obtained from the scan is unlikely to possess complete and accurate prior information on semantics or geometry. Simultaneously, computational resources for point cloud processing are often very limited. Consequently, the actual repair efficiency and effectiveness are poor, resulting in low accuracy of the output model. Summary of the Invention The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a three-dimensional model completion and repair method and system based on point cloud vectorized skeleton matching that is simple to implement, has high completion and repair efficiency, low computational and hardware resource consumption, and good completion and repair effect.

[0006] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows: A 3D model completion and repair method based on point cloud vectorized skeleton matching, comprising the following steps: Step S01. Obtain the original point cloud data and original panoramic image data collected in the specified scene, and extract edge points respectively to obtain the point cloud edge point set and the image edge point set; Step S02. Construct graph structures using the point cloud edge point set and the image edge point set to extract the skeleton, thereby obtaining the point cloud skeleton and the image skeleton; Step S03. Register and match the point cloud skeleton and the image skeleton and fuse them to form a complete skeleton; Step S04. Perform integrity detection on the complete skeleton, search for the missing model regions, and obtain the spatial location of the missing model regions; Step S05. Based on the spatial location of the missing area of ​​the model, backtrack to the corresponding image area in the original panoramic image data to obtain the image pixel set of the missing area of ​​the model from the original panoramic image data; Step S06. After converting the image pixel set in the image region obtained by backtracking in step S05 into three-dimensional point cloud data, the original point cloud data is completed and repaired to obtain the final repaired complete point cloud output.

[0007] Furthermore, step S03 includes: Step S301. Register the point cloud skeleton and the image skeleton to obtain the spatial transformation matrix R and translation vector t between the point cloud skeleton and the image skeleton; Step S302. Use the spatial transformation matrix R and the translation vector t to fuse the point cloud skeleton and the image skeleton, so as to register the point cloud skeleton and the image skeleton to the same three-dimensional spatial coordinate system to form the complete skeleton.

[0008] Furthermore, in step S301, the three-dimensional spatial coordinates of vertices V in the extracted point cloud skeleton and the image skeleton are... Registration is performed to obtain the spatial transformation matrix and translation vector between the vertices V of the point cloud skeleton and the image skeleton, and these are respectively used as the spatial transformation matrix R and translation vector t between the point cloud skeleton and the image skeleton.

[0009] Furthermore, in step S302, by following... +t fuses the registered point cloud skeleton and the image skeleton to form the complete skeleton, wherein For a complete skeleton, For the point cloud skeleton, This is the image skeleton.

[0010] Furthermore, in step S04, when searching for incomplete regions of the model, the value of each vertex V in the complete skeleton is calculated. The distance between the target vertex and other points is considered if the minimum distance between the target vertex and other points exceeds a preset threshold. If the target vertex is marked as the model defect, the spatial coordinates corresponding to the current model defect will be output. .

[0011] Furthermore, in step S06, the three-dimensional point cloud data obtained by converting the backtracked image pixel set is added to the corresponding model defect area in the original point cloud data to obtain the final repaired complete point cloud output.

[0012] Furthermore, in step S02, the first graph structure is constructed using the point cloud edge point set. To obtain the sparse skeleton of the point cloud, where Representing vertices, each vertex is the three-dimensional spatial coordinate of each edge point in the point cloud. The vectorized representation of 1 represents an edge, and the value of each edge is the value of the corresponding edge point in the total point cloud data. The minimum Euclidean distance between the center and the remaining points; constructing a second graph structure using the image edge point set. To obtain a dense vectorized skeleton of the image, where ,in 2 represents a vertex, and each vertex is selected from the original image data. The position coordinates of 25 adjacent points The vectorized representation of For each edge, the value is the color value of the corresponding edge point in the original image data, which is the value of the 25 neighboring points. The average value.

[0013] Furthermore, in step S01, when extracting edge points from the original point cloud data, the distance between each point in the original point cloud data and the other points is calculated. When the minimum distance between the target point and the other points is greater than a preset threshold, the target point is marked as an edge point, and all the marked edge points constitute the point cloud edge point set. When extracting edge points from the original panoramic image data, the distance between the color value of each pixel in the panoramic image data and the surrounding pixels is calculated. When the minimum distance between the color value of the target pixel and the surrounding pixels is greater than a preset threshold, the target pixel is marked as an edge point, and all the marked edge points constitute the image edge point set.

[0014] A 3D model completion and repair system based on point cloud vectorized skeleton matching includes: The edge point extraction module acquires the raw point cloud data and raw panoramic image data collected in the specified scene and extracts edge points respectively to obtain the point cloud edge point set and the image edge point set; The skeleton extraction module is used to construct graph structures using point cloud edge point sets and image edge point sets respectively to extract the skeleton, resulting in point cloud skeleton and image skeleton. The registration and fusion module is used to register and match point cloud skeletons and image skeletons and fuse them to form a complete skeleton; The integrity detection module is used to perform integrity detection on the complete skeleton, search for the missing model regions, and obtain the spatial location of the missing model regions. The missing pixel backtracking module is used to backtrack to the corresponding image region in the original panoramic image data based on the spatial location of the missing region of the model, so as to obtain the image pixel set of the missing region of the model from the original panoramic image data. The completion and repair module is used to convert the image pixel set in the image region obtained by the incomplete pixel backtracking module into three-dimensional point cloud data, and then complete and repair the original point cloud data to obtain the final repaired complete point cloud output.

[0015] A 3D model completion and repair system based on point cloud vectorized skeleton matching includes a processor and a memory. The memory is used to store computer programs, and the processor is used to execute the computer programs to perform the methods described above.

[0016] Compared with existing technologies, the advantages of this invention are as follows: This invention extracts edge points from the original point cloud data and the original panoramic image data, and extracts the point cloud skeleton and image skeleton based on the graph structure of the point cloud data vectorized representation of the edge point set. It can quickly extract the high-density 3D point cloud skeleton based on the image. At the same time, after performing integrity detection on the complete skeleton formed by fusing the point cloud skeleton and the image skeleton, the missing areas of the model are determined. The position information of the missing areas of the model is used to trace back to the corresponding image pixel areas in the original image data to obtain the point cloud data for repairing and completing the 3D model. It can quickly and accurately complete and repair the missing parts of the point cloud in the 3D model, reduce the required hardware resource consumption, and improve the robustness of the repair. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating the implementation process of the 3D model completion and repair method based on point cloud vectorized skeleton matching in this embodiment.

[0018] Figure 2 This is a schematic diagram of the structure of the 3D model completion and repair system based on point cloud vectorized skeleton matching in this embodiment. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0020] like Figure 1 As shown, the detailed steps of the 3D model completion and repair method based on point cloud vectorized skeleton matching in this embodiment include: Step S01. Obtain the original point cloud data and original panoramic image data collected in the specified scene, and extract the edge points respectively to obtain the point cloud edge point set and the image edge point set.

[0021] In this embodiment, point cloud data and panoramic image data are collected at the same location in outdoor and non-enclosed scenes to obtain raw point cloud data and raw panoramic image data, and then edge points are extracted from the raw point cloud data and raw panoramic image data.

[0022] In this embodiment, the specific steps for edge point extraction of the original point cloud data include: calculating the distance between each point in the original point cloud data and the other points; when the minimum distance between the target point and the other points is greater than a preset threshold, the target point is marked as an edge point, and the edge points obtained by all the markings constitute the point cloud edge point set; when extracting edge points from the original panoramic image data, calculating the distance between the color value of each pixel in the panoramic image data and the surrounding pixels; when the minimum distance between the color value of the target pixel and the surrounding pixels is greater than a preset threshold, the target pixel is marked as an edge point, and the edge points obtained by all the markings constitute the image edge point set.

[0023] Using the collected scene point cloud data as Point cloud data The panoramic image data collected at the same location has a size of panoramic images For example, for the collected size is Point cloud data When performing edge point extraction, specifically for each point Coordinates, calculate the Euclidean distance of each point to the other N-1 points, and when the minimum distance between a point and the N-1 points is greater than a threshold. When this happens, the point is marked as an edge point, based on the point cloud data. The point cloud edge point set is obtained by marking all edge points in the middle. For the acquired size... panoramic images When performing edge point extraction, specifically the color value of each pixel... Calculate the Euclidean distance between a pixel and its 25 surrounding pixels. If a pixel's minimum distance to its 24 surrounding pixels exceeds a threshold, then... At that time, this point is marked as an edge point, from the panoramic image. The set of edge points in the image is obtained by marking all the edge points in the image.

[0024] Step S02. Construct graph structures using point cloud edge point sets and image edge point sets respectively to extract the skeleton, obtaining point cloud skeleton and image skeleton.

[0025] In this embodiment, the first graph structure is constructed using the point cloud edge point set. To obtain the sparse skeleton of the point cloud, where Representing vertices, each vertex is the three-dimensional spatial coordinate of each edge point in the point cloud. The vectorized representation of 1 represents an edge, and the value (weight coefficient) of each edge is the weight of the corresponding edge point in the entire point cloud data. The minimum Euclidean distance between the center point and the remaining points. This is achieved by constructing a graph structure based on the point cloud edge point set. , graph structure The midpoints correspond to the edge points of each point cloud, and the edges correspond to the minimum distances between edge points, thus utilizing this graph structure. Extract the sparse skeleton of the point cloud.

[0026] In this embodiment, the second graph structure is constructed using the image edge point set. To obtain a dense vectorized skeleton of the image, where ,in 2 represents a vertex, and each vertex is selected from the original image data. The position coordinates of 25 adjacent points The vectorized representation of For each edge, the value is the color value of the corresponding edge point in the original image data, which is the value of the 25 neighboring points. The average value. That is, constructing a graph structure using the image edge point set. , graph structure The midpoint corresponds to the vectorized position coordinates of adjacent points in the image, and the edges are the average color values ​​of adjacent points, thus utilizing this graph structure. Extract the dense vectorized skeleton from the image.

[0027] Because graph structures possess spatial structure and can preserve spatial information, and because point cloud data is discrete data with relatively low data volume and density, while panoramic image data is continuous data with high data volume and density, this embodiment utilizes graph structures to construct a sparse graph structure for point cloud data, taking into account the characteristics of point cloud data and image data. A dense graph structure is constructed from the image data using graph structures. Furthermore, by introducing a vectorization method, we can accurately extract sparse skeletons from point clouds and dense vectorized skeletons from images while effectively preserving spatial information.

[0028] Step S03. Register and match the point cloud skeleton and the image skeleton and fuse them to form a complete skeleton.

[0029] Since the point cloud sparse skeleton and the image dense vectorized skeleton extracted in step S02 are in different coordinate systems, they cannot be directly fused. Therefore, it is necessary to first register the point cloud sparse skeleton and the image dense vectorized skeleton to obtain the spatial positional relationship between the point cloud sparse skeleton and the image dense vectorized skeleton, and then use this spatial positional relationship to fuse the point cloud sparse skeleton and the image dense vectorized skeleton to form the final complete skeleton.

[0030] In this embodiment, the detailed steps for registering, matching, and fusing the point cloud skeleton and the image skeleton to form a complete skeleton include: Step S301. Register the point cloud skeleton and the image skeleton to obtain the spatial transformation matrix R and translation vector t between the point cloud skeleton and the image skeleton; Step S302. Use the spatial transformation matrix R and translation vector t to fuse the point cloud skeleton and the image skeleton, so as to register the point cloud skeleton and the image skeleton to the same three-dimensional spatial coordinate system to form a complete skeleton.

[0031] In step S301 above, specifically by extracting the sparse skeleton of the point cloud... and image dense vectorized skeleton The three-dimensional spatial coordinates of vertex V in the diagram Registration is performed to obtain the spatial transformation matrix and translation vector between the vertices V of the point cloud skeleton and the image skeleton, so as to obtain the spatial transformation matrix R and translation vector t between the point cloud skeleton and the image skeleton.

[0032] In step S302 above, specifically by following... +t merges the registered point cloud skeleton and the image skeleton to form a complete skeleton, where For a complete skeleton, For the point cloud skeleton, This is the image skeleton.

[0033] That is, the sparse skeleton obtained by extraction in this embodiment and dense skeleton The three-dimensional spatial coordinates of vertex V in the diagram Registration is performed to obtain a sparse skeleton. and dense skeleton The spatial transformation matrix R and translation vector t of vertex V in the model are used to sparse the skeleton. and dense skeleton The transformation registers the two skeletons to the same three-dimensional coordinate system, resulting in a new, complete skeleton. The specific registration method can be selected from existing technologies based on actual needs.

[0034] Step S04. Perform integrity detection on the complete skeleton, search for the missing model regions, and obtain the spatial location of the missing model regions.

[0035] In this embodiment, the specific steps for searching incomplete regions of the model include: calculating the value of each vertex V in the complete skeleton. The distance between the target vertex and the other N-1 points; if the minimum distance between the target vertex and the other N-1 points is greater than a preset threshold. Then, the target vertex is marked as a model defect, and the spatial coordinates corresponding to the current model defect are output. .

[0036] Step S05. Based on the spatial location of the missing area in the model, backtrack to the corresponding image area in the original panoramic image data to obtain the image pixel set of the missing area in the model from the original panoramic image data.

[0037] After finding the missing area of ​​the model in step S04, the missing area of ​​the model is used to trace back to the image area corresponding to the original panoramic image data. Thus, the point cloud can be effectively repaired and completed using the original panoramic image data. Compared with the traditional method of using interpolation to fill and repair the model, this method can effectively improve the repair accuracy and reduce the computational complexity.

[0038] Step S06. After converting the image pixel set in the image region obtained by backtracking in step S05 into three-dimensional point cloud data, the original point cloud data is completed and repaired to obtain the final repaired complete point cloud output.

[0039] In this embodiment, the 3D point cloud data obtained by converting the backtracked image pixel set is added to the corresponding missing area of ​​the model in the original point cloud data to obtain the final repaired complete point cloud output. Specifically, according to the spatial coordinates of the missing area output in step S05, Tracing back to the region corresponding to that point in panoramic image I, and then taking all the pixels within that region... Converted into a 3D point cloud through spatial transformation. Record all repaired points as And add it to the original point cloud P to obtain the repaired complete point cloud. This allows us to obtain complete, high-quality point clouds.

[0040] This invention extracts edge points from raw point cloud data and raw panoramic image data. Based on the edge point set, it uses a graph structure to extract the point cloud skeleton and the image skeleton. Since the skeleton extraction process uses a graph structure that represents the point cloud data in a vectorized form, it can quickly extract high-density 3D point cloud skeletons based on images. Then, it merges the point cloud skeleton and the image skeleton to form a complete skeleton. After performing integrity detection on the complete skeleton, it can determine the missing areas of the model. Using the location information of the missing areas of the model, it traces back to the corresponding image pixel areas in the original image data, converts them into point cloud data, and uses them to repair and complete the 3D model. It can accurately complete and repair the missing parts of the point cloud in the 3D model, effectively reducing the amount of computation in the repair process, improving the repair efficiency, and also reducing the required hardware resource consumption and improving the robustness of the repair.

[0041] like Figure 2 As shown, the 3D model completion and repair system based on point cloud vectorized skeleton matching in this embodiment includes: The edge point extraction module acquires the raw point cloud data and raw panoramic image data collected in the specified scene and extracts edge points respectively to obtain the point cloud edge point set and the image edge point set; The skeleton extraction module is used to construct graph structures using point cloud edge point sets and image edge point sets respectively to extract the skeleton, resulting in point cloud skeleton and image skeleton. The registration and fusion module is used to register and match point cloud skeletons and image skeletons and fuse them to form a complete skeleton; The integrity detection module is used to detect the integrity of the complete skeleton, search for the missing areas of the model, and obtain the spatial location of the missing areas. The missing pixel backtracking module is used to backtrack to the corresponding image region in the original panoramic image data based on the spatial location of the missing region of the model, so as to obtain the image pixel set of the missing region of the model from the original panoramic image data; The completion and repair module is used to convert the image pixel set in the image region obtained by the incomplete pixel backtracking module into three-dimensional point cloud data, and then complete and repair the original point cloud data to obtain the final repaired complete point cloud output.

[0042] In this embodiment, the edge point extraction module includes a point cloud edge point extraction unit for extracting edge points from the original point cloud data, and an image edge point extraction unit for extracting edge points from the original panoramic image data. The skeleton extraction module includes a point cloud skeleton extraction unit for constructing a sparse graph structure using the point cloud edge point set to obtain a sparse skeleton of the point cloud, and an image skeleton extraction unit for constructing a dense graph structure using the image edge point set to obtain a dense vectorized skeleton of the image.

[0043] In this embodiment, the registration and fusion module specifically includes: The registration parameter acquisition unit is used to register the point cloud skeleton and the image skeleton, and to obtain the spatial transformation matrix R and translation vector t between the point cloud skeleton and the image skeleton. The fusion unit is used to fuse the point cloud skeleton and the image skeleton using the spatial transformation matrix R and the translation vector t, so as to register the point cloud skeleton and the image skeleton to the same three-dimensional spatial coordinate system to form a complete skeleton.

[0044] In this embodiment, when the incomplete pixel backtracking module searches for incomplete regions of the model, it calculates the value of each vertex V in the complete skeleton. The distance between the target vertex and other points is considered if the minimum distance between the target vertex and other points exceeds a preset threshold. If the target vertex is marked as the model defect, the spatial coordinates corresponding to the current model defect will be output. .

[0045] The 3D model completion and repair device based on point cloud vectorized skeleton matching in this embodiment corresponds one-to-one with the above-mentioned 3D model completion and repair method based on point cloud vectorized skeleton matching, and will not be described in detail here.

[0046] In another embodiment, the 3D model completion and repair system based on point cloud vectorized skeleton matching of the present invention may further include a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the method described above.

[0047] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0048] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for completing and repairing a 3D model based on point cloud vectorized skeleton matching, characterized in that the steps include... include: Step S01. Obtain the original point cloud data and original panoramic image data collected in the specified scene, and extract edge points respectively to obtain the point cloud edge point set and the image edge point set; Step S02. Construct graph structures using the point cloud edge point set and the image edge point set to extract the skeleton, thereby obtaining the point cloud skeleton and the image skeleton; Step S03. Register and match the point cloud skeleton and the image skeleton and fuse them to form a complete skeleton; Step S04. Perform integrity detection on the complete skeleton, search for the missing model regions, and obtain the spatial location of the missing model regions; Step S05. Based on the spatial location of the missing area of ​​the model, backtrack to the corresponding image area in the original panoramic image data to obtain the image pixel set of the missing area of ​​the model from the original panoramic image data; Step S06. After converting the image pixel set in the image region obtained by backtracking in step S05 into three-dimensional point cloud data, the original point cloud data is completed and repaired to obtain the final repaired complete point cloud output.

2. The 3D model completion and repair method based on point cloud vectorized skeleton matching according to claim 1, characterized in that, Step S03 includes: Step S301. Register the point cloud skeleton and the image skeleton to obtain the spatial transformation matrix R and translation vector t between the point cloud skeleton and the image skeleton; Step S302. Use the spatial transformation matrix R and the translation vector t to fuse the point cloud skeleton and the image skeleton, so as to register the point cloud skeleton and the image skeleton to the same three-dimensional spatial coordinate system to form the complete skeleton.

3. The 3D model completion and repair method based on point cloud vectorized skeleton matching according to claim 2, characterized in that, In step S301, the three-dimensional spatial coordinates of the extracted point cloud skeleton and the vertex V in the image skeleton are obtained by... Registration is performed to obtain the spatial transformation matrix and translation vector between the vertices V of the point cloud skeleton and the image skeleton, and these are respectively used as the spatial transformation matrix R and translation vector t between the point cloud skeleton and the image skeleton.

4. The 3D model completion and repair method based on point cloud vectorized skeleton matching according to claim 2, characterized in that, In step S302, by following +t fuses the registered point cloud skeleton and the image skeleton to form the complete skeleton, wherein For a complete skeleton, For the point cloud skeleton, This is the image skeleton.

5. The 3D model completion and repair method based on point cloud vectorized skeleton matching according to claim 1, characterized in that, In step S04, when searching for incomplete regions of the model, the value of each vertex V in the complete skeleton is calculated. The distance between the target vertex and other points is considered if the minimum distance between the target vertex and other points exceeds a preset threshold. Then, the target vertex is marked as a model defect, and the spatial coordinates corresponding to the current model defect are output. .

6. The 3D model completion and repair method based on point cloud vectorized skeleton matching according to any one of claims 1 to 5, characterized in that, In step S06, the three-dimensional point cloud data obtained by converting the backtracked image pixel set is added to the corresponding model defect area in the original point cloud data to obtain the final repaired complete point cloud output.

7. The 3D model completion and repair method based on point cloud vectorized skeleton matching according to any one of claims 1 to 5, characterized in that, In step S02, the first graph structure is constructed using the point cloud edge point set. To obtain the sparse skeleton of the point cloud, where Representing vertices, each vertex is the three-dimensional spatial coordinate of each edge point in the point cloud. The vectorized representation of 1 represents an edge, and the value of each edge is the value of the corresponding edge point in the total point cloud data. The minimum Euclidean distance between the center and the remaining points; constructing a second graph structure using the image edge point set. To obtain a dense vectorized skeleton of the image, where ,in 2 represents a vertex, and each vertex is selected from the original image data. The position coordinates of 25 adjacent points The vectorized representation of For each edge, the value is the color value of the corresponding edge point in the original image data, which is the value of the 25 neighboring points. The average value.

8. The 3D model completion and repair method based on point cloud vectorized skeleton matching according to any one of claims 1 to 5, characterized in that, In step S01, when extracting edge points from the original point cloud data, the distance between each point in the original point cloud data and the other points is calculated. When the minimum distance between the target point and the other points is greater than a preset threshold, the target point is marked as an edge point, and the edge points obtained by all the markings constitute the point cloud edge point set. When extracting edge points from the original panoramic image data, the distance between the color value of each pixel in the panoramic image data and the surrounding pixels is calculated. When the minimum distance between the color value of the target pixel and the surrounding pixels is greater than a preset threshold, the target pixel is marked as an edge point. All the marked edge points constitute the image edge point set.

9. A 3D model completion and repair system based on point cloud vectorized skeleton matching, characterized in that, include: The edge point extraction module acquires the raw point cloud data and raw panoramic image data collected in the specified scene and extracts edge points respectively to obtain the point cloud edge point set and the image edge point set; The skeleton extraction module is used to construct graph structures using point cloud edge point sets and image edge point sets respectively to extract the skeleton, resulting in point cloud skeleton and image skeleton. The registration and fusion module is used to register and match point cloud skeletons and image skeletons and fuse them to form a complete skeleton; The integrity detection module is used to perform integrity detection on the complete skeleton, search for the missing model regions, and obtain the spatial location of the missing model regions. The missing pixel backtracking module is used to backtrack to the corresponding image region in the original panoramic image data based on the spatial location of the missing region of the model, so as to obtain the image pixel set of the missing region of the model from the original panoramic image data. The completion and repair module is used to convert the image pixel set in the image region obtained by the incomplete pixel backtracking module into three-dimensional point cloud data, and then complete and repair the original point cloud data to obtain the final repaired complete point cloud output.

10. A 3D model completion and repair system based on point cloud vectorized skeleton matching, comprising a processor and a memory, wherein the memory is used to store computer programs, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 8.

Images