Mechanical part three-dimensional reconstruction method and system
By calculating the main curvature of each point in the SFM point cloud and generating new adjacent points, combined with the shape and dense processing of Gaussian points, the problem of inaccurate modeling in the three-dimensional reconstruction of mechanical parts is solved, and higher accuracy and Adaptability.
Patent Information
- Application Number
- CN202411721141.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-02-14
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the three-dimensional reconstruction of mechanical parts, the prior art is difficult to accurately capture complex surfaces and details, resulting in inaccurate modeling.
By calculating the main curvature of each point in the SFM point cloud, a new adjacent point is generated, and the shape of the Gaussian point is determined based on the main curvature, and the Gaussian point intensive processing is performed to improve the accuracy of three-dimensional reconstruction.
It improves the accuracy of three-dimensional reconstruction of mechanical parts, can better capture the details and complex surfaces of parts, and enhances the accuracy and adaptability of modeling.
Smart Images
Figure CN119444998A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of three-dimensional reconstruction, and in particular to a three-dimensional reconstruction method and system for mechanical parts. Background Art
[0002] Through 3D reconstruction, the geometric shape, size and structural features of parts can be accurately captured, the digital archiving and sharing of parts can be realized, and important basic data can be provided for the improvement and optimization of parts. In addition, 3D reconstruction can also be used to detect problems such as wear and defects of parts, which helps to improve the production quality and service life of parts. The general practice is to take 2D images of mechanical parts from multiple angles, and then obtain point clouds through methods such as SFM, and perform 3D reconstruction based on the point clouds. The precision requirements of mechanical parts are high, and the 3D reconstruction method needs to have high precision and be able to capture the details of parts. 3D Gaussian Splatting is an effective 3D modeling and rendering method, and it has the potential in the reconstruction of complex surfaces of mechanical parts. However, 3D Gaussian Splatting is based on sparse point clouds, and the point clouds generated by the SFM method are also sparse. However, the structure of mechanical parts is complex, with many edges and corners. When reconstructing mechanical parts in 3D, inaccurate modeling is likely to occur. Summary of the invention
[0003] Based on this, the present invention provides a method and system for three-dimensional reconstruction of mechanical parts to solve at least one of the above technical problems.
[0004] To achieve the above object, the present invention provides a method for three-dimensional reconstruction of mechanical parts, the method comprising the following steps:
[0005] Step 1, obtaining the SFM point cloud of the mechanical part according to the multi-view image, calculating the principal curvature of each point in the point cloud, and taking the maximum absolute value of the maximum curvature and the minimum curvature in the principal curvature as the curvature of the point; according to the curvature of the point and the density of the point within a preset range, using the multi-view image to generate new neighboring points for the points in the SFM point cloud;
[0006] Step 2, determining the shape of the Gaussian point when the Gaussian point is initialized based on the principal curvature of the point in the SFM point cloud. When densifying the Gaussian point, if the Gaussian point to be densified is initialized by the SFM point cloud, and there are neighboring points of the Gaussian point to be densified that are not initialized as Gaussian points, the Gaussian point to be densified is densified according to the neighboring points of the Gaussian point to be densified;
[0007] Step 3: After the training is completed, the trained Gaussian points are used to complete the three-dimensional reconstruction of the mechanical parts.
[0008] Preferably, the method of generating new neighboring points for points in the SFM point cloud using the multi-view images according to the curvature of the points and the density of the points within a preset range is specifically as follows:
[0009] determining a preset number based on the curvature of the points and the density of the points within a preset range;
[0010] Obtain the position of a point in the SFM point cloud in each view image, select a preset number of target pixel points from pixel points within a certain range centered on the position based on the gradient of the pixel point, and transform the target pixel point into a three-dimensional space to obtain a three-dimensional neighboring point;
[0011] The three-dimensional neighboring points of the same point in the SFM point cloud in all view images are deduplicated, and the preset number of three-dimensional neighboring points are retained as new neighboring points of the point.
[0012] Preferably, the shape of the Gaussian point when the Gaussian point is initialized is determined based on the principal curvature of the point in the SFM point cloud, specifically:
[0013] Obtaining the shortest distance between the point to be initialized in the SFM point cloud and other points in the SFM point cloud, and obtaining the curvature of the point to be initialized;
[0014] If the curvature is less than a threshold value, the shape of the Gaussian point is a sphere, and the radius of the sphere is determined based on the closest distance and the curvature; otherwise, the shape of the Gaussian point is an ellipsoid, and the shape of the ellipsoid is determined based on the closest distance and the principal curvature.
[0015] Preferably, the densifying of the Gaussian points to be densified according to the neighboring points of the Gaussian points to be densified is specifically as follows:
[0016] Determine the type of the densified Gaussian point. If the type is split, obtain all neighboring points that are not initialized as Gaussian points and add them to the set. Add the Gaussian point to be densified to the set, cluster the points in the set into two categories according to the distance, find the point in the set that is closest to the center of each cluster, and determine the shape of the Gaussian point when the closest point is initialized as a Gaussian point according to the principal curvature of the closest point.
[0017] If the type is clone, obtain all neighboring points that are not initialized as Gaussian points corresponding to the Gaussian point to be densified and add them to the set, obtain the centroid of the interval formed by all points in the set, find the point in the set closest to the centroid, and determine the shape of the Gaussian point when the nearest point is initialized as a Gaussian point based on the principal curvature of the nearest point.
[0018] Preferably, the step 2 further includes:
[0019] If the Gaussian points to be densified are not initialized by the SFM point cloud, or there are no neighboring points that are not initialized as Gaussian points, the Gaussian points to be densified are densified according to the default method in 3D Gaussian sputtering.
[0020] In addition, the present invention provides a mechanical parts three-dimensional reconstruction system, the system comprising the following modules:
[0021] A point cloud generation module is used to obtain the SFM point cloud of the mechanical part according to the multi-view image, calculate the principal curvature of each point in the point cloud, and take the maximum absolute value of the maximum curvature and the minimum curvature in the principal curvature as the curvature of the point; according to the curvature of the point and the density of the point within a preset range, use the multi-view image to generate new neighboring points for the points in the SFM point cloud;
[0022] A Gaussian point optimization module is used to determine the shape of the Gaussian point when the Gaussian point is initialized based on the principal curvature of the point in the SFM point cloud. When the Gaussian point is densified, if the Gaussian point to be densified is initialized by the SFM point cloud, and there are neighboring points of the Gaussian point to be densified that are not initialized as Gaussian points, the Gaussian point to be densified is densified according to the neighboring points of the Gaussian point to be densified;
[0023] The 3D reconstruction module is used to complete the 3D reconstruction of mechanical parts using the trained Gaussian points after the training is completed.
[0024] Preferably, the method of generating new neighboring points for points in the SFM point cloud using the multi-view images according to the curvature of the points and the density of the points within a preset range is specifically as follows:
[0025] determining a preset number based on the curvature of the points and the density of the points within a preset range;
[0026] Obtain the position of a point in the SFM point cloud in each view image, select a preset number of target pixel points from pixel points within a certain range centered on the position based on the gradient of the pixel point, and transform the target pixel point into a three-dimensional space to obtain a three-dimensional neighboring point;
[0027] The three-dimensional neighboring points of the same point in the SFM point cloud in all view images are deduplicated, and the preset number of three-dimensional neighboring points are retained as new neighboring points of the point.
[0028] Preferably, the shape of the Gaussian point when the Gaussian point is initialized is determined based on the principal curvature of the point in the SFM point cloud, specifically:
[0029] Obtaining the shortest distance between the point to be initialized in the SFM point cloud and other points in the SFM point cloud, and obtaining the curvature of the point to be initialized;
[0030] If the curvature is less than a threshold value, the shape of the Gaussian point is a sphere, and the radius of the sphere is determined based on the closest distance and the curvature; otherwise, the shape of the Gaussian point is an ellipsoid, and the shape of the ellipsoid is determined based on the closest distance and the principal curvature.
[0031] Preferably, the densifying of the Gaussian points to be densified according to the neighboring points of the Gaussian points to be densified is specifically as follows:
[0032] Determine the type of the densified Gaussian point. If the type is split, obtain all neighboring points that are not initialized as Gaussian points and add them to the set. Add the Gaussian point to be densified to the set, cluster the points in the set into two categories according to the distance, find the point in the set that is closest to the center of each cluster, and determine the shape of the Gaussian point when the closest point is initialized as a Gaussian point according to the principal curvature of the closest point.
[0033] If the type is clone, obtain all neighboring points that are not initialized as Gaussian points corresponding to the Gaussian point to be densified and add them to the set, obtain the centroid of the interval formed by all points in the set, find the point in the set closest to the centroid, and determine the shape of the Gaussian point when the nearest point is initialized as a Gaussian point based on the principal curvature of the nearest point.
[0034] Preferably, the Gaussian point optimization module is further used for:
[0035] If the Gaussian points to be densified are not initialized by the SFM point cloud, or there are no neighboring points that are not initialized as Gaussian points, the Gaussian points to be densified are densified according to the default method in 3D Gaussian sputtering.
[0036] Finally, the present invention provides a computer program, which, when executed by a processor, implements the method as described above.
[0037] In the three-dimensional reconstruction of mechanical parts, the present invention determines the shape of the Gaussian points when initializing the Gaussian points based on the principal curvature of the points in the SFM point cloud. Points with large curvature are initialized as smaller Gaussian points, which can better obtain the details of the mechanical parts. Points with small curvature are initialized as larger Gaussian points, which can reduce the number of Gaussian points and improve the calculation speed.
[0038] In addition, when the present invention densifies Gaussian points, if the Gaussian points to be densified are initialized by the SFM point cloud, and there are neighboring points of the Gaussian points to be densified that have not been initialized as Gaussian points, the Gaussian points to be densified are densified according to the neighboring points of the Gaussian points to be densified, which can improve the adaptability of the densification, avoid using the same densification method for all Gaussian points, and can make the densed Gaussian points more consistent with the structure of mechanical parts. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 This is a flow chart of Embodiment 1;
[0040] Figure 2 Schematic diagram of Gaussian point splitting process;
[0041] Figure 3 Schematic diagram of Gaussian point cloning process
[0042] Figure 4 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION
[0043] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.
[0044] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.
[0045] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.
[0046] See also Figure 1 The present invention provides a method for three-dimensional reconstruction of mechanical parts, the method comprising the following steps:
[0047] Step 1, obtaining the SFM point cloud of the mechanical part according to the multi-view image, calculating the principal curvature of each point in the point cloud, and taking the maximum absolute value of the maximum curvature and the minimum curvature in the principal curvature as the curvature of the point; according to the curvature of the point and the density of the point within a preset range, using the multi-view image to generate new neighboring points for the points in the SFM point cloud;
[0048] SFM (Structure from Motion) is a method that uses multiple images taken from different perspectives to generate a 3D point cloud. The point cloud generated by the SFM method is an SFM point cloud, and the camera pose, such as position and direction, can also be obtained. The SFM point cloud is generally a sparse point cloud.
[0049] The degree of bending is different in different parts of mechanical parts. For the part with larger bending, it is generally necessary to use a smaller Gaussian sphere or Gaussian ellipsoid, so that the details of the bending part can be depicted. For the points in the SFM point cloud, the principal curvature of each point is calculated. The principal curvature includes two curvatures, namely the maximum principal curvature and the minimum principal curvature. The curvature may be positive, 0, or negative. In order to reflect the degree of bending of the point, the present invention takes the maximum absolute value of the maximum curvature and the minimum curvature in the principal curvature as the curvature of the point, that is, the absolute value of the maximum curvature and the absolute value of the minimum curvature in the principal curvature are calculated, and the maximum absolute value of the two absolute values is taken as the curvature of the point. For example, the maximum curvature of the principal curvature of a point P (1.2, 0.5, 2.3) is 0.2, and the minimum curvature is -0.3, then the curvature of this point is 0.3.
[0050] A point with a large curvature means that the surface geometry at that point changes greatly. More neighboring points are generated in these areas to better describe the details of the area. At the same time, if the point density around a point is low, it means that the area is relatively sparse and more neighboring points need to be generated; if the neighborhood of the point is already relatively dense, the generated neighboring points can be reduced. The density refers to the number of points that already exist in the neighborhood of the point. In a specific embodiment, the multi-view image is used to generate new neighboring points for points in the SFM point cloud based on the curvature of the point and the density of the point within a preset range, specifically:
[0051] The preset number is determined based on the curvature of the point and the density of the point within the preset range; for example, if the curvature of a point in the SFM point cloud is 0.8, and there are 5 neighboring points within the preset range, which are points with high curvature and low density, the preset number is set to 10. In a specific embodiment, the preset number is determined based on the curvature of the point and the density of the point within the preset range, specifically: the curvature is divided into three intervals of high, medium and low, and a curvature weight is set for each interval, and the density is divided into three intervals of high, medium and low, and a density weight is set for each interval; the curvature interval into which the curvature of the point falls is determined, and then the curvature weight is determined, and the density interval into which the density of the point falls is determined, and then the density weight is determined; the sum of the curvature weight and the density weight is used as the weighting coefficient, and the product of the weighting coefficient and the preset value is used as the preset number. Among them, the curvature weight corresponding to the three intervals of high, medium and low curvature decreases in turn, and the density weight corresponding to the three intervals of high, medium and low density increases in turn. Of course, the above division into three intervals is only one implementation method, and it can also be divided into two intervals or four intervals, or even more intervals.
[0052] Obtain the position of a point in the SFM point cloud in each view image, select a preset number of target pixel points from pixel points within a certain range centered on the position based on the gradient of the pixel point, and transform the target pixel point into a three-dimensional space to obtain a three-dimensional neighboring point;
[0053] The midpoint of the SFM point cloud comes from the calculation of the image. The specific method of obtaining the SFM point cloud through multi-view images belongs to the existing technology and will not be repeated here. Similarly, the points in the SFM point cloud can be projected onto images of various viewpoints to obtain the position of the midpoint of the SFM point cloud in the image. For example, the position of a point P (1.2, 0.5, 2.3) in the SFM point cloud on image 1 is (25, 263). Similarly, the position of point P (1.2, 0.5, 2.3) on image 2 can be obtained as (125, 57).
[0054] In each image, with the pixel position of the point projection as the center, select neighboring pixels within a certain range, such as the 8 nearest neighbors or 15 nearest neighbors of (25,263). Pixels with larger gradients are usually located at the edge of the object or where the features are more obvious. Selecting these pixels can better describe the details of the three-dimensional surface. From the pixels in these neighborhoods, select a certain number of target pixels according to the gradient size. The target pixel represents the neighboring points of the point on the image plane and is used to generate new neighboring points.
[0055] Through the camera parameters (intrinsic and extrinsic parameters) and depth information in the multi-view image, each selected target pixel point is transformed from two-dimensional pixel coordinates to three-dimensional coordinates, that is, back-projection from the image plane to three-dimensional space is performed.
[0056] De-duplicate the three-dimensional neighboring points of the same point in the SFM point cloud in all view images, and retain the preset number of three-dimensional neighboring points as the new neighboring points of the point. Since each point may appear in images from multiple viewpoints, the three-dimensional neighboring points generated from different viewpoints may overlap. In order to avoid duplicate points, it is necessary to deduplicate the neighboring points generated from different viewpoints. If the positions of the neighboring points generated by multiple viewpoints are the same or very close (for example, less than a certain threshold), only one of them is retained. If the number of points after de-duplication exceeds the preset number, the nearest several neighboring points can be selected based on distance or other priority rules such as gradient or curvature.
[0057] Step 2, determining the shape of the Gaussian point when the Gaussian point is initialized based on the principal curvature of the point in the SFM point cloud. When densifying the Gaussian point, if the Gaussian point to be densified is initialized by the SFM point cloud, and there are neighboring points of the Gaussian point to be densified that are not initialized as Gaussian points, the Gaussian point to be densified is densified according to the neighboring points of the Gaussian point to be densified;
[0058] In places with large curvature, it is more advantageous to use smaller Gaussian points in order to express the details of mechanical parts. In a specific embodiment, the shape of the Gaussian points when initializing the Gaussian points is determined based on the principal curvature of the points in the SFM point cloud, specifically:
[0059] Obtaining the shortest distance between the point to be initialized in the SFM point cloud and other points in the SFM point cloud, and obtaining the curvature of the point to be initialized;
[0060] If the curvature is less than a threshold value, the shape of the Gaussian point is a sphere, and the radius of the sphere is determined based on the closest distance and the curvature; otherwise, the shape of the Gaussian point is an ellipsoid, and the shape of the ellipsoid is determined based on the closest distance and the principal curvature.
[0061] When the curvature of the point to be initialized is small, it means that the point is located in an area where the geometric changes are relatively flat, and the surface curvature is not obvious. In this case, the point is initialized as an isotropic Gaussian point. The Gaussian points are spherically distributed. In a more specific embodiment, the radius is determined as follows: k* closest distance*exp(-curvature). In the present invention, unless otherwise specified, the principal curvature, maximum curvature, minimum curvature, and curvature have different meanings, wherein the principal curvature includes two curvatures, namely the maximum curvature and the minimum curvature, and the curvature represents the maximum absolute value of the maximum curvature and the minimum curvature, that is, curvature = max(|maximum principal curvature|,|minimum principal curvature|).
[0062] When the curvature is large, it means that the point is in an area where the local surface changes are more complex, which may be a corner or edge. At this time, the surface is severely curved and the geometric features are obvious. In this case, it is initialized to an ellipsoid shape. The shape and size of the ellipsoid are different in different directions, which can more accurately describe the local geometric changes on complex surfaces. In a more specific embodiment, the shape of the ellipsoid is determined based on the closest distance and the principal curvature, specifically: the length, width and height of the ellipsoid are: k*closest distance*exp(-curvature), k*closest distance*exp(-min(|maximum principal curvature|,|minimum principal curvature|)), k*closest distance*exp(-(|maximum principal curvature|+|minimum principal curvature|) / 2). Among them, k is the adjustment coefficient, 0<k<1.
[0063] During the training process, Gaussian points will be continuously optimized, and Gaussian points will be continuously added and deleted. Adding Gaussian points, that is, densifying Gaussian points, is divided into two operations, namely cloning and splitting. In 3D Gaussian sputtering, cloning is to copy a new Gaussian point from the Gaussian point to be cloned, and splitting is to split the Gaussian point to be split into two Gaussian points. However, in 3D Gaussian sputtering, the position and shape of the densified points are the same for all Gaussian points, but different parts of mechanical parts have different geometric features, which easily leads to the inability to adaptively handle local geometric complexity well. In complex areas, it is easy for the points to be too sparsely distributed or inaccurately described, resulting in a decrease in reconstruction accuracy. In a preferred embodiment, the Gaussian points to be densified are densified according to the neighboring points of the Gaussian points to be densified, specifically as follows:
[0064] Determine the type of the densified Gaussian point. If the type is split, obtain all neighboring points that are not initialized as Gaussian points and add them to the set. Add the Gaussian point to be densified to the set, cluster the points in the set into two categories according to the distance, find the point in the set that is closest to the center of each cluster, and determine the shape of the Gaussian point when the closest point is initialized as a Gaussian point according to the principal curvature of the closest point.
[0065] When the type of Gaussian point to be densified is split, first you need to get the neighboring points of the point, which have not yet been initialized as Gaussian points. Assuming that the neighboring points of the densified Gaussian point p0 are p1-p5, but p2 has been initialized as a Gaussian point, then the points in the set are p0, p1, p3-p5, cluster the points in the set, and divide them into two categories according to the distance between the points. The purpose of clustering is to divide the points into two regions so that these points can represent different geometric feature regions. For each cluster, find the point closest to the cluster center. This point will be used as the point representing the cluster for subsequent Gaussian point initialization. For example, if the center of the first cluster is p2 and the center of the second cluster is p5, the shape of p2 initialized as a Gaussian point is determined according to the principal curvature of p2, and the shape of p5 initialized as a Gaussian point is determined according to the principal curvature of p5, such as Figure 2 As shown. The shape of the initialization Gaussian point is determined according to the principal curvature in the manner described above, which will not be repeated here.
[0066] If the type is clone, obtain all neighboring points that are not initialized as Gaussian points corresponding to the Gaussian point to be densified and add them to the set, obtain the centroid of the interval formed by all points in the set, find the point in the set closest to the centroid, and determine the shape of the Gaussian point when the nearest point is initialized as a Gaussian point based on the principal curvature of the nearest point.
[0067] If the type of the Gaussian point to be densified is clone, all its neighboring points that are not initialized as Gaussian points are still obtained and added to the set. This is similar to the split type, collecting all uninitialized points in the neighborhood of the point for subsequent densification processing, but does not include the Gaussian point to be densified. All points in the set are regarded as a region, and the centroid of this region (that is, the geometric center of all points) is calculated. The point closest to this centroid is found as the representative point of the region. This closest point will be used for subsequent Gaussian point initialization, such as Figure 3 The shape of the initial Gaussian point is determined according to the principal curvature in the same manner as described above, which will not be repeated here.
[0068] The step 2 further includes: if the Gaussian points to be densified are not initialized by the SFM point cloud, or there are no neighboring points to be densified that are not initialized as Gaussian points, the Gaussian points to be densified are densified according to a default method in 3D Gaussian sputtering.
[0069] Step 3: After the training is completed, the trained Gaussian points are used to complete the three-dimensional reconstruction of the mechanical parts.
[0070] In the process of 3D reconstruction using 3D Gaussian sputtering, the goal of the training phase is to optimize the parameters of the Gaussian points (position, shape, size, color, etc.) so that these Gaussian points can accurately represent the 3D structure of the object. When the training is completed, it means that the Gaussian points have been continuously adjusted and optimized to well represent the geometric shape and surface details of the mechanical parts. Gaussian points can be converted into 3D meshes through calculation and rendering software to generate a fine 3D model, or directly rendered through Gaussian point clouds to achieve visualization of mechanical parts and then achieve 3D reconstruction of mechanical parts.
[0071] See also Figure 4 The present invention provides a three-dimensional reconstruction system for mechanical parts, the system comprising the following modules:
[0072] A point cloud generation module is used to obtain the SFM point cloud of the mechanical part according to the multi-view image, calculate the principal curvature of each point in the point cloud, and take the maximum absolute value of the maximum curvature and the minimum curvature in the principal curvature as the curvature of the point; according to the curvature of the point and the density of the point within a preset range, use the multi-view image to generate new neighboring points for the points in the SFM point cloud;
[0073] A Gaussian point optimization module is used to determine the shape of the Gaussian point when the Gaussian point is initialized based on the principal curvature of the point in the SFM point cloud. When the Gaussian point is densified, if the Gaussian point to be densified is initialized by the SFM point cloud, and there are neighboring points of the Gaussian point to be densified that are not initialized as Gaussian points, the Gaussian point to be densified is densified according to the neighboring points of the Gaussian point to be densified;
[0074] The 3D reconstruction module is used to complete the 3D reconstruction of mechanical parts using the trained Gaussian points after the training is completed.
[0075] Preferably, the method of generating new neighboring points for points in the SFM point cloud using the multi-view images according to the curvature of the points and the density of the points within a preset range is specifically as follows:
[0076] determining a preset number based on the curvature of the points and the density of the points within a preset range;
[0077] Obtain the position of a point in the SFM point cloud in each view image, select a preset number of target pixel points from pixel points within a certain range centered on the position based on the gradient of the pixel point, and transform the target pixel point into a three-dimensional space to obtain a three-dimensional neighboring point;
[0078] The three-dimensional neighboring points of the same point in the SFM point cloud in all view images are deduplicated, and the preset number of three-dimensional neighboring points are retained as new neighboring points of the point.
[0079] Preferably, the shape of the Gaussian point when the Gaussian point is initialized is determined based on the principal curvature of the point in the SFM point cloud, specifically:
[0080] Obtaining the shortest distance between the point to be initialized in the SFM point cloud and other points in the SFM point cloud, and obtaining the curvature of the point to be initialized;
[0081] If the curvature is less than a threshold value, the shape of the Gaussian point is a sphere, and the radius of the sphere is determined based on the closest distance and the curvature; otherwise, the shape of the Gaussian point is an ellipsoid, and the shape of the ellipsoid is determined based on the closest distance and the principal curvature.
[0082] Preferably, the densifying of the Gaussian points to be densified according to the neighboring points of the Gaussian points to be densified is specifically as follows:
[0083] Determine the type of the densified Gaussian point. If the type is split, obtain all neighboring points that are not initialized as Gaussian points and add them to the set. Add the Gaussian point to be densified to the set, cluster the points in the set into two categories according to the distance, find the point in the set that is closest to the center of each cluster, and determine the shape of the Gaussian point when the closest point is initialized as a Gaussian point according to the principal curvature of the closest point.
[0084] If the type is clone, obtain all neighboring points that are not initialized as Gaussian points corresponding to the Gaussian point to be densified and add them to the set, obtain the centroid of the interval formed by all points in the set, find the point in the set closest to the centroid, and determine the shape of the Gaussian point when the nearest point is initialized as a Gaussian point based on the principal curvature of the nearest point.
[0085] Preferably, the Gaussian point optimization module is further used for:
[0086] If the Gaussian points to be densified are not initialized by the SFM point cloud, or there are no neighboring points that are not initialized as Gaussian points, the Gaussian points to be densified are densified according to the default method in 3D Gaussian sputtering.
[0087] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.
[0088] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.
Claims
1. A method for three-dimensional reconstruction of mechanical parts, characterized in that: The method comprises the following steps: Step 1, obtaining the SFM point cloud of the mechanical part according to the multi-view image, calculating the principal curvature of each point in the point cloud, and taking the maximum absolute value of the maximum curvature and the minimum curvature in the principal curvature as the curvature of the point; according to the curvature of the point and the density of the point within a preset range, using the multi-view image to generate new neighboring points for the points in the SFM point cloud; Step 2, determining the shape of the Gaussian point when the Gaussian point is initialized based on the principal curvature of the point in the SFM point cloud. When densifying the Gaussian point, if the Gaussian point to be densified is initialized by the SFM point cloud, and there are neighboring points of the Gaussian point to be densified that are not initialized as Gaussian points, the Gaussian point to be densified is densified according to the neighboring points of the Gaussian point to be densified; Step 3: After the training is completed, the trained Gaussian points are used to complete the three-dimensional reconstruction of the mechanical parts.
2. The method according to claim 1, characterized in that The method of generating new neighboring points for points in the SFM point cloud using the multi-view images according to the curvature of the points and the density of the points within a preset range is specifically as follows: determining a preset number based on the curvature of the points and the density of the points within a preset range; Obtain the position of a point in the SFM point cloud in each view image, select a preset number of target pixel points from pixel points within a certain range centered on the position based on the gradient of the pixel point, and transform the target pixel point into a three-dimensional space to obtain a three-dimensional neighboring point; The three-dimensional neighboring points of the same point in the SFM point cloud in all view images are deduplicated, and the preset number of three-dimensional neighboring points are retained as new neighboring points of the point.
3. The method according to claim 1, characterized in that The shape of the Gaussian point when the Gaussian point is initialized is determined based on the principal curvature of the point in the SFM point cloud, specifically: Obtaining the shortest distance between the point to be initialized in the SFM point cloud and other points in the SFM point cloud, and obtaining the curvature of the point to be initialized; If the curvature is less than a threshold value, the shape of the Gaussian point is a sphere, and the radius of the sphere is determined based on the closest distance and the curvature; otherwise, the shape of the Gaussian point is an ellipsoid, and the shape of the ellipsoid is determined based on the closest distance and the principal curvature.
4. The method according to claim 1, characterized in that The densification of the Gaussian points to be densified according to the neighboring points of the Gaussian points to be densified is specifically as follows: Determine the type of the densified Gaussian point. If the type is split, obtain all neighboring points that are not initialized as Gaussian points and add them to the set. Add the Gaussian point to be densified to the set, cluster the points in the set into two categories according to the distance, find the point in the set that is closest to the center of each cluster, and determine the shape of the Gaussian point when the closest point is initialized as a Gaussian point according to the principal curvature of the closest point. If the type is clone, obtain all neighboring points that are not initialized as Gaussian points corresponding to the Gaussian point to be densified and add them to the set, obtain the centroid of the interval formed by all points in the set, find the point in the set closest to the centroid, and determine the shape of the Gaussian point when the nearest point is initialized as a Gaussian point based on the principal curvature of the nearest point.
5. The method according to claim 1, characterized in that The step 2 further includes: If the Gaussian points to be densified are not initialized by the SFM point cloud, or there are no neighboring points that are not initialized as Gaussian points, the Gaussian points to be densified are densified according to the default method in 3D Gaussian sputtering.
6. A three-dimensional reconstruction system for mechanical parts, characterized in that: The system includes the following modules: Point cloud generation module, used to obtain the SFM point cloud of mechanical parts according to multi-view images, calculate the principal curvature of each point in the point cloud, and take the maximum absolute value of the maximum curvature and the minimum curvature in the principal curvature as the curvature of the point; Generating new neighboring points for points in the SFM point cloud using the multi-view images according to the curvature of the points and the density of the points within a preset range; A Gaussian point optimization module is used to determine the shape of the Gaussian point when the Gaussian point is initialized based on the principal curvature of the point in the SFM point cloud. When the Gaussian point is densified, if the Gaussian point to be densified is initialized by the SFM point cloud, and there are neighboring points of the Gaussian point to be densified that are not initialized as Gaussian points, the Gaussian point to be densified is densified according to the neighboring points of the Gaussian point to be densified; The 3D reconstruction module is used to complete the 3D reconstruction of mechanical parts using the trained Gaussian points after the training is completed.
7. The system according to claim 6, characterized in that The method of generating new neighboring points for points in the SFM point cloud using the multi-view images according to the curvature of the points and the density of the points within a preset range is specifically as follows: determining a preset number based on the curvature of the points and the density of the points within a preset range; Obtain the position of a point in the SFM point cloud in each view image, select a preset number of target pixel points from pixel points within a certain range centered on the position based on the gradient of the pixel point, and transform the target pixel point into a three-dimensional space to obtain a three-dimensional neighboring point; The three-dimensional neighboring points of the same point in the SFM point cloud in all view images are deduplicated, and the preset number of three-dimensional neighboring points are retained as new neighboring points of the point.
8. The system according to claim 6, characterized in that The shape of the Gaussian point when the Gaussian point is initialized is determined based on the principal curvature of the point in the SFM point cloud, specifically: Obtaining the shortest distance between the point to be initialized in the SFM point cloud and other points in the SFM point cloud, and obtaining the curvature of the point to be initialized; If the curvature is less than a threshold value, the shape of the Gaussian point is a sphere, and the radius of the sphere is determined based on the closest distance and the curvature; otherwise, the shape of the Gaussian point is an ellipsoid, and the shape of the ellipsoid is determined based on the closest distance and the principal curvature.
9. The system according to claim 6, characterized in that The densification of the Gaussian points to be densified according to the neighboring points of the Gaussian points to be densified is specifically as follows: Determine the type of the densified Gaussian point. If the type is split, obtain all neighboring points that are not initialized as Gaussian points and add them to the set. Add the Gaussian point to be densified to the set, cluster the points in the set into two categories according to the distance, find the point in the set that is closest to the center of each cluster, and determine the shape of the Gaussian point when the closest point is initialized as a Gaussian point according to the principal curvature of the closest point. If the type is clone, obtain all neighboring points that are not initialized as Gaussian points corresponding to the Gaussian point to be densified and add them to the set, obtain the centroid of the interval formed by all points in the set, find the point in the set closest to the centroid, and determine the shape of the Gaussian point when the nearest point is initialized as a Gaussian point based on the principal curvature of the nearest point.
10. A computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 5.