Multi-modal fusion and physical constraint three-dimensional point cloud geometric modeling system and method

Abstract

The invention relates to the field of computer graphics, reverse engineering and additive manufacturing, discloses a multi-modal fusion and physical constraint three-dimensional point cloud geometric modeling system and method, and aims at overcoming the defects that traditional three-dimensional reconstruction entity attributes are missing, key features are prone to being lost, semantic understanding does not exist, and process automation is low. The system comprises a point cloud preprocessing module, a semantic segmentation and recognition module, a mixed geometric reconstruction module, a multi-component assembly and physical constraint solution module and an entity model output module. The method comprises the following steps: denoising and sampling an original point cloud, realizing semantic and instance double segmentation through a Point Net + + network, endowing a geometry-function composite label, completing hybrid reconstruction through improved Poisson reconstruction, parameterized B-Rep fitting and standard component library matching, and outputting a multi-format engineering available model after physical constraint verification and automatic correction of a violation model. According to the method, automatic, high-fidelity and high-practicability conversion from the point cloud to the engineering-level entity model is realized, and the method is adaptive to multiple industrial application scenes.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics, reverse engineering and additive manufacturing, specifically to a three-dimensional point cloud geometric modeling system and method with multimodal fusion and physical constraints. Background Technology

[0002] 3D reconstruction technology has wide applications in industrial inspection, cultural relic digitization, customized manufacturing, and robot vision. Its core is converting 3D point cloud data into usable geometric models. Existing 3D reconstruction technologies (such as Poisson reconstruction, rolling sphere method, and greedy projection triangulation) have four main shortcomings. First, they lack entity attributes, resulting in models that are mostly "empty shells" of surface triangular meshes, lacking internal entity attributes (material density, anisotropy) and boundary representations, making them unsuitable for direct use in engineering simulation (CAE) or computer-aided manufacturing (CAM). Second, they lose key features; traditional methods easily lose key geometric features such as sharp edges, corners, and precision threads during noise smoothing, leading to discrepancies between the model and the actual object. The core shortcomings of the traditional Poisson reconstruction algorithm are particularly prominent: ① It uses a single-scale implicit function fitting, resulting in poor adaptability to non-uniform density point clouds, easily overfitting in high-density areas and blurring features in low-density areas; ② It has a high smoothing priority. Regarding feature preservation, even slight noise can cause sharp edges and corners to be smoothed out, resulting in reconstruction errors exceeding ±0.05mm for precision structures of industrial parts (such as cutting edges and thread crests); ③ It fails to differentiate between surface types (smooth surfaces / feature surfaces), uniformly using fixed smoothing parameters, making it impossible to balance global smoothness with local feature integrity; ④ Semantic understanding is lacking, the reconstruction process lacks semantic recognition capabilities, and cannot automatically distinguish standard features such as holes, slots, planes, and cylinders, requiring manual annotation; ⑤ The process has low automation, requiring extensive manual interaction (such as CAD software repair and solidification operations) from point cloud to solid model, leading to fragmented processes and low efficiency. Therefore, there is an urgent need to design an automated, high-fidelity 3D reconstruction method and system capable of generating complete engineering-meaning solid models to overcome the practical limitations of traditional technologies. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a three-dimensional point cloud geometric modeling system and method with multimodal fusion and physical constraints, aiming to solve the problems in the background technology.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a 3D point cloud geometric modeling system based on multimodal fusion and physical constraints, comprising: The point cloud preprocessing module is used to receive raw 3D point cloud data and perform denoising and downsampling preprocessing operations; The semantic segmentation and recognition module is connected to the data output end of the point cloud preprocessing module. It is used to perform semantic segmentation and instance segmentation on the preprocessed 3D point cloud data using a deep learning network, identify and classify regular geometric feature point cloud clusters, freeform surface feature point cloud clusters and standard part point cloud clusters, and assign semantic labels to each point cloud cluster. The hybrid geometry reconstruction module interfaces with the labeled point cloud cluster output of the semantic segmentation and recognition module. It has built-in regular feature fitting unit, free surface reconstruction unit and standard part matching unit, which respectively implement parametric B-Rep feature fitting, feature-preserving implicit surface reconstruction and standard part library matching and replacement for different types of point cloud clusters, to complete the independent geometry reconstruction of each component. The multi-component assembly and physical constraint solving module interfaces with the component model output of the hybrid geometry reconstruction module. It is used to reconstruct the assembly relationship between parts based on the geometric shape and relative position of each component. It performs manufacturability, assemblability and functional constraint verification through the physical constraint solver, and automatically corrects the model that violates the constraints. The solid model output module interfaces with the verified model output end of the multi-part assembly and physical constraint solving module to output solid geometric models in B-Rep format, procedural history tree format, and printable solid mesh format.

[0005] Furthermore, the deep learning network used in the semantic segmentation and recognition module is the PointNet++ deep learning network, and the semantic labels assigned to each point cloud cluster include composite labels of feature attributes and functional attributes. The composite labels include planar mounting surface, cylindrical through hole, freeform surface appearance surface, thread, bolt, and bearing.

[0006] Furthermore, the regular feature fitting unit uses an optimization algorithm to fit CAD features with precise parameters and topological relationships, thereby achieving parametric reconstruction of regular geometric features; The freeform surface reconstruction unit is based on an improved Poisson reconstruction algorithm, which constructs an implicit function that integrates global smoothness and local feature preservation through multi-scale point cloud decomposition, sharp edge candidate point detection, dynamic smoothing factor adjustment, and feature constraint embedding.

[0007] Furthermore, the freeform surface reconstruction unit is based on an improved Poisson reconstruction algorithm. This algorithm constructs a latent function that integrates global smoothness and local feature preservation through multi-scale point cloud decomposition, sharp-edge candidate point detection, dynamic smoothing factor adjustment, and feature constraint embedding. The specific process is as follows: Freeform surface point cloud preprocessing; A secondary purification process is carried out on the point cloud clusters labeled "freeform surface_appearance surface" after semantic segmentation. A radius filter with a neighborhood radius of 0.03 mm and a minimum number of neighborhood points of 10 is used to remove isolated noise points and eliminate the interference of noise on subsequent feature detection. At the same time, the normal vector of each point is calculated by the covariance matrix eigenvalue decomposition method, and the direction of the normal vector is adjusted through global consistency to ensure that the normal vectors all point to the outside of the surface. Multi-scale point cloud decomposition; Based on the preprocessed qualified point cloud clusters, the Gaussian pyramid decomposition strategy is used to decompose them into associated point clouds with three scales: fine layer, intermediate layer and coarse layer. Point clouds at each layer are correlated by downsampling to voxel sizes corresponding to scale values ​​and upsampling by interpolation. Sharp-edge candidate point detection; Based on the obtained fine-grained point cloud, candidate points for sharp edges are selected using a dual criterion of "curvature threshold + normal vector mutation". The specific process is as follows: Curvature calculation: For each point in the fine-layer point cloud, calculate its... The covariance matrix of nearest neighbors, through eigenvalues Calculate curvature value Set curvature threshold Points with curvature > 0.5 are classified as high curvature points; This represents the largest eigenvalue of the covariance matrix; Represents the intermediate eigenvalues ​​of the covariance matrix; This represents the smallest eigenvalue of the covariance matrix; Normal vector mutation detection: Calculate the current point and... The angle between the normal vectors of the nearest neighboring points is used to determine points where the normal vector changes abruptly if the angle is greater than 30°. Sharp edge candidate point determination: Points that simultaneously satisfy the conditions of high curvature point and normal vector abrupt change point are marked as sharp edge candidate points, and their spatial coordinates, normal vector and point cloud density of the region are recorded simultaneously. Dynamic smoothing factor and feature protection weight settings; Based on the selected candidate points with sharp edges and the characteristics of the point cloud regions, differentiated parameters are assigned to point clouds in different regions to balance surface smoothness and protection of sharp edge features: Smoothing factor Smooth regions are the regions where non-sharp-edge candidate points are located. The value ranges from 0.3 to 0.5, representing the region where the candidate points for sharp edges are located. The value is set to 0.8~1.0 to weaken the smoothing effect in sharp edge regions and preserve features; Feature protection weight Assigning candidate points to sharp edges The weights of ordinary points are set to 1.0, thereby increasing the constraint priority of sharp-edge regions in implicit function iteration; Adaptive adjustment based on point cloud noise level: When the noise percentage is >5%, smooth the region. Reduced to 0.3, sharp edge area Increased to 1.5; when noise percentage is <2%, smooth area Increased to 0.5, sharp edge area Maintain a value of 1.2 to ensure parameters are adapted to different noise conditions; Iterative reconstruction of implicit functions in constraint embedding; Based on the global contour constructed from the coarse-layer point cloud, the implicit function is iteratively optimized by integrating multi-scale point cloud information and set sharp edge constraints. The surface with an implicit function value of 0 is the target freeform surface. The process of representing the three-dimensional coordinates of a point in space is as follows: Implicit function initialization: The implicit function is constructed using the traditional Poisson reconstruction method, taking the normal vector field of the rough layer point cloud as input to initialize the global implicit function. ; Multi-scale iterative fusion: First, substitute the normal vector information of the intermediate layer point cloud into... After 5 iterations, the implicit function that optimizes global smoothness is obtained. Then, substitute the normal vector information of the fine-grained point cloud with the constraints of the sharp-edge candidate points into the equation. After 10 iterations, an implicit function that preserves local sharp edge features is obtained. ; Forced constraint satisfaction: During the iteration process, the implicit function gradient direction of candidate points with sharp edges is forced to be consistent with the normal vector of the corresponding point, i.e. , Representing implicit functions gradient, The normal vector of the candidate point of the sharp edge; Iteration termination condition: The L2 norm of the implicit function difference between two consecutive iterations is less than 10. -6 The iteration will terminate when the number of iterations reaches 20. Surface extraction and optimization; From implicit functions The triangular mesh surface is extracted and post-processing optimization is performed, including redundant face removal, sharp edge enhancement, and watertightness repair, to complete the reconstruction of the freeform surface.

[0008] Furthermore, the standard parts matching unit in the hybrid geometry reconstruction module has a built-in preset parametric CAD parts library, which includes ISO and GB standard parts. By extracting the size and shape features of the point cloud clusters, the best native CAD model is matched in the parts library and the point cloud clusters are replaced with standard CAD models.

[0009] Furthermore, the physical constraint solver in the multi-component assembly and physical constraint solving module has built-in constraint types including: manufacturability constraints with wall thickness not lower than the lower limit of 3D printing or casting processes, assemblability constraints with no interference between parts, and functional constraints with through holes penetrating parts.

[0010] Furthermore, the specific types of multi-format models output by the entity model output module are: B-Rep format, procedural history tree format, and printable entity mesh.

[0011] A method for geometric modeling 3D point clouds based on multimodal fusion and physical constraints is implemented using a 3D point cloud geometric modeling system, including the following steps: Step S1: Obtain the original 3D point cloud data and input the original 3D point cloud data into the point cloud preprocessing module. Perform denoising and downsampling operations in sequence to improve the quality of the point cloud data. The denoising uses a statistical filtering algorithm, and the downsampling uses a voxel grid sampling algorithm. Step S2: Input the preprocessed 3D point cloud data into the semantic segmentation and recognition module. Through the PointNet++ deep learning network, semantic segmentation and instance segmentation are carried out simultaneously to identify regular geometric feature point cloud clusters, freeform surface feature point cloud clusters and standard part point cloud clusters. Among them, the regular geometric feature point cloud clusters include planes, cylinders and cones. Each point cloud cluster is assigned a corresponding semantic label. Step S3: Based on the point cloud clusters with semantic labels, start the hybrid geometry reconstruction module to perform targeted reconstruction: For regular geometric feature point cloud clusters, use parametric B-Rep fitting and optimize the parameters using the least squares method to obtain accurate parametric CAD features; for free-form surface feature point cloud clusters, use an improved Poisson reconstruction algorithm and introduce feature preservation constraints to complete implicit surface reconstruction; for standard part point cloud clusters, match the best model in the preset parametric CAD parts library and complete model replacement. Step S4: Based on the relative positions of each model reconstructed in Step S3 in the original 3D point cloud data and the assigned semantic labels, the assembly relationship between the parts is reconstructed based on the multi-part assembly and physical constraint solving module. The assembly relationship includes fitting, alignment, and coaxiality. The reconstructed assembly relationship is constrained and verified by the physical constraint solver. If the verification result is a violation of the constraint, the constraint solving and model correction loop is started to adjust the model parameters until the assembly constraints are met. Step S5: Output the overall model that passed the verification in step S4 as a multi-format solid geometric model through the solid model output module.

[0012] An electronic device includes a processor, a memory, and a bus, wherein the processor and the memory are connected via the bus, the memory is used to store a set of program code, and the processor is used to call the program code stored in the memory to execute a three-dimensional point cloud geometric modeling method based on multimodal fusion and physical constraints.

[0013] A non-volatile computer storage medium stores computer-executable instructions that execute a three-dimensional point cloud geometric modeling method based on multimodal fusion and physical constraints.

[0014] Compared with existing technologies, the present invention has the following advantages: (1) This invention innovates the entire process of point cloud preprocessing, semantic segmentation and recognition, hybrid geometric reconstruction, physical constraint solving and multi-format output. First, the PointNet++ network is used to achieve dual segmentation of point cloud semantics and instances and assign geometric-functional composite labels. Then, the improved Poisson reconstruction takes into account the global smoothness and sharp edge features of free surface. Combined with the parameterized fitting of regular features and the matching of standard parts library, the hybrid reconstruction is completed. Finally, the physical constraint is checked and the illegal model is automatically corrected to output multiple types of engineering-usable models. This solves the defects of traditional reconstruction, such as lack of semantics, easy loss of features and lack of engineering attributes of the model. It realizes the automatic, high-fidelity and highly practical transformation of point cloud to engineering-level entity model.

[0015] (2) This invention achieves semantic and instance dual segmentation of point clouds through the PointNet++ network on the semantic segmentation and recognition module, and assigns a composite semantic label of "geometric shape + engineering function" to the point cloud cluster. It can automatically distinguish regular geometric features, free surface features and standard parts without manual labeling of feature types, which greatly improves the automation and accuracy of point cloud classification.

[0016] (3) In this invention, the freeform surface reconstruction unit adopts an improved Poisson reconstruction algorithm. Through multi-scale point cloud decomposition, accurate detection of sharp edge candidate points and dynamic parameter constraints, it achieves both global smoothness and local sharp edge features of the freeform surface, solves the technical defect of easy smoothing of key geometric features in traditional Poisson reconstruction, and improves the fidelity of freeform surface reconstruction. By setting up a multi-part assembly and physical constraint solving module, the assembly relationship between parts can be automatically reconstructed and manufacturability, assemblability and functional constraint verification can be completed. For non-compliant models, the correction loop can be automatically started, forming a closed loop from point cloud reconstruction to assembly constraint verification, ensuring the engineering practicality of the output model and reducing the workload of subsequent manual adjustment. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the solid geometry model generation process of the present invention.

[0018] Figure 2 This is a schematic diagram of the improved Poisson reconstruction algorithm of the present invention.

[0019] Figure 3 This is a schematic diagram illustrating the point cloud to B-Rep feature generation of the present invention. Detailed Implementation

[0020] like Figure 1 As shown, the present invention provides a technical solution: a 3D point cloud geometric modeling system based on multimodal fusion and physical constraints, comprising: The point cloud preprocessing module is used to receive raw 3D point cloud data and perform denoising and downsampling preprocessing operations; The semantic segmentation and recognition module is connected to the data output end of the point cloud preprocessing module. It is used to perform semantic segmentation and instance segmentation on the preprocessed 3D point cloud data using a deep learning network, identify and classify regular geometric feature point cloud clusters, freeform surface feature point cloud clusters and standard part point cloud clusters, and assign semantic labels to each point cloud cluster. The hybrid geometry reconstruction module interfaces with the labeled point cloud cluster output of the semantic segmentation and recognition module. It has built-in regular feature fitting unit, free surface reconstruction unit and standard part matching unit, which respectively implement parametric B-Rep feature fitting, feature-preserving implicit surface reconstruction and standard part library matching and replacement for different types of point cloud clusters, to complete the independent geometry reconstruction of each component. The multi-component assembly and physical constraint solving module interfaces with the component model output of the hybrid geometry reconstruction module. It is used to reconstruct the assembly relationship between parts based on the geometric shape and relative position of each component. It performs manufacturability, assemblability and functional constraint verification through the physical constraint solver, and automatically corrects the model that violates the constraints. The solid model output module interfaces with the verified model output end of the multi-part assembly and physical constraint solving module to output solid geometric models in B-Rep format, procedural history tree format, and printable solid mesh format.

[0021] The semantic segmentation and recognition module uses the PointNet++ deep learning network, and the semantic labels assigned to each point cloud cluster include composite labels of feature attributes and functional attributes. The composite labels specifically include plane_mounting surface, cylinder_through hole, freeform surface_appearance surface, thread, bolt, bearing, etc.

[0022] like Figure 2 As shown, the regular feature fitting unit uses optimization algorithms such as the least squares method to fit CAD features with precise parameters (radius, height, angle) and topological relationships, thereby realizing the parametric reconstruction of regular geometric features; The freeform surface reconstruction unit is based on an improved Poisson reconstruction algorithm. This algorithm improves upon this algorithm through four core improvements: multi-scale point cloud decomposition, sharp-edge candidate point detection, dynamic smoothing factor adjustment, and feature constraint embedding. It constructs a latent function that integrates global smoothness and local feature preservation. The specific process is as follows: 1. Preprocessing of freeform surface point clouds; A secondary purification process was performed on the point cloud clusters labeled "freeform surface_appearance surface" after semantic segmentation. A radius filter with a neighborhood radius of 0.03 mm and a minimum number of neighborhood points of 10 was used to remove isolated noise points and eliminate the interference of noise on subsequent feature detection. At the same time, the normal vector of each point was calculated by the covariance matrix eigenvalue decomposition method, and the direction of the normal vector was adjusted through global consistency to ensure that the normal vectors all point to the outside of the surface, providing qualified basic point cloud data for subsequent multi-scale decomposition and implicit function construction.

[0023] 2. Multi-scale point cloud decomposition; Based on the preprocessed qualified point cloud clusters, a Gaussian pyramid decomposition strategy is used to decompose them into a three-level scale of associated point clouds: The fine layer (scale σ1=0.02mm) preserves the fine features of the original point cloud and is used to capture local structures such as sharp edges and small protrusions. The intermediate layer (scale σ2=0.05mm) uses Gaussian filtering to smooth out minor noise and balance the overall surface shape. A rough layer (scale σ3=0.1mm) significantly reduces point cloud density, enabling rapid construction of global surface contours to improve algorithm efficiency; Point clouds at each layer are correlated by downsampling to voxel sizes corresponding to the scale values ​​and upsampling to complete the data through interpolation, ensuring the consistency of point cloud information at multiple scales and providing multi-dimensional data support for subsequent sharp edge detection and implicit function iteration.

[0024] 3. Sharp-edge candidate point detection; Based on the obtained fine-grained point cloud, candidate points for sharp edges are selected using a dual criterion of "curvature threshold + normal vector mutation". The specific process is as follows: Curvature calculation: For each point in the fine-layer point cloud, calculate its... Neighbors ( The covariance matrix of ) is obtained through eigenvalues. Calculate curvature value Set curvature threshold Points with curvature > 0.5 are classified as high curvature points; It represents the largest eigenvalue of the covariance matrix, corresponding to the degree of dispersion of the nearest neighbor points in the "most dispersed direction" (main direction). The larger the value, the more dispersed the distribution of the nearest neighbor points in this direction. It represents the intermediate eigenvalue of the covariance matrix, corresponding to the degree of dispersion of the nearest neighbor points in the "secondary dispersion direction", reflecting the extension of the region in the secondary direction; The smallest eigenvalue of the covariance matrix corresponds to the degree of dispersion of the nearest neighbor points along the "most concentrated direction". The smaller the value, the more compact the distribution of the nearest neighbor points in this direction (which also means that the curvature of the region may be higher). Normal vector mutation detection: Calculate the current point and... The angle between the normal vectors of the nearest neighboring points is used to determine points where the normal vector changes abruptly if the angle is greater than 30°. Sharp edge candidate point determination: Points that simultaneously satisfy the conditions of high curvature point and normal vector abrupt change point are marked as sharp edge candidate points, and their spatial coordinates, normal vector and point cloud density of the region are recorded simultaneously.

[0025] 4. Dynamic smoothing factor and feature protection weight settings; Based on the selected candidate points with sharp edges and the characteristics of the point cloud regions, differentiated parameters are assigned to point clouds in different regions to balance surface smoothness and protection of sharp edge features: Smoothing factor Smooth region (region where non-sharp edge candidate points are located) The value ranges from 0.3 to 0.5, representing the region where the candidate points for sharp edges are located. The value is set to 0.8~1.0 to weaken the smoothing effect in sharp edge regions and preserve features;

[0026] Feature protection weight Assigning candidate points to sharp edges The weights of ordinary points are set to 1.0, thereby increasing the constraint priority of sharp-edge regions in implicit function iteration; The above parameters can be adaptively adjusted according to the point cloud noise level: when the noise percentage is >5%, the smoothing region... Reduced to 0.3, sharp edge area Increased to 1.5; when noise percentage is <2%, smooth area It can be increased to 0.5, sharp edge area Keep it at 1.2 to ensure that the parameters are adapted to different noise conditions.

[0027] 5. Implicit function iterative reconstruction based on constraint embedding; Based on the global contour constructed from the coarse-layer point cloud, the implicit function is iteratively optimized by integrating multi-scale point cloud information and set sharp edge constraints. (The surface with an implicit function value of 0 is the target freeform surface). The process of representing the three-dimensional coordinates of a point in space is as follows: Implicit function initialization: The implicit function is constructed using the traditional Poisson reconstruction method, taking the normal vector field of the rough layer point cloud as input to initialize the global implicit function. ; Multi-scale iterative fusion: First, substitute the normal vector information of the intermediate layer point cloud into... After 5 iterations, the implicit function that optimizes global smoothness is obtained. Then, the normal vector information of the fine-layer point cloud is combined with the constraints of the sharp-edge candidate points (fixed normal vector direction, feature protection weights). Substitute After 10 iterations, an implicit function that preserves local sharp edge features is obtained. ; Forced constraint satisfaction: During the iteration process, the implicit function gradient direction of candidate points with sharp edges is forced to be consistent with the normal vector of that point, i.e. ( Representing implicit functions gradient, (as the normal vector of candidate points for sharp edges), to avoid over-smoothing of sharp edges; Iteration termination condition: The L2 norm of the implicit function difference between two consecutive iterations is less than 10. -6 The iteration will terminate when the number of iterations reaches 20.

[0028] 6. Surface extraction and optimization; Using the MarchingCubes algorithm to extract implicit functions Extract the triangular mesh surface and perform post-processing optimization: Redundant facet removal: Remove tiny faces with an area <0.001mm² to reduce model redundancy; Sharp edge reinforcement: For the triangular facets where the candidate points of sharp edges are located, adjust the facet normal vectors so that the angle between the normal vectors of the two facets of the sharp edge is ≤5° from the actual object. Water tightness repair: A boundary expansion-based triangulation hole filling algorithm is used to repair the tiny holes generated during the extraction process, ensuring that the water tightness of the surface reaches more than 99.5%, and completing the high-precision reconstruction of the free surface.

[0029] The standard parts matching unit in the hybrid geometry reconstruction module has a built-in preset parametric CAD parts library. This parts library contains ISO and GB standard parts. By extracting the size and shape features of point cloud clusters, it can match the best native CAD model in the parts library and complete the replacement of point cloud clusters with standard CAD models, thereby improving the accuracy and efficiency of standard parts reconstruction.

[0030] The physical constraint solver in the multi-component assembly and physical constraint solving module has built-in constraint types including: manufacturability constraints with wall thickness not lower than the lower limit of 3D printing or casting processes, assemblability constraints with no interference between parts, and functional constraints with through holes penetrating parts. The engineering adaptability of the model is ensured through the comprehensive verification of the three types of constraints.

[0031] The specific types of multi-format models output by the entity model output module are as follows: B-Rep format includes STEP and IGES general engineering formats; procedural history tree format includes custom JSON / XML editable formats; and printable entity mesh is a high-precision mesh with watertight properties and internal voxel information to meet the application needs of different engineering scenarios.

[0032] A method for geometric modeling 3D point clouds based on multimodal fusion and physical constraints includes the following steps: Step S1: Obtain the original 3D point cloud data and input the original 3D point cloud data into the point cloud preprocessing module. Perform denoising and downsampling operations in sequence to improve the quality of the point cloud data. The denoising adopts a statistical filtering algorithm, and the downsampling adopts a voxel grid sampling algorithm.

[0033] Step S2: Input the preprocessed 3D point cloud data into the semantic segmentation and recognition module. Through the PointNet++ deep learning network, semantic segmentation and instance segmentation are carried out simultaneously to identify regular geometric feature point cloud clusters, freeform surface feature point cloud clusters, and standard part point cloud clusters. Among them, the regular geometric feature point cloud clusters include planes, cylinders, and cones. Each point cloud cluster is assigned a corresponding semantic label.

[0034] Step S3: Based on the point cloud clusters with semantic labels, the hybrid geometry reconstruction module is activated to perform targeted reconstructions: For regular geometric feature point cloud clusters, a parametric B-Rep fitting method is used, and parameter optimization is performed using the least squares method to obtain accurate parametric CAD features; for free-form surface feature point cloud clusters, an improved Poisson reconstruction algorithm is used, while introducing feature preservation constraints to complete implicit surface reconstruction; for standard part point cloud clusters, the best model is matched in the preset parametric CAD parts library and the model is replaced. The generated parametric B-Rep is as follows: Figure 3 As shown.

[0035] Step S4: Based on the relative positions of each model reconstructed in Step S3 in the original 3D point cloud data, and the assigned semantic labels, the assembly relationship between parts is reconstructed based on the multi-part assembly and physical constraint solving module. The assembly relationship includes fitting, alignment, and coaxiality. The reconstructed assembly relationship is constrained and verified by the physical constraint solver. If the verification result is a violation of the constraint, the constraint solving and model correction loop is started to adjust the model parameters until the assembly constraints are met.

[0036] Step S5: Through the solid model output module, output the overall model that has passed the verification in step S4 as a multi-format solid geometric model; including B-Rep format that can be imported into CAD / CAE / CAM software, procedural history tree format that can be edited afterward, and solid mesh format for 3D printing.

[0037] An electronic device includes a processor, a memory, and a bus, wherein the processor and the memory are connected via the bus, the memory is used to store a set of program code, and the processor is used to call the program code stored in the memory to execute a three-dimensional point cloud geometric modeling method based on multimodal fusion and physical constraints.

[0038] A non-volatile computer storage medium stores computer-executable instructions that execute a three-dimensional point cloud geometric modeling method based on multimodal fusion and physical constraints.

[0039] Before implementation, this invention requires hardware and software. The hardware includes a laser scanner (accuracy ±0.01mm) and a computer (CPU i7-13700K, GPU RTX 4090). The software includes PointNet++ network (based on the PyTorch framework), CloudCompare (point cloud preprocessing), and SolidWorks (model verification). The laser scanner is connected to the computer, and the laser scanner is used to scan a 3D object. A 3D point cloud of the scanned object is generated in the computer, and the generated 3D point cloud is used to create a solid geometric model using the technical solution of this invention.

[0040] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A 3D point cloud geometric modeling system based on multimodal fusion and physical constraints, characterized in that, include: The point cloud preprocessing module is used to receive raw 3D point cloud data and perform noise reduction and downsampling preprocessing operations; The semantic segmentation and recognition module is connected to the data output end of the point cloud preprocessing module. It is used to perform semantic segmentation and instance segmentation on the preprocessed 3D point cloud data using a deep learning network, identify and classify regular geometric feature point cloud clusters, freeform surface feature point cloud clusters and standard part point cloud clusters, and assign semantic labels to each point cloud cluster. The hybrid geometry reconstruction module interfaces with the labeled point cloud cluster output of the semantic segmentation and recognition module. It has built-in regular feature fitting unit, free surface reconstruction unit and standard part matching unit, which respectively implement parametric B-Rep feature fitting, feature-preserving implicit surface reconstruction and standard part library matching and replacement for different types of point cloud clusters, to complete the independent geometry reconstruction of each component. The multi-component assembly and physical constraint solving module interfaces with the component model output of the hybrid geometry reconstruction module. It is used to reconstruct the assembly relationship between parts based on the geometric shape and relative position of each component. It performs manufacturability, assemblability and functional constraint verification through the physical constraint solver, and automatically corrects the model that violates the constraints. The solid model output module interfaces with the verified model output end of the multi-part assembly and physical constraint solving module to output solid geometric models in B-Rep format, procedural history tree format, and printable solid mesh format.

2. The 3D point cloud geometric modeling system with multimodal fusion and physical constraints according to claim 1, characterized in that: The semantic segmentation and recognition module uses the PointNet++ deep learning network, and the semantic labels assigned to each point cloud cluster include composite labels of feature attributes and functional attributes. The composite labels include plane_mounting surface, cylinder_through hole, freeform surface_appearance surface, thread, bolt, and bearing.

3. The 3D point cloud geometric modeling system with multimodal fusion and physical constraints according to claim 2, characterized in that: The regular feature fitting unit uses an optimization algorithm to fit CAD features with accurate parameters and topological relationships, thereby realizing the parametric reconstruction of regular geometric features. The freeform surface reconstruction unit is based on an improved Poisson reconstruction algorithm, which constructs an implicit function that integrates global smoothness and local feature preservation through multi-scale point cloud decomposition, sharp edge candidate point detection, dynamic smoothing factor adjustment, and feature constraint embedding.

4. The 3D point cloud geometric modeling system with multimodal fusion and physical constraints according to claim 3, characterized in that: The freeform surface reconstruction unit is based on an improved Poisson reconstruction algorithm. This algorithm constructs a latent function that integrates global smoothness and local feature preservation through multi-scale point cloud decomposition, sharp-edge candidate point detection, dynamic smoothing factor adjustment, and feature constraint embedding. The specific process is as follows: Freeform surface point cloud preprocessing; A secondary purification process is carried out on the point cloud clusters labeled "freeform surface_appearance surface" after semantic segmentation. A radius filter with a neighborhood radius of 0.03 mm and a minimum number of neighborhood points of 10 is used to remove isolated noise points and eliminate the interference of noise on subsequent feature detection. At the same time, the normal vector of each point is calculated by the covariance matrix eigenvalue decomposition method, and the direction of the normal vector is adjusted through global consistency to ensure that the normal vectors all point to the outside of the surface. Multi-scale point cloud decomposition; Based on the preprocessed qualified point cloud clusters, the Gaussian pyramid decomposition strategy is used to decompose them into associated point clouds with three scales: fine layer, intermediate layer and coarse layer. Point clouds at each layer are correlated by downsampling to voxel sizes corresponding to scale values ​​and upsampling by interpolation. Sharp-edge candidate point detection; Based on the obtained fine-layer point cloud, candidate points for sharp edges are selected using a dual criterion of "curvature threshold + normal vector mutation". The specific process is as follows: Curvature calculation: For each point in the fine-layer point cloud, calculate its... The covariance matrix of nearest neighbors, through eigenvalues Calculate curvature value Set curvature threshold Points with curvature > 0.5 are classified as high curvature points; This represents the largest eigenvalue of the covariance matrix; Represents the intermediate eigenvalues ​​of the covariance matrix; This represents the smallest eigenvalue of the covariance matrix; Normal vector mutation detection: Calculate the current point and... The angle between the normal vectors of the nearest neighboring points is used to determine points where the normal vector changes abruptly if the angle is greater than 30°. Sharp edge candidate point determination: Points that simultaneously satisfy the conditions of high curvature point and normal vector abrupt change point are marked as sharp edge candidate points, and their spatial coordinates, normal vector and point cloud density of the region are recorded simultaneously. Dynamic smoothing factor and feature protection weight settings; Based on the selected candidate points with sharp edges and the characteristics of the point cloud regions, differentiated parameters are assigned to point clouds in different regions to balance surface smoothness and protection of sharp edge features: Smoothing factor Smooth regions are the regions where non-sharp-edge candidate points are located. The value ranges from 0.3 to 0.5, representing the region where the candidate points for sharp edges are located. The value is set to 0.8~1.0 to weaken the smoothing effect in sharp edge regions and preserve features; Feature protection weight Assigning candidate points to sharp edges The weights of ordinary points are set to 1.0, thereby increasing the constraint priority of sharp-edge regions in implicit function iteration; Adaptive adjustment based on point cloud noise level: When the noise percentage is >5%, smooth the region. Reduced to 0.3, sharp edge area Increased to 1.5; when noise percentage is <2%, smooth area Increased to 0.5, sharp edge area Maintain a value of 1.2 to ensure parameters are adapted to different noise conditions; Iterative reconstruction of implicit functions in constraint embedding; Based on the global contour constructed from the coarse-layer point cloud, the implicit function is iteratively optimized by integrating multi-scale point cloud information and set sharp edge constraints. The surface with an implicit function value of 0 is the target freeform surface. The process of representing the three-dimensional coordinates of a point in space is as follows: Implicit function initialization: The implicit function is constructed using the traditional Poisson reconstruction method, taking the normal field of the rough layer point cloud as input to initialize the global implicit function. ; Multi-scale iterative fusion: First, substitute the normal vector information of the intermediate layer point cloud into... After 5 iterations, the implicit function that optimizes global smoothness is obtained. ; Then, substitute the normal vector information of the fine-layer point cloud with the constraints of the sharp-edge candidate points into the equation. After 10 iterations, an implicit function that preserves local sharp edge features is obtained. ; Forced constraint satisfaction: During the iteration process, the implicit function gradient direction of candidate points with sharp edges is forced to be consistent with the normal vector of the corresponding point, i.e. , Representing implicit functions gradient, The normal vector of the candidate point of the sharp edge; Iteration termination condition: The L2 norm of the implicit function difference between two consecutive iterations is less than 10. -6 The iteration will terminate when the number of iterations reaches 20. Surface extraction and optimization; From implicit functions The triangular mesh surface is extracted and post-processing optimization is performed, including redundant face removal, sharp edge enhancement, and watertightness repair, to complete the reconstruction of the freeform surface.

5. The 3D point cloud geometric modeling system with multimodal fusion and physical constraints according to claim 4, characterized in that: The standard parts matching unit in the hybrid geometry reconstruction module has a built-in preset parametric CAD parts library, which includes ISO and GB standard parts. By extracting the size and shape features of the point cloud clusters, it matches the best native CAD model in the parts library and completes the replacement of the point cloud clusters with the standard CAD model.

6. The 3D point cloud geometric modeling system with multimodal fusion and physical constraints according to claim 5, characterized in that: The physical constraint solver in the multi-component assembly and physical constraint solving module has built-in constraint types including: manufacturability constraints with wall thickness not lower than the lower limit of 3D printing or casting processes, assemblability constraints with no interference between parts, and functional constraints with through holes penetrating parts.

7. The 3D point cloud geometric modeling system with multimodal fusion and physical constraints according to claim 6, characterized in that: The specific types of multi-format models output by the entity model output module are: B-Rep format, procedural history tree format, and printable entity mesh.

8. A method for geometric modeling three-dimensional point clouds based on multimodal fusion and physical constraints, implemented based on the three-dimensional point cloud geometric modeling system based on multimodal fusion and physical constraints as described in any one of claims 1-7, characterized in that, Includes the following steps: Step S1: Obtain the original 3D point cloud data, input the original 3D point cloud data into the point cloud preprocessing module, and perform denoising and downsampling operations in sequence to improve the quality of the point cloud data; The noise reduction uses a statistical filtering algorithm, and the downsampling uses a voxel grid sampling algorithm. Step S2: Input the preprocessed 3D point cloud data into the semantic segmentation and recognition module. Through the PointNet++ deep learning network, semantic segmentation and instance segmentation are carried out simultaneously to identify regular geometric feature point cloud clusters, freeform surface feature point cloud clusters and standard part point cloud clusters. Among them, the regular geometric feature point cloud clusters include planes, cylinders and cones. Each point cloud cluster is assigned a corresponding semantic label. Step S3: Based on the point cloud clusters with semantic labels, start the hybrid geometry reconstruction module to perform targeted reconstruction: For regular geometric feature point cloud clusters, use parametric B-Rep fitting and optimize the parameters using the least squares method to obtain accurate parametric CAD features; for free-form surface feature point cloud clusters, use an improved Poisson reconstruction algorithm and introduce feature preservation constraints to complete implicit surface reconstruction; for standard part point cloud clusters, match the best model in the preset parametric CAD parts library and complete model replacement. Step S4: Based on the relative positions of each model reconstructed in Step S3 in the original 3D point cloud data and the assigned semantic labels, the assembly relationship between the parts is reconstructed based on the multi-part assembly and physical constraint solving module. The assembly relationship includes fitting, alignment, and coaxiality. The reconstructed assembly relationship is constrained and verified by the physical constraint solver. If the verification result is a violation of the constraint, the constraint solving and model correction loop is started to adjust the model parameters until the assembly constraints are met. Step S5: Output the overall model that passed the verification in step S4 as a multi-format solid geometric model through the solid model output module.

9. An electronic device, characterized in that, It includes a processor, a memory, and a bus, wherein the processor and the memory are connected via the bus, wherein the memory is used to store a set of program code, and the processor is used to call the program code stored in the memory to execute the three-dimensional point cloud geometric modeling method of multimodal fusion and physical constraints as described in claim 8.

10. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer can execute instructions to perform the three-dimensional point cloud geometric modeling method with multimodal fusion and physical constraints as described in claim 8.

Images