Mold three-dimensional high-precision modeling method and system based on multi-modal data fusion

Abstract

The invention relates to the technical field of three-dimensional model design of molds, and provides a mold three-dimensional high-precision modeling method and system based on multi-modal data fusion, and the method comprises the steps: collecting an RGB image, a high dynamic range image and a structured light depth image of an entity mold, and carrying out the reconstruction to obtain a globally consistent dense point cloud; denoising and semantic segmentation are carried out on the point cloud, and a cavity function area, a core function area and a runner function area are recognized; performing differential parametric modeling based on an identification result: performing geometric primitive fitting on regular geometric features, performing NURBS curved surface reconstruction on a free-form surface, and performing swept volume or rotator fitting on a flow channel to obtain a parameterized geometry, a parameterized curved surface and a parameterized channel model; and applying boundary continuity constraint to fuse the parameterized geometry, the parameterized curved surface and the parameterized channel model, and constructing a hybrid parameterized three-dimensional model. According to the method, rapid reconstruction of a high-precision, high-semantic and full-parameterized three-dimensional model of an entity mold can be realized.

Description

Technical Field

[0001] This invention relates to the fields of 3D model design and image recognition of molds, and in particular to a high-precision 3D modeling method and system for molds based on multimodal data fusion. Background Technology

[0002] As a key process equipment in industrial production, the design and manufacturing precision of molds directly determines the quality of the final product. With the manufacturing industry moving towards intelligent and digital transformation, design, simulation, and manufacturing based on 3D digital models have become industry standards. In this context, the ability to quickly and accurately obtain editable parametric CAD models of physical molds through 3D scanning is crucial for mold replication, repair, modification, and digital archiving. Currently, conventional 3D modeling techniques face numerous technical challenges when dealing with molds with complex surfaces and mixed geometric features (such as the coexistence of free-form surfaces and regular structures). First, reconstruction accuracy is insufficient. The surface of physical molds often contains diffuse reflection areas, highly reflective areas (such as polished cavities), and complex structures such as deep grooves and narrow slits. Single scanning technologies (such as pure optical scanning or pure structured light scanning) struggle to maintain high accuracy across all scenarios. Existing technologies often employ a single data source, or, while using multiple data sources, lack an effective fusion and reconstruction mechanism, resulting in voids in the reconstructed point cloud in reflective areas and loss of detail in deep grooves, failing to meet the high-precision requirements of industrial applications. Second, the reconstruction is disconnected from the design intent. Conventional 3D reconstruction processes stop at generating triangular meshes or point cloud models, which lack an understanding of the mold's functional structure. Reconstruction systems cannot automatically identify and differentiate key functional areas such as cavities, cores, and runners. Therefore, subsequent parametric modeling heavily relies on manual interaction for region division and feature recognition, a cumbersome and inefficient process prone to subjective errors. Furthermore, mold models typically include free-form surfaces (such as product exterior surfaces), regular geometric features (such as positioning planes and bolt holes), and swept / rotated features (such as runners). Existing parametric methods struggle to automatically and adaptively apply optimal modeling strategies to these heterogeneous features within the same workflow, ensuring smooth boundary connections. This results in final models that require extensive manual adjustments, leading to unsatisfactory parametric levels and reusability.

[0003] Therefore, there is an urgent need in this field for a high-precision 3D modeling method for molds based on multimodal data fusion, in order to overcome the above-mentioned technical defects and realize the rapid reconstruction of high-precision, semantically rich, and fully parametric 3D models of solid molds. Summary of the Invention

[0004] To address the shortcomings of the existing technologies, this invention provides a method and system for high-precision 3D modeling of molds based on multimodal data fusion, so as to achieve rapid reconstruction of high-precision, semantically rich, and fully parametric 3D models of physical molds.

[0005] In a first aspect, the present invention provides a method for high-precision 3D modeling of molds based on multimodal data fusion, comprising: S1. The automated acquisition system is controlled to acquire multimodal image data of the physical mold according to a preset shooting sequence. The multimodal image data includes RGB images, high dynamic range images, and structured light depth images. S2. Perform three-dimensional reconstruction on the multimodal image data to obtain a globally consistent dense point cloud, and perform noise reduction and semantic segmentation on the dense point cloud to identify at least one functional region corresponding to the cavity, core and flow channel of the physical mold from the dense point cloud. S3. Based on the functional regions identified by the semantic segmentation, perform differentiated parametric modeling, which includes: for the regular geometric features in the functional regions of the cavity and the core, using geometric primitives to fit and generate parametric geometry; for the free-form surfaces in the functional regions of the cavity and the core, using NURBS surface reconstruction to generate parametric surfaces; and for the identified flow channel functional regions, using swept volume or volume of revolution to fit and generate parametric channel models. S4. Apply boundary continuity constraints to fuse the parametric geometry, the parametric surface, and the parametric channel model to construct a hybrid parametric 3D model of the solid mold.

[0006] Secondly, the present invention provides a high-precision three-dimensional modeling system for molds based on multimodal data fusion, wherein the high-precision three-dimensional modeling system for molds based on multimodal data fusion uses the above-mentioned high-precision three-dimensional modeling method for molds based on multimodal data fusion.

[0007] Compared with the prior art, the beneficial effects of this invention are as follows: This invention provides a method and system for high-precision 3D modeling of molds based on multimodal data fusion. The method includes: controlling an automated acquisition system to acquire multimodal image data of a physical mold according to a preset shooting sequence; the multimodal image data includes RGB images, high dynamic range images, and structured light depth images; performing 3D reconstruction on the multimodal image data to obtain a globally consistent dense point cloud; and performing denoising and semantic segmentation on the dense point cloud to identify at least one functional region corresponding to the cavity, core, and flow channel of the physical mold from the dense point cloud; and performing 3D reconstruction based on the functional regions identified by the semantic segmentation. The invention employs differentiated parametric modeling, which includes: generating parametric geometries by fitting geometric primitives to the regular geometric features in the functional areas of the cavity and the core; generating parametric surfaces by reconstructing free-form surfaces in the functional areas of the cavity and the core using NURBS surface modeling; generating parametric channel models by fitting swept volumes or volumes of revolution to the identified flow channel functional areas; and applying boundary continuity constraints to fuse the parametric geometries, parametric surfaces, and parametric channel models to construct a hybrid parametric 3D model of the solid mold. This invention enables rapid reconstruction of high-precision, highly semantic, and fully parametric 3D models of solid molds. Attached Figure Description

[0008] The accompanying drawings, which are provided to further illustrate the invention and constitute a part of this invention, are illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention.

[0009] Figure 1 This is a flowchart illustrating a high-precision 3D modeling method for molds based on multimodal data fusion, according to an embodiment of the present invention. Figure 2 This is another flowchart illustrating the high-precision 3D modeling method for molds based on multimodal data fusion, as described in this embodiment of the invention. Detailed Implementation

[0010] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0011] See Figures 1-2This invention provides a method and system for high-precision 3D modeling of molds based on multimodal data fusion. The system utilizes a method based on multimodal data fusion, which includes the following steps: S1. The automated acquisition system is controlled to acquire multimodal image data of the physical mold according to a preset shooting sequence. The multimodal image data includes RGB images, high dynamic range images, and structured light depth images. S2. Perform three-dimensional reconstruction on the multimodal image data to obtain a globally consistent dense point cloud, and perform noise reduction and semantic segmentation on the dense point cloud to identify at least one functional region corresponding to the cavity, core and flow channel of the physical mold from the dense point cloud. S3. Based on the functional regions identified by the semantic segmentation, perform differentiated parametric modeling, which includes: for the regular geometric features in the functional regions of the cavity and the core, using geometric primitives to fit and generate parametric geometry; for the free-form surfaces in the functional regions of the cavity and the core, using NURBS surface reconstruction to generate parametric surfaces; and for the identified flow channel functional regions, using swept volume or volume of revolution to fit and generate parametric channel models. S4. Apply boundary continuity constraints to fuse the parametric geometry, the parametric surface, and the parametric channel model to construct a hybrid parametric 3D model of the solid mold.

[0012] In this embodiment, the automated acquisition system acquires multimodal image data of the physical mold according to a preset shooting sequence. This multimodal image data includes RGB images, high dynamic range images, and structured light depth images. This addresses the challenges of single scanning techniques (such as pure optical scanning or pure structured light scanning) in maintaining high accuracy across all scenes, as well as the issues of voids in the reconstructed point cloud in reflective areas and loss of detail in deep grooves. Structured light depth images can accurately acquire the three-dimensional information of most surfaces; high dynamic range images, through multi-exposure synthesis, can effectively suppress overexposure in highly reflective areas (such as polished cavities) and preserve details; RGB images provide rich texture and color information, assisting in subsequent semantic segmentation. In this embodiment, the multimodal image data ensures the integrity and high accuracy of three-dimensional data in complex scenes ranging from diffuse reflection to high reflectivity.

[0013] Furthermore, denoising and semantic segmentation are performed on the dense point cloud to identify at least one functional region corresponding to the cavity, core, and runner of the physical mold. This solves the problems of disconnect between reconstruction and design intent, the inability of the reconstruction system to automatically identify and distinguish key functional regions of the mold, and the heavy reliance on manual interaction for region division and feature recognition in subsequent parametric modeling. The solution provided in this embodiment can automatically understand the structural composition of the mold, associating the physical point cloud with functional regions (cavity, core, runner), laying the foundation for subsequent differentiated and intelligent parametric modeling and reducing manual intervention.

[0014] Furthermore, based on the functional regions identified by the semantic segmentation, differentiated parametric modeling is performed. For regular geometric features, geometric primitives are used for fitting; for free-form surfaces, NURBS surface reconstruction is used; and for flow channels, swept volumes or volumes of revolution are used for fitting. This solves the problem that existing parametric methods struggle to automatically and adaptively employ optimal modeling strategies for these heterogeneous features (free-form surfaces, regular geometric features, swept / rotated features) within the same workflow, achieving adaptive and optimal parametric reconstruction for heterogeneous geometric features. In this embodiment, guided by semantic information, industry-standard and most suitable CAD modeling methods are used for different types of features: geometric primitives ensure the accuracy and parametric nature of regular features, NURBS ensures the smoothness and adjustability of free-form surfaces, and swept / rotated volumes efficiently and accurately reconstruct features such as flow channels, fundamentally improving the parametric quality and editability of the reconstructed model. Among them, NURBS (Non-Uniform Rational B-Splines) is a powerful and flexible mathematical model that, through control points, weights, and non-uniform node vectors, can accurately represent almost all shapes, from standard geometries (such as circles and spheres) to the most complex freeform surfaces, in a unified way. CAD refers to Computer-Aided Design.

[0015] Furthermore, by applying boundary continuity constraints to fuse the parametric geometry, parametric surfaces, and parametric channel models to construct a hybrid parametric 3D model of the solid mold, the problem of difficulty in ensuring smooth boundary connections, resulting in a final model that requires extensive manual repairs, can be solved. This allows for the automatic construction of a seamless hybrid parametric model. The method provided in this embodiment not only generates independent parametric components but also ensures smooth transitions at the junctions of different features (such as freeform surfaces and regular planes) by applying continuity constraints (e.g., G1 and G2 continuity), generating a complete, unified, and high-quality CAD model. This reduces the workload of subsequent manual repairs and enhances the model's usability and engineering value.

[0016] Furthermore, before acquiring the multimodal image dataset, a handheld spectrometer can be used to measure the material reflectivity of the surfaces of different functional areas of the physical mold. Based on the reflectivity data, image acquisition parameters are dynamically configured for each planned shooting pose in the automated acquisition system. Specifically, this may include: for highly reflective areas with reflectivity higher than a preset threshold, configuring the high dynamic range image to be acquired using a multi-frame low-exposure time series, and reducing the projector output power of the structured light depth image; for diffuse reflection areas with reflectivity lower than a preset threshold, configuring a single frame with normal exposure and increasing the projector output power. Using a handheld spectrometer to measure the material reflectivity of the surfaces of different functional areas of the physical mold, and dynamically configuring image acquisition parameters for each planned shooting pose in the automated acquisition system based on the reflectivity data, can suppress overexposure and specular reflection interference in highlight areas, while ensuring the signal-to-noise ratio of detail and depth information in diffuse reflection areas, laying a reliable data foundation for subsequent high-precision reconstruction.

[0017] Furthermore, based on the semantic segmentation results, a spatial topological relationship diagram can be constructed between the cavity, the core, and the flow channel functional areas. Nodes represent functional areas, and edges represent the adjacency relationships between areas. When performing differentiated parametric modeling, the spatial topological relationship diagram is referenced: when two adjacent functional areas are reconstructed as parametric geometry and parametric surfaces respectively, geometric symmetry constraints or concentricity constraints based on the topological relationships in the spatial topological relationship diagram are additionally applied to the boundary continuity constraints. Elevating the semantic segmentation results to the level of constructing a spatial topological relationship diagram, and using this topological knowledge to guide and constrain the subsequent parametric modeling process, ensures that the reconstructed model is not only geometrically accurate but also conforms to the design intent. For example, it can automatically ensure that symmetrically distributed reinforcing ribs on the mold are completely symmetrical, or ensure that the flow channel inlet and the cavity center remain concentric, thereby significantly reducing the workload of subsequent manual adjustments and improving the intelligence and direct usability of the reconstructed model.

[0018] Preferably, in step S1, when controlling the automated acquisition system to acquire multimodal image data of the physical mold according to a preset shooting sequence, the following steps are taken: before acquisition, multiple reference marks of known size are arranged in the space around the physical mold, and a scale reference piece with precise scale is placed; based on the evaluation of the geometric shape and complexity of the physical mold, its surface is divided into multiple acquisition zones, and a set of shooting poses covering its entire surface is planned for each zone, wherein the shooting poses of adjacent zones ensure that their fields of view have an overlap area of ​​not less than a preset value.

[0019] In this embodiment, the placement of reference markers and scale reference components of known dimensions provides a dual guarantee. Reference markers offer stable and reliable reference points for subsequent camera calibration and image stitching from different perspectives, reducing accumulated errors and ensuring the global consistency and rigidity of the entire point cloud model. Scale reference components directly endow the reconstructed model with a true physical scale, ensuring that the final generated CAD model is not merely a proportional model of relative dimensions, but a dimensionally accurate digital model that can be directly used for subsequent engineering design and manufacturing. This is crucial for industrial products like molds, which have extremely high dimensional accuracy requirements. Furthermore, by partitioning the mold surface and planning comprehensive shooting poses, the problem of data loss caused by object geometric occlusion or limited sensor field of view can be systematically solved. This planned acquisition strategy in this embodiment avoids the arbitrariness and blind spots that may exist in manual operation, ensuring that even hard-to-reach areas such as deep grooves, narrow slits, and concave cavities are effectively covered. Simultaneously, the pre-set, sufficient overlapping areas provide rich common information for subsequent multi-view matching and data fusion, effectively preventing model distortion or discontinuity caused by data stitching misalignment.

[0020] Preferably, in step S1, when acquiring the multimodal image data, the method further includes: controlling the automated acquisition system to move sequentially to each planned shooting pose, first synchronously acquiring the RGB image and the high dynamic range image (HDR image), and automatically triggering the acquisition of the structured light depth image when an overexposed area is detected or a highly reflective or deep groove area is entered, by analyzing the image pixel brightness in real time or according to the predefined complex area identifier, so as to adapt the optimal data acquisition mode to different surface areas of the physical mold.

[0021] In this embodiment, RGB and HDR images are used for rapid evaluation, with HDR images already mitigating overexposure to some extent. Furthermore, through real-time image analysis or a predefined knowledge base, challenging areas unsuitable for optical scanning (such as highly reflective points and deep grooves with severe optical shadows) are accurately identified. Structured light depth image acquisition is automatically triggered, utilizing its active projection of coded light spots to directly obtain reliable 3D information for these challenging areas, thus achieving optimal adaptation of the data acquisition mode. It is important to note that indiscriminately acquiring data from all modalities across all regions would result in a massive data volume and long acquisition time. This embodiment uses conditional triggering, ensuring that the potentially more time-consuming structured light scanning operation is applied only to the most critical local areas. For conventional areas such as diffuse reflection, subsequent multi-view stereo matching is sufficient using only high-quality HDR images. This combined active and passive, on-demand triggering strategy ensures complete and unobstructed data in key challenging areas such as reflectivity and deep grooves, while avoiding unnecessary data redundancy and wasted acquisition time, achieving a balance between efficiency and accuracy.

[0022] Preferably, in step S2, when performing three-dimensional reconstruction on the multimodal image data, the following steps are included: using the arranged reference markers, the Zhang Zhengyou calibration method is used to calibrate the camera intrinsic parameters and correct lens distortion of the acquisition system, and the absolute scale of the three-dimensional space is restored using the scale reference component; based on the corrected image set, the motion recovery structure algorithm is executed to obtain sparse point cloud and camera pose, and then the structured light depth image is fused as a geometric constraint to generate the globally consistent dense point cloud through a multi-view stereo matching algorithm.

[0023] In this embodiment, camera intrinsic parameter calibration and lens distortion correction are the cornerstones of accuracy for all vision-based 3D reconstruction techniques. This embodiment precisely quantifies the camera's imaging geometry and eliminates image distortion introduced by the lens itself, ensuring the accuracy of calculating 3D spatial relationships from 2D images. Combining the absolute scale recovered from the scale reference component elevates the entire reconstruction process from relative reconstruction to absolute measurement, resulting in a model with realistic physical dimensions, thus meeting the stringent dimensional accuracy requirements of industrial inspection and replication. In this embodiment, Structured Motion Reconstruction (SfM) can robustly recover camera motion trajectories and sparse scene structures from RGB / HDR image sequences, but its Dense Reconstruction (MVS) is prone to failure in weak texture and reflective areas. At this point, using structured light depth images as geometric constraints is equivalent to providing the MVS algorithm with prior depth information and high-confidence guidance in challenging areas. This fusion fully utilizes the advantages of SfM's good global consistency and structured light's accurate depth measurement and strong anti-interference capabilities in local areas, resulting in a globally optimal overall dense point cloud that achieves high fidelity in detailed and challenging areas. Meanwhile, structured light depth data can provide a reliable reference framework and search range limitation for multi-view stereo matching. In areas where it is difficult to make correct matching based solely on image texture, the depth values ​​provided by structured light can greatly reduce the matching search space, effectively avoiding noise, outliers, or large-area holes caused by matching ambiguity in the MVS algorithm, thereby improving the success rate of the entire reconstruction process and the quality of the final point cloud.

[0024] Preferably, when generating the globally consistent dense point cloud using a multi-view stereo matching algorithm, the process includes: converting the structured light depth image into a depth reference map corresponding to the corrected image set; during the execution of the multi-view stereo matching algorithm, using the depth reference map as a strong constraint on the depth calculation range, applying it to low-texture or high-reflectivity areas identified by the corrected image set, and fusing the matching results of all views in the multi-view stereo matching algorithm to generate the globally consistent dense point cloud.

[0025] In this embodiment, the structured light depth image is converted into a depth reference map corresponding to the corrected image set. This allows the MVS algorithm to determine, at the pixel level, which specific location (low-texture or high-reflectivity area) in the image is likely to be near which depth value a point in 3D space is located. Simultaneously, using the depth reference map as a strong constraint on the depth calculation range means the algorithm can prioritize fine-grained matching calculations within a reliable depth range, rather than blindly and time-consumingly searching through an infinite number of possible depths. Furthermore, low-texture areas (such as smooth planes) lack unique features for matching, while the texture of high-reflectivity areas (such as polished metal) can disappear or even vanish due to drastic changes in viewpoint. This embodiment provides reliable geometric answers for these specific areas through active depth information. The depth reference map can guide the MVS algorithm to compensate for the lack of visual information, ensuring the integrity and continuity of the final dense point cloud.

[0026] Preferably, in step S2, when performing semantic segmentation on the dense point cloud, a point cloud segmentation network based on deep learning is used. The point cloud segmentation network is a PointNet++ or RandLA-Net architecture, trained on a mold point cloud dataset containing labeled cavity, core and flow channel regions, to classify the input point cloud point by point and output semantic segmentation results with functional region labels.

[0027] In this embodiment, by employing advanced point cloud deep learning networks (such as PointNet++ and RandLA-Net), complex local and global features can be effectively learned and extracted directly from massive, unstructured point cloud data. These networks, trained on a large amount of labeled data, have internalized design knowledge of different types of molds, enabling them to automatically identify and accurately label key functional areas such as cavities, cores, and flow channels with extremely high accuracy. This changes the outdated model that relies on manual visual identification and segmentation, transforming a tedious, subjective, and error-prone task into an efficient, objective, and repeatable automated process. The selected network architecture (especially RandLA-Net) possesses efficient processing capabilities for large-scale point clouds and powerful local feature aggregation, enabling stable feature boundary identification for molds with complex geometries and uneven point cloud density, and is less susceptible to noise and changes in point cloud distribution. The final output of semantic segmentation can be a point-by-point classification result with precise functional labels, enabling the interpretation of point cloud data and providing the most direct driving information for subsequent differentiated modeling. The system can clearly identify which points belong to the freeform surface that needs to be fitted by NURBS and which point sets correspond to the pin holes that can be fitted using cylindrical surface primitives. This allows for precise invocation of the appropriate modeling algorithms, ensuring that the entire process is automated and intelligent.

[0028] Preferably, in step S3, the geometric primitive includes a plane, a cylindrical surface, a conical surface, or a sphere; when using the geometric primitive to fit and generate a parameterized geometry, a random sampling consensus algorithm is used to robustly estimate the optimal parameters of the geometric primitive from the point set corresponding to the regular geometric features.

[0029] It should be noted that molds contain numerous standard, regular features such as positioning surfaces, guide pin holes, and bolt holes. These features were defined in the original CAD design using geometric primitives such as planes and cylindrical surfaces. This embodiment uses geometric primitives for direct fitting, rather than approximating with complex freeform surfaces, to directly recover the most essential mathematical expressions of these features (such as the normal vector and distance of a plane, and the axis and radius of a cylinder). This allows the reconstructed model to recover its true parameters as a regular geometric body, and the generated CAD model has editable parameters (such as diameter, length, and angle) that are completely consistent with the original design intent, greatly improving the parametric quality and engineering value of the reconstructed model. Furthermore, in actual scanning, point cloud data inevitably contains noise, outliers, and confounding points from adjacent features. Traditional fitting algorithms (such as least squares) are very sensitive to these outliers and are prone to producing biased fitting results. The Random Sample Consensus Algorithm (RANSAC), through an iterative mechanism of random sampling and consistency verification, can robustly estimate the optimal model parameters from datasets containing a large amount of noise.

[0030] Preferably, in step S4, the boundary continuity constraint is a G1 or G2 continuity constraint; the fusion is achieved by solving a constraint optimization problem, the construction and solution of which includes: defining the boundary control parameters of the parametric geometry, the parametric surface, and the parametric channel model as optimization variables; establishing the optimization objective as minimizing the overall geometric deviation between the hybrid parametric 3D model and the original dense point cloud; transforming the G1 or G2 continuity constraint into mathematical equations and applying them as necessary constraints to the connection boundaries between the parametric geometry, the parametric surface, and the parametric channel model; and completing the fusion by solving the constraint optimization problem.

[0031] It should be noted that this embodiment, by applying higher-order continuity constraints such as G1 (tangential continuity) or G2 (curvature continuity), ensures that, for example, a NURBS freeform surface and a planar primitive not only have no gaps at the connection boundary, but also that the transition is smooth and seamless. This eliminates the sharp edges or wrinkles caused by traditional simple splicing, making the reconstructed model visually and physically closer to the original design. This meets the high requirements of molds for surface finish and smooth fluid flow, and avoids stress concentration or marks left on the product in subsequent CAE analysis due to poor connection. In this embodiment, while ensuring continuity, the model accurately maintains its fidelity to the original scan data. By constructing a constraint optimization problem with the goal of minimizing overall geometric deviation, it cleverly balances the sometimes conflicting requirements of smoothness and accuracy. The optimization process does not arbitrarily smooth the model, but rather, under the strict constraint of G1 / G2 continuity, fine-tunes the boundary control parameters of all parametric elements to find a solution that best matches the original scanned point cloud. This ensures that the final fused hybrid model achieves smooth boundary connections while preserving the high-precision geometric details obtained from the solid mold scan to the greatest extent possible. Furthermore, this embodiment formalizes the fusion problem into a clearly constrained optimization problem, transforming the manual splicing and repair work previously done by experienced CAD engineers based on experience and intuition into a mathematical problem with a clear objective function and constraints. This allows for the automatic, accurate, and efficient completion of this complex task, ensuring consistent result quality and improving the automation and efficiency of the entire modeling process. Here, CAE refers to Computer-Aided Engineering.

[0032] Preferably, after step S4, step S5 is further included: importing the constructed hybrid parametric three-dimensional model into finite element analysis software for mechanical or thermodynamic performance simulation; based on the stress, strain, or temperature field distribution results obtained from the simulation, parametric adjustment and multi-objective optimization are performed on the high-risk areas identified in the model to generate an optimized mold model that meets the preset performance indicators.

[0033] It should be noted that this embodiment extends the endpoint of 3D modeling from simple geometric reproduction to optimization and re-creation based on physical performance. Since step S4 generates a fully parametric model, its size and shape can be easily adjusted, providing fundamental convenience for performance optimization. Through FEA simulation, the stress and heat conditions of the mold in actual use can be pre-simulated in virtual space, accurately locating high-risk areas such as stress concentration, uneven cooling, and potential deformation or cracking, achieving an improvement from passive replication to active performance optimization. It is important to note that this embodiment seamlessly integrates CAD (Computer-Aided Design) and CAE (Computer-Aided Engineering) into a coherent workflow. Based on simulation feedback, the parametric model is directly iterated and automatically optimized to generate a new model with superior mechanical or thermodynamic performance. This data-driven scientific decision-making model can replace traditional trial-and-error methods or experience-based estimations, enabling the identification and elimination of potential design defects before physical manufacturing. This significantly reduces the number of subsequent mold trials and revisions, shortens development time, lowers development costs, and fundamentally improves the quality and service life of the final mold product.

[0034] Preferably, after step S5, step S6 is further included: importing the optimized mold model into the computer-aided manufacturing system to automatically generate toolpath code for driving the CNC machine tool; using the toolpath code to process a solid mold test piece, and performing three-dimensional scanning and deviation analysis on the test piece, and feeding back the analysis results to the model parameters or machining process parameters.

[0035] It should be noted that this embodiment extends the digital chain further to the manufacturing and testing stages. Utilizing the optimized, fully parametric model, the CAM (Computer-Aided Manufacturing) system can automatically and accurately generate CNC machining instructions, driving the machine tool to directly process the optimal design into a physical prototype. This ensures that the design intent is transferred to the manufacturing stage without loss, achieving the ideal state of "model as product," effectively reducing distortion and errors in the information transmission process. This embodiment further verifies the entire design-to-manufacturing process by performing high-precision 3D scanning on the machined prototype and conducting deviation analysis (comparing it with the original digital model). If a systematic deviation is found, its root cause can be traced: if it is a model accuracy issue, the model parameters are adjusted accordingly; if it is machine tool machining error, tool wear, or inappropriate process parameters, the machining process parameters are compensated or optimized.

[0036] It should be noted that the above embodiments are merely preferred embodiments of the present invention, and the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention, and the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for high-precision 3D modeling of molds based on multimodal data fusion, characterized in that, Includes the following steps: S1. The automated acquisition system is controlled to acquire multimodal image data of the physical mold according to a preset shooting sequence. The multimodal image data includes RGB images, high dynamic range images, and structured light depth images. S2. Perform three-dimensional reconstruction on the multimodal image data to obtain a globally consistent dense point cloud, and perform noise reduction and semantic segmentation on the dense point cloud to identify at least one functional region corresponding to the cavity, core and flow channel of the physical mold from the dense point cloud. S3. Based on the functional regions identified by the semantic segmentation, perform differentiated parametric modeling, which includes: for the regular geometric features in the functional regions of the cavity and the core, using geometric primitives to fit and generate parametric geometry; for the free-form surfaces in the functional regions of the cavity and the core, using NURBS surface reconstruction to generate parametric surfaces; and for the identified flow channel functional regions, using swept volume or volume of revolution to fit and generate parametric channel models. S4. Apply boundary continuity constraints to fuse the parametric geometry, the parametric surface, and the parametric channel model to construct a hybrid parametric 3D model of the solid mold.

2. The method according to claim 1, characterized in that, In step S1, when the automated acquisition system acquires multimodal image data of the physical mold according to the preset shooting sequence, it includes: before acquisition, arranging multiple reference marks of known size in the space around the physical mold and placing a scale reference piece with precise scale; based on the evaluation of the geometric shape and complexity of the physical mold, dividing its surface into multiple acquisition zones, and planning a set of shooting poses covering its entire surface for each zone, wherein the shooting poses of adjacent zones ensure that their fields of view have an overlap area of ​​not less than a preset value.

3. The method according to claim 2, characterized in that, In step S1, when acquiring the multimodal image data, the method further includes: controlling the automated acquisition system to move sequentially to each planned shooting pose, first synchronously acquiring the RGB image and the high dynamic range image, and automatically triggering the acquisition of the structured light depth image when an overexposed area is detected or a highly reflective or deep groove area is entered, by analyzing the image pixel brightness in real time or according to the predefined complex area identifier, so as to adapt the optimal data acquisition mode to different surface areas of the physical mold.

4. The method according to claim 2, characterized in that, In step S2, when performing three-dimensional reconstruction on the multimodal image data, the following steps are included: using the arranged reference markers, the Zhang Zhengyou calibration method is used to calibrate the camera intrinsic parameters and correct lens distortion of the acquisition system, and the absolute scale of the three-dimensional space is restored using the scale reference component; based on the corrected image set, the motion recovery structure algorithm is executed to obtain sparse point cloud and camera pose, and then the structured light depth image is fused as a geometric constraint, and the globally consistent dense point cloud is generated through a multi-view stereo matching algorithm.

5. The method according to claim 4, characterized in that, When generating the globally consistent dense point cloud using a multi-view stereo matching algorithm, the process includes: converting the structured light depth image into a depth reference map corresponding to the corrected image set; during the execution of the multi-view stereo matching algorithm, using the depth reference map as a strong constraint on the depth calculation range, applying it to low-texture or high-reflectivity areas identified by the corrected image set, and fusing the matching results of all views in the multi-view stereo matching algorithm to generate the globally consistent dense point cloud.

6. The method according to claim 1, characterized in that, In step S2, when performing semantic segmentation on the dense point cloud, a point cloud segmentation network based on deep learning is used. The point cloud segmentation network is a PointNet++ or RandLA-Net architecture, trained on a mold point cloud dataset containing labeled cavity, core and flow channel regions, to classify the input point cloud point by point and output semantic segmentation results with functional region labels.

7. The method according to claim 1, characterized in that, In step S3, the geometric primitives include planes, cylindrical surfaces, conical surfaces, or spheres; when using geometric primitives to fit and generate parameterized geometries, a random sampling consensus algorithm is used to robustly estimate the optimal parameters of the geometric primitives from the point set corresponding to the regular geometric features.

8. The method according to claim 1, characterized in that, In step S4, the boundary continuity constraint is a G1 or G2 continuity constraint; the fusion is achieved by solving a constraint optimization problem, the construction and solution of which include: defining the boundary control parameters of the parametric geometry, the parametric surface, and the parametric channel model as optimization variables; establishing the optimization objective as minimizing the overall geometric deviation between the hybrid parametric 3D model and the original dense point cloud; transforming the G1 or G2 continuity constraint into mathematical equations and applying them as necessary constraints to the connection boundaries between the parametric geometry, the parametric surface, and the parametric channel model; and completing the fusion by solving the constraint optimization problem.

9. The method according to any one of claims 1-8, characterized in that, After step S4, step S5 is also included: importing the constructed hybrid parametric three-dimensional model into finite element analysis software for mechanical or thermodynamic performance simulation; based on the stress, strain or temperature field distribution results obtained from the simulation, parametric adjustment and multi-objective optimization are performed on the high-risk areas identified in the model to generate an optimized mold model that meets the preset performance indicators.

10. A high-precision 3D modeling system for molds based on multimodal data fusion, characterized in that, The high-precision 3D modeling system for molds based on multimodal data fusion uses the high-precision 3D modeling method for molds based on multimodal data fusion as described in any one of claims 1-9.

Images