Additive manufacturing accurate registration method based on multi-modal fusion

Through the multimodal fusion additive manufacturing precision registration method, the sensor data and X-ray scanning data are integrated, and the machine learning model is trained in combination with performance evaluation functions, which solves the problem that a single sensor is difficult to fully detect pores and molten pool states, achieving high Accurate defect prediction and manufacturing quality assurance.

CN119068025APending Publication Date: 2024-12-03GUANGDONG UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410291316.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-03-14
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In L-PBF additive manufacturing, it is difficult to fully detect the continuous changes in pore state and molten pool state in a single sensor accurately registers, and it is impossible to effectively evaluate the actual defects and quality of standard parts after processing.

Method used

Using multimodal fusion additive manufacturing precision registration method, the machine learning model is evaluated by integrating multimodal sensor data and X-ray computed tomography data, combined with performance evaluation functions, and the thermoplatform data set and XCT reference data are realized The high-precision registration of the set includes data acquisition, preprocessing, image registration and machine learning model training for multimodal sensor modules and X-ray computed tomography modules.

Benefits of technology

Improves registration accuracy between the Hemppat dataset and the XCT reference dataset, ensures effective cooperation in the multimodal fusion platform, and improves the accuracy of defect prediction and the manufacturing quality of the final components.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068025A_ABST
    Figure CN119068025A_ABST
Patent Text Reader

Abstract

The invention discloses an additive manufacturing precise registration method based on multi-modal fusion. The method comprises the steps that a heat map and XCT reference data are collected at the same time; the collected data are preprocessed; carrying out image registration; establishing a machine learning model; a machine learning model is obtained after training; registration of the heat map data set and the XCT reference data set is achieved through the trained machine learning model, and a pre-registration result is obtained; and evaluating the machine learning model through a performance evaluation function, if the evaluation is passed, taking a pre-registration result as a final registration result, and if the evaluation is not passed, returning the machine learning model to the training step for training again. According to the method, sensor data of different types are integrated, and the machine learning model is evaluated in combination with the performance evaluation function, so that registration of the heat map data set and the XCT reference data set is realized through the high-precision machine learning model, and the registration accuracy between the heat map data set and the XCT reference data set is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of additive manufacturing, and in particular to a precise registration method for additive manufacturing based on multimodal fusion. Background Art

[0002] During the processing of standard parts using additive manufacturing methods, the occurrence of irregularities such as internal pores, cracks or surface roughness can have a negative impact on the quality of the final part. Poor process parameterization, such as scanning speed and laser power, can cause irregularities in the workpiece. The registration function can be obtained by spatially transforming the moving image registered to the fixed image. The accuracy of the registration is crucial for evaluating the measurement uncertainty.

[0003] In the context of predicting internal porosity based on sensor data, the key to achieving successful predictions by applying machine learning algorithms is to accurately allocate the space between the sensor signal and the porosity information. In L-PBF (Laser Powder Bed Fusion), accurate registration is particularly important due to the presence of irregular micro-sizes. In addition, differences in data format and dimensionality caused by different measurement methods further increase the challenges of registration.

[0004] In general, ensuring accurate registration of thermal imaging in-situ monitoring data and XCT (X-ray micro-computed tomography) reference data is crucial for defect prediction in L-PBF additive manufacturing. This registration process not only helps to develop accurate prediction models, but also helps to ensure the manufacturing quality of the final component.

[0005] In the L-PBF additive manufacturing process, single sensor accurate registration is considered as a method to detect defects. Although the single sensor accurate registration method can quickly obtain the processing quality feedback of the surface of the standard part during the processing, there is a lack of in-depth research on its direct application. The main problem is that although the single sensor accurate registration method can ensure that the surface printing data of the standard part being processed can be collected in time, under normal circumstances, the printing of each layer in the L-PBF additive manufacturing process may affect the porosity and melt pool state of the four or five layers below the workpiece. Therefore, it is difficult to fully detect the continuous changes in the porosity state and melt pool state during the processing of standard parts based on the single sensor accurate registration method, and it cannot effectively evaluate the actual defects and quality of the standard parts after processing. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a precise registration method for additive manufacturing based on multimodal fusion. The method integrates different types of sensor data and evaluates the machine learning model in combination with a performance evaluation function, so that the registration of the thermal map dataset and the XCT reference dataset can be achieved through a high-precision machine learning model, thereby improving the registration accuracy between the thermal map dataset and the XCT reference dataset, and ensuring that they can work effectively together in the multimodal fusion standard parts processing platform.

[0007] To achieve the above purpose, the technical solution provided by the present invention is:

[0008] A precise registration method for additive manufacturing based on multimodal fusion, comprising:

[0009] S1, using a multimodal sensor module and an X-ray computed tomography module to simultaneously collect thermal images and XCT reference data;

[0010] S2, preprocessing the acquired thermogram and XCT reference data;

[0011] S3, image registration based on the preprocessed thermal map and XCT reference data;

[0012] S4. Establish a machine learning model with a mapping relationship from feature data to accurate registration;

[0013] S5, extract and annotate key features from the registered thermal image and XCT reference data;

[0014] S6. Train the machine learning model through the annotated key features to obtain a trained machine learning model;

[0015] S7, realizing registration of the thermal image dataset and the XCT reference dataset through the trained machine learning model to obtain a pre-registration result;

[0016] S8. Evaluate the machine learning model through the performance evaluation function. If the evaluation passes, the pre-registration result is used as the final registration result. If the evaluation fails, the machine learning model returns to step S6 for training again.

[0017] Furthermore, the collected thermogram and XCT reference data are preprocessed including: denoising, normalization, spatial alignment, outlier processing, smoothing and data format unification.

[0018] Furthermore, image registration includes preliminary registration and non-rigid registration;

[0019] Initial registration includes:

[0020] Extract feature points or feature areas from the thermal map and XCT reference data, and achieve preliminary alignment through feature matching and transformation estimation;

[0021] Non-rigid registration includes:

[0022] Taking into account the deformation and distortion of parts, more accurate capture and correction of actual changes of parts during machining can be achieved through deformation field estimation, deformation field application, optimization and final registration.

[0023] Furthermore, the performance evaluation function is as follows:

[0024] Performance

[0025] =ω1×Registration Consistency Metric+ω2

[0026] ×Registration Accuracy+ω3×Registration Coverage+ω4

[0027] ×Registration Completeness Score+ω5

[0028] ×Geometric Alignment Score+ω6×Integrated_data_factor

[0029] subject to:

[0030]

[0031] In the above formula, Performance is the overall comprehensive evaluation value, ω1, ω2, ω3, ω4, ω5, ω6 are the weights corresponding to each parameter respectively; Registration Consistency Metric is the registration consistency metric, which is used to evaluate the consistency and stability of the registration algorithm in different data sets or time points;

[0032] Registration Accuracy is the registration accuracy, which is an indicator used to measure the accuracy of the image registration process. The registration accuracy indicator evaluates the accuracy of the alignment, that is, the degree of closeness between the obtained registration result and the true or ideal registration; Registration Coverage is the registration coverage rate, which is used to measure the proportion of the area in the image after registration is correctly aligned; Registration Completeness Score is the registration completeness score, which is used to more comprehensively evaluate the quality of image registration, considering both the accuracy of the registration and the coverage of the entire image, so that the evaluation result is more practical and applicable;

[0033] Geometric Alignment Score is the geometric alignment score, which comprehensively considers the degree of geometric alignment of the registered image, including the accuracy of rotation, translation and scaling. It is evaluated by calculating the geometric transformation error between the registered image and the reference image. Integrated_data_factor is a comprehensive indicator of the working environment, including the working environment temperature, humidity and atmospheric pressure environmental parameters.

[0034] Furthermore, the image is segmented into different regions or grids, and the proportion of correctly aligned pixels in each region is calculated; then, the average of these proportions is taken as the registration coverage.

[0035] Furthermore, the calculation formula of the integrated working environment index Integrated_data_factor is as follows:

[0036] Integrated_datc_factor

[0037] =ω t ×Normalized_Temperature+ω h ×Normalized_Humidity+ω p ×Normalized_Pressure

[0038] in:

[0039] Normalized_Temperature represents the normalized working environment temperature value;

[0040] Normalized_Humidity indicates the normalized working environment humidity value;

[0041] Normalized_Pressure represents the normalized atmospheric pressure value;

[0042] ωt ,ω h ,ω p They respectively represent the weights of working environment temperature, working environment humidity and atmospheric pressure in the comprehensive evaluation.

[0043] Compared with the existing technology, the principle and advantages of this solution are as follows:

[0044] 1. This technical solution integrates different types of sensor data and evaluates the machine learning model in combination with a performance evaluation function, so that the registration of the thermal map dataset and the XCT reference dataset can be achieved through a high-precision machine learning model, thereby improving the registration accuracy between the thermal map dataset and the XCT reference dataset, ensuring that they can work effectively together in the multimodal fusion standard parts processing platform.

[0045] 2. Image registration includes preliminary registration and non-rigid registration. Through these two key steps, a gradual transition from initial data to more accurate registration results can be achieved, providing more accurate input for subsequent machine learning models, thereby improving the accuracy of defect prediction in L-PBF additive manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the services required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 This is a principle flow chart of an additive manufacturing precision registration method based on multimodal fusion according to the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with specific embodiments:

[0049] like Figure 1 As shown, the additive manufacturing precise registration method based on multimodal fusion described in this embodiment includes the following steps:

[0050] S1, using a multimodal sensor module and an X-ray computed tomography module to simultaneously collect thermal images and XCT reference data;

[0051] In this step, the sensor is calibrated before acquisition to ensure that the output matches the actual scene. Key environmental parameters are recorded and the sensors are synchronized in time to collect data. When preparing the sample, consider its position and posture, and perform continuous acquisition to capture dynamic changes throughout the manufacturing process.

[0052] S2. Preprocess the acquired thermal map and XCT reference data, including: denoising, normalization, spatial alignment, outlier processing, smoothing, and data format unification to ensure data quality and consistency. By carefully designing and executing these steps, a high-quality data foundation is provided for subsequent registration and analysis, laying a solid foundation for the training of machine learning algorithms and the optimization of system performance.

[0053] S3, image registration based on the preprocessed thermal map and XCT reference data;

[0054] Image registration includes preliminary registration and non-rigid registration;

[0055] Initial registration includes:

[0056] Extract feature points or feature areas from the thermal map and XCT reference data, and achieve preliminary alignment through feature matching and transformation estimation;

[0057] Non-rigid registration includes:

[0058] Taking into account the deformation and distortion of parts, more accurate capture and correction of actual changes of parts during machining can be achieved through deformation field estimation, deformation field application, optimization and final registration.

[0059] S4. Establish a machine learning model with a mapping relationship from feature data to accurate registration;

[0060] This step specifically includes: selecting an appropriate machine learning algorithm, dividing the training set and validation set, designing the model architecture, selecting the loss function, performing model training and hyperparameter adjustment, so as to establish a machine learning model that can map from feature data to accurate alignment.

[0061] S5, extract and annotate key features from the registered thermal image and XCT reference data;

[0062] S6. Train the machine learning model through the annotated key features to obtain a trained machine learning model:

[0063] S7, realizing registration of the thermal image dataset and the XCT reference dataset through the trained machine learning model to obtain a pre-registration result;

[0064] S8. Evaluate the machine learning model through the performance evaluation function. If the evaluation passes, the pre-registration result is used as the final registration result. If the evaluation fails, the machine learning model returns to step S6 for training again.

[0065] Among them, the performance evaluation function is as follows:

[0066] Performance

[0067] =ω1×Registration Consistency Metric+ω2×Registration Accuracy+ω3×Registration Coverage+ω4×Registration Completeness Score+ω5×GeometricAlignment Score+ω6×Integrated_data_factor

[0068] subject to:

[0069]

[0070] In the above formula, Performance is the overall comprehensive evaluation value, ω1, ω2, ω3, ω4, ω5, ω6 are the weights corresponding to each parameter respectively; Registration Consistency Metric is the registration consistency metric, which is used to evaluate the consistency and stability of the registration algorithm in different data sets or time points;

[0071] Registration Accuracy is the registration accuracy, which is an indicator used to measure the accuracy of the image registration process. The registration accuracy indicator evaluates the accuracy of the alignment, that is, the degree of closeness between the obtained registration result and the true or ideal registration; Registration Coverage is the registration coverage rate, which is used to measure the proportion of the area in the image after registration is correctly aligned; Registration Completeness Score is the registration completeness score, which is used to more comprehensively evaluate the quality of image registration, considering both the accuracy of the registration and the coverage of the entire image, so that the evaluation result is more practical and applicable;

[0072] Geometric Alignment Score is the geometric alignment score, which comprehensively considers the degree of geometric alignment of the registered image, including the accuracy of rotation, translation and scaling. It is evaluated by calculating the geometric transformation error between the registered image and the reference image. Integrated_data_factor is a comprehensive indicator of the working environment, including the working environment temperature, humidity and atmospheric pressure environmental parameters.

[0073] In the above, the calculation formula of the integrated working environment index Integrated_data_factor is as follows:

[0074] Integrated_data_factor

[0075] =ω t ×Normalized_Temperature+ω h ×Normalized_Humidity+ω p ×Normalized_Pressure

[0076] in:

[0077] Normalized_Temperature represents the normalized working environment temperature value;

[0078] Normalized_Humidity indicates the normalized working environment humidity value;

[0079] Normalized_Pressure represents the normalized atmospheric pressure value;

[0080] ω t ,ω h ,ω p They respectively represent the weights of working environment temperature, working environment humidity and atmospheric pressure in the comprehensive evaluation.

[0081] If Performance is less than the set value, the evaluation passes; otherwise, the evaluation fails.

[0082] This embodiment integrates different types of sensor data and evaluates the machine learning model in combination with a performance evaluation function, so that the registration of the thermal map dataset and the XCT reference dataset can be achieved through a high-precision machine learning model, thereby improving the registration accuracy between the thermal map dataset and the XCT reference dataset, and ensuring that they can work effectively together in the multimodal fusion standard parts processing platform.

[0083] More specifically, image registration includes preliminary registration and non-rigid registration. Through these two key steps, a gradual transition from initial data to more accurate registration results can be achieved, providing more accurate input for subsequent machine learning models, thereby improving the accuracy of defect prediction in L-PBF additive manufacturing.

[0084] The embodiments described above are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A precise registration method for additive manufacturing based on multimodal fusion, characterized in that: include: S1, using a multimodal sensor module and an X-ray computed tomography module to simultaneously collect thermal images and XCT reference data; S2, preprocessing the acquired thermogram and XCT reference data; S3, image registration based on the preprocessed thermal map and XCT reference data; S4. Establish a machine learning model with a mapping relationship from feature data to accurate registration; S5, extract and annotate key features from the registered thermal image and XCT reference data; S6. Train the machine learning model through the annotated key features to obtain a trained machine learning model; S7, realizing registration of the thermal image dataset and the XCT reference dataset through the trained machine learning model to obtain a pre-registration result; S8. Evaluate the machine learning model through the performance evaluation function. If the evaluation passes, the pre-registration result is used as the final registration result. If the evaluation fails, the machine learning model returns to step S6 for training again.

2. According to claim 1, a precise registration method for additive manufacturing based on multimodal fusion is characterized in that: The preprocessing of the acquired thermogram and XCT reference data includes: denoising, normalization, spatial alignment, outlier processing, smoothing and data format unification.

3. The method for accurate registration of additive manufacturing based on multimodal fusion according to claim 1, characterized in that: Image registration includes preliminary registration and non-rigid registration; Initial registration includes: Extract feature points or feature areas from the thermal map and XCT reference data, and achieve preliminary alignment through feature matching and transformation estimation; Non-rigid registration includes: Taking into account the deformation and deformation of parts, more accurate capture and correction of actual changes of parts during processing can be achieved through deformation field estimation, deformation field application, optimization and final registration.

4. The method for accurate registration of additive manufacturing based on multimodal fusion according to claim 1, characterized in that: The performance evaluation function is as follows: Performance=ω1×Registration Consistency Metric+ω2×RegistrationAccuracy+ω3×Registration Coverage+ω4×Registration Completeness Score+ω5×Geometric Alignment Score+ω6×Integrated_data_factor subject to: In the above formula, Performance is the total comprehensive evaluation value, ω1, ω2, ω3, ω4, ω5, ω6 are the weights corresponding to each parameter respectively; Registration Consistency Metric is a registration consistency metric used to evaluate the consistency and stability of the registration algorithm in different data sets or time points; Registration Accuracy is the registration accuracy, which is an indicator used to measure the accuracy of the image registration process. The registration accuracy indicator evaluates the accuracy of the alignment, that is, the degree of closeness between the obtained registration result and the true or ideal registration; Registration Coverage is the registration coverage rate, which is used to measure the proportion of the area in the image after registration is correctly aligned; Registration Completeness Score is the registration completeness score, which is used to more comprehensively evaluate the quality of image registration, considering both the accuracy of the registration and the coverage of the entire image, so that the evaluation result is more practical and applicable; Geometric Alignment Score is the geometric alignment score, which comprehensively considers the degree of geometric alignment of the registered image, including the accuracy of rotation, translation and scaling. It is evaluated by calculating the geometric transformation error between the registered image and the reference image. Integrated_data_factor is a comprehensive indicator of the working environment, including the working environment temperature, humidity and atmospheric pressure environmental parameters.

5. The method for accurate registration of additive manufacturing based on multimodal fusion according to claim 4, characterized in that: The image is segmented into different regions or grids, and the proportion of correctly aligned pixels in each region is calculated; then, the registration coverage is taken as the average of these proportions.

6. The method for accurate registration of additive manufacturing based on multimodal fusion according to claim 4, characterized in that: The calculation formula of the integrated working environment index Integrated_data_factor is as follows: Integrated_data_factor =ω t ×Normalized_Temperature+ω h ×Normalized_Humidity+ω p ×Normalized_Pressure in: Normalized_Temperature represents the normalized working environment temperature value; Normalized_Humidity indicates the normalized working environment humidity value; Normalized_Pressure represents the normalized atmospheric pressure value; ω t ,ω h ,ω p They respectively represent the weights of working environment temperature, working environment humidity and atmospheric pressure in the comprehensive evaluation.