3D printing abnormity alarm method and system based on dual-channel detection and decision fusion

Abstract

The invention provides a 3D printing abnormity alarm method and system based on dual-channel detection and decision fusion, and belongs to the field of 3D printing. The method comprises the steps that optical image data and infrared image data in the 3D printing process are synchronously collected and preprocessed; performing multi-mode enhancement on defects in the preprocessed optical image and infrared image data by using a data enhancement frame to obtain an enhanced optical data set and an enhanced infrared data set; training a first YOLOv12-DAD model by using the enhanced optical data set, performing defect detection, and outputting a first detection result; training a second YOLOv12-DAD model by using the enhanced infrared data set, performing defect detection, and outputting a second detection result; performing decision-level fusion on the first detection result and the second detection result to generate a fusion detection result; and triggering a grading early warning signal according to the fusion detection result. According to the method, the coverage range and accuracy of defect detection are improved, and the calculation complexity of traditional feature level fusion is avoided.

Description

Technical Field

[0001] This invention belongs to the field of 3D printing technology, and in particular relates to a 3D printing anomaly alarm method and system based on dual-channel detection and decision fusion. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] 3D printing is rapidly expanding into high-end applications such as aerospace, medical, and mold making because it can directly form complex components. However, the transient behavior of the molten pool and the state of the powder bed are easily affected by the combined effects of laser power, scanning path, and powder spreading quality. Defects such as spatter, spheroidization, poor fusion, and keyholes often occur within milliseconds, leading to reduced density and fluctuations in mechanical properties of the finished product.

[0004] To ensure consistent forming, current methods primarily employ single sensors, such as optical cameras or infrared thermal imagers, for quality monitoring. Infrared thermal imaging can effectively record the temperature distribution of the molten pool and monitor the dynamic thermal behavior of the selective laser melting (SLM) process; however, it can only detect external surface temperature and morphology, failing to directly visualize internal defects, and is also costly and requires accurate material emissivity. CCD / CMOS cameras can monitor the surface quality of the powder and cured layers at high resolution and determine defect locations via CT scanning; however, they suffer from low sampling frequency, high cost, and interference from sputtering and ionization can affect image acquisition.

[0005] Meanwhile, most existing studies rely on feature-level or pixel-level fusion, which requires mixing different modal data within the same network and normalizing them uniformly. Although this can achieve high detection accuracy, it is difficult to train, computationally intensive, and costly to deploy in real time at the edge. In addition, feature-level fusion loses the intuitive visualization capability of a one-to-one correspondence between "defects and sensors," making it difficult for operators to quickly locate the source of the problem. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a 3D printing anomaly alarm method and system based on dual-channel detection and decision fusion. It utilizes a high-resolution optical camera and an infrared thermal imager to simultaneously acquire visible light images and temperature fields during the printing process, and combines them with an improved YOLOv12 target detection network to perform multimodal data fusion analysis to achieve defect detection and early warning.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a 3D printing anomaly alarm method based on dual-channel detection and decision fusion; A 3D printing anomaly alarm method based on dual-channel detection and decision fusion includes: Optical and infrared image data are acquired simultaneously during the 3D printing process, and the optical and infrared images are preprocessed respectively. A data augmentation framework was used to perform multimodal augmentation on defects in preprocessed optical and infrared image data, resulting in augmented optical and infrared datasets. The first YOLOv12-DAD model is trained using the enhanced optical dataset and used for defect detection, outputting the first detection result; the second YOLOv12-DAD model is trained using the enhanced infrared dataset and used for defect detection, outputting the second detection result. The first and second detection results are fused at the decision level to generate a fused detection result; a graded early warning signal is triggered based on the fused detection result.

[0008] As a further technical solution, the simultaneous acquisition of optical and infrared image data during the 3D printing process, and the preprocessing of the optical and infrared images respectively, includes: Optical images are cropped to remove background noise, and CLAHE specular enhancement filtering technology is used to enhance image details; infrared images are radiometrically calibrated and dynamically emissivity compensated to ensure the accuracy of temperature data. Based on the printing bed corner calibration plate, the optical image and infrared image are aligned in spatial coordinates by feature point matching; and the image registration algorithm is used to register the images in the same coordinate system. The high-frequency features of the registered image are extracted using a high-pass filter and edge detection algorithm, and the image quality is optimized by combining histogram equalization, highlight suppression and color difference correction.

[0009] As a further technical solution, the method utilizes a data augmentation framework to perform multimodal enhancement on defects in the preprocessed optical and infrared image data, resulting in enhanced optical and infrared datasets, including: The data augmentation framework is invoked to simulate different defect types through a generative adversarial network, and the augmented defects are added to the preprocessed optical and infrared image data to obtain augmented optical and infrared datasets.

[0010] As a further technical solution, the first YOLOv12-DAD model and the second YOLOv12-DAD model include: The DAD inspection head module is used to output defect category, confidence level, and bounding box information. The SeverityNet regression network module is used to calculate the severity score of defects.

[0011] As a further technical solution, the first detection result includes: a first defect category, a first confidence level, and a first bounding box; The second detection result includes a second defect category, a second confidence level, and a second bounding box.

[0012] As a further technical solution, the first detection result and the second detection result are fused at the decision level to generate a fused detection result, and a graded early warning signal is triggered based on the fused detection result, including: The confidence levels of the first and second test results are calculated using a weighted average. A comprehensive assessment is conducted by combining the defect severity score and the frequency of consecutive occurrences. The final warning level is determined based on preset threshold rules.

[0013] As a further technical solution, the warning levels include Level I alert warning, Level II manual intervention warning, and Level III emergency shutdown warning.

[0014] The second aspect of the present invention provides a 3D printing anomaly alarm system based on dual-channel detection and decision fusion.

[0015] A 3D printing anomaly alarm system based on dual-channel detection and decision fusion includes: The image acquisition and preprocessing module is configured to: synchronously acquire optical and infrared image data during the 3D printing process, and preprocess the optical and infrared images respectively; The data augmentation module is configured to: use a data augmentation framework to perform multimodal augmentation on defects in preprocessed optical and infrared image data to obtain augmented optical and infrared datasets; The detection result output module is configured to: train a first YOLOv12-DAD model using the enhanced optical dataset, perform defect detection, and output a first detection result; train a second YOLOv12-DAD model using the enhanced infrared dataset, perform defect detection, and output a second detection result. The decision-level fusion module is configured to: perform decision-level fusion on the first detection result and the second detection result to generate a fused detection result; and trigger a graded early warning signal based on the fused detection result.

[0016] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in the first aspect of the present invention.

[0017] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in the first aspect of the present invention.

[0018] The above one or more technical solutions have the following beneficial effects: This invention achieves comprehensive monitoring of surface morphology defects and internal thermal anomalies during 3D printing through the collaborative operation of optical and infrared dual-channel sensors, significantly improving the coverage and accuracy of defect detection. The decision-level fusion strategy avoids the computational complexity of traditional feature-level fusion. Combined with an improved YOLOv12 model and SeverityNet regression network, it maintains detection accuracy while keeping the system response time within 10ms, fully meeting industrial-grade real-time requirements. An intelligent three-level early warning mechanism enables closed-loop quality control, from minor anomaly alerts to emergency shutdown for severe defects, providing operators with clear decision-making support.

[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0021] Figure 1 This is a flowchart of the method in the first embodiment.

[0022] Figure 2 This is a system structure diagram of the second embodiment. Detailed Implementation

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] Example 1 This embodiment discloses a 3D printing anomaly alarm method based on dual-channel detection and decision fusion; like Figure 1 As shown, the 3D printing anomaly alarm method based on dual-channel detection and decision fusion includes: Step S1: Simultaneously acquire optical and infrared image data during the 3D printing process, and preprocess the optical and infrared images respectively; Step S2: Use a data augmentation framework to perform multimodal augmentation on defects in the preprocessed optical and infrared image data to obtain augmented optical and infrared datasets. Step S3: Train the first YOLOv12-DAD model using the enhanced optical dataset, perform defect detection, and output the first detection result; train the second YOLOv12-DAD model using the enhanced infrared dataset, perform defect detection, and output the second detection result. Step S4: Perform decision-level fusion on the first detection result and the second detection result to generate a fused detection result; trigger a graded early warning signal based on the fused detection result.

[0027] Specifically, it also includes the following: Step S1: Simultaneously acquire optical and infrared image data during the 3D printing process, and preprocess the optical and infrared images respectively.

[0028] After printing begins, a synchronization trigger is invoked to acquire optical image frames and infrared thermal imaging frames within the same laser scanning cycle, ensuring strict temporal correspondence between the two data streams. Simultaneously, a unified timestamp is written to the acquired optical and infrared images, and geometric coordinate transformation is performed based on a pre-calibrated extrinsic matrix. Only spatial alignment is achieved; pixel-level overlay or channel fusion is not performed, thus preserving the integrity of the original feature information of each channel.

[0029] Furthermore, preprocessing is performed on the optical and infrared images, including: Step S11 involves cropping the optical image to remove background noise and non-critical information, ensuring the clarity and quality of key areas. Then, using High-Effect High-Reflectance Filtering (CLAHE), details in both bright and dark areas are enhanced, improving the visibility of subtle defects. These operations improve accuracy in subsequent processing steps, especially for high-reflectance images, focusing on capturing anomalies such as droplet spatter, molten pool necking, and powder accumulation. For infrared images, radiometric calibration is performed first, specifically by comparing the response of a standard blackbody source at a known temperature with the infrared sensor's response, ensuring accurate temperature data. To better adapt to the thermal characteristics of different materials, dynamic emissivity compensation is implemented, dynamically adjusting the material's emissivity based on material type and environmental changes during printing, making the infrared temperature data more reliable. This process focuses on capturing anomalies such as abnormally high local temperatures, voids caused by deep molten pits / vapor chambers, and abnormal cooling spots.

[0030] Step S12: Align the acquired optical and infrared images using a calibration plate at the printing bed corner to ensure that the spatial coordinates of the two images are consistent. Geometric transformations between the images are performed using image registration algorithms, such as feature-point matching algorithms or motion estimation techniques based on optical flow, to ensure accurate registration of the images in the same coordinate system.

[0031] Step S13 involves further preprocessing the registered image by extracting high-frequency features and optimizing image details using a high-pass filter and edge detection algorithms (such as Canny edge detection). Subsequently, image enhancement algorithms, such as histogram equalization, are used to enhance image contrast and combine highlight suppression and color difference correction to reduce interference from lighting changes and environmental factors.

[0032] Based on predefined 3D printing defect type classification coding and severity grading quantification rules, defect annotation is synchronously performed on spatially aligned dual-channel frame sequences through manual visual verification or semi-automatic boundary recognition algorithms to generate training samples with multimodal labels.

[0033] Step S2: Use a data augmentation framework to perform multimodal augmentation on defects in the preprocessed optical and infrared image data to obtain augmented optical and infrared datasets.

[0034] In this embodiment, the MM-DiffAug data augmentation framework is used to perform multimodal augmentation on defects in optical and infrared images, respectively. The specific augmentation methods are as follows: For optical image defect enhancement, the MM-DiffAug (Multi-Modal DiffusionAugmentation) framework is employed, utilizing a Generative Adversarial Network (GAN) to enhance defects in the images. By simulating different defect types (such as molten droplet spatter, molten pool necking into a sphere, powder accumulation, etc.), optical image data similar to real defects is generated. Specifically, a conditional generator produces random noise, defect categories, and background images, while a discriminator evaluates the differences between the generated defect images and the real images, determining modal consistency. These generated images preserve the structural and textural features of the optical images and incorporate multiple defect patterns, resulting in an enhanced optical dataset that covers a wider range of defect scenarios, enhancing the diversity of the training data.

[0035] For infrared images, the same MM-DiffAug framework is used to enhance defects in the infrared images through a generative adversarial network. During this process, defects in the infrared images (such as overheated areas and uneven cooling) are generated and added to the preprocessed infrared image data. The generated infrared images simulate different thermal distributions and defect patterns to ensure the diversity of the infrared dataset and improve the model's ability to identify thermal anomalies and temperature changes. The enhanced optical and infrared datasets retain the original features of the optical and infrared images, respectively, and are used for subsequent training and validation.

[0036] Furthermore, the trained GAN is used to generate new defect images, and the quality of the generated images is evaluated to ensure that the images conform to the actual defect characteristics. The generated images are then fused with the original dataset to increase the diversity and scale of the dataset.

[0037] Step S3: Train the first YOLOv12-DAD model using the enhanced optical dataset, perform defect detection, and output the first detection result; train the second YOLOv12-DAD model using the enhanced infrared dataset, perform defect detection, and output the second detection result.

[0038] The enhanced optical and infrared datasets were divided into training, validation, and test sets to train a first YOLOv12-DAD model and a second YOLOv12-DAD model. Both models include: a DAD detection head module for outputting defect categories, confidence levels, and bounding box information; and a SeverityNet regression network module for calculating defect severity scores. Specifically: A YOLO-v12-RGB model is trained on the enhanced optical dataset, and a DAD detection head is used to output the first detection result, which includes: the first defect category. First confidence level and the first border By utilizing the key formula for bounding box regression, the relative offset predicted by the neural network is transformed into the actual detection box in the image, enabling precise target localization and resizing, thereby improving the accuracy of target detection. The key formula for bounding box regression is:

[0039] in This represents the absolute coordinates of the center point of the final predicted bounding box within the entire image. This defines the width and height of the final predicted bounding box. This represents the center point offset of the original output of the neural network (before activation). The width and height offsets (logarithmic scale) of the original output of the neural network. The reference coordinates are the top-left corner of the grid cell containing the prediction box; Define the width and height of the predefined Anchor (prior box) for scale reference. The actual width and height of the predicted bounding box are obtained by exponentially calculating the width and height offsets predicted by the network, mapping them to a scaling factor that is always positive, and then multiplying it by the width and height of the anchor box.

[0040] The YOLO-v12-IR model is trained independently on the enhanced infrared dataset, and a second detection result is output, which includes: a second defect category. Second confidence level Second border .

[0041] The first and second detection results are output and written to the detection result buffer to be fused according to the timestamp, for use in subsequent severity assessment and decision-level fusion.

[0042] For the first and second detection results, the Region of Interest (ROI) is cropped and filled within their respective channels. A standardized region corresponding to the target defect is extracted from the dual-channel image, removing background interference and ensuring consistency between optical and infrared data. This facilitates subsequent feature extraction, classification, and decision fusion, which are then fed into the corresponding SeverityNet regression network to obtain the defect severity and score. Specifically, for the bounding box corresponding to each detection result, region cropping is performed within the same sensor channel to extract the local image sub-block (ROI) where the defect is located. The cropped ROI is scaled and padded according to a preset size to ensure that the input resolution is compatible with the SeverityNet network structure while maintaining the defect features without distortion. The normalized ROI is then input into the corresponding SeverityNet regression model to calculate the defect severity score in real time, as shown in the following formula:

[0043] in, The feature representation of the j-th POI in the infrared channel is small; This is the weight matrix, used to perform linear feature transformation on the pooled features; The feature map of the j-th infrared channel ROI region; For bias terms; This is a global average pooling operation.

[0044] Step S4: Perform decision-level fusion on the first detection result and the second detection result to generate a fused detection result; trigger a graded early warning signal based on the fused detection result.

[0045] Combining the highest confidence levels of both paths, the formula is as follows:

[0046] in, This represents the highest confidence level.

[0047] The Smart Alarm Engine (SAE) event engine comprehensively judges the defect category, confidence level, number of consecutive frames and frequency of occurrence within each time window to generate corresponding risk level parameters.

[0048] Set the highest confidence threshold and generate alarm levels based on the threshold set. When the risk level reaches the preset threshold, the event engine triggers the corresponding warning in the order of Level I / II / III: Level I prompt, Level II manual intervention, and Level III emergency shutdown, and simultaneously drives the sound and light or PLC output.

[0049] Meanwhile, after the event is triggered, the system packages and uploads the defect record, associated G-code line number, and real-time environmental parameters to the quality database to enable subsequent traceability and statistical analysis.

[0050] Example 2 This embodiment discloses a 3D printing anomaly alarm system based on dual-channel detection and decision fusion; like Figure 2 As shown, the 3D printing anomaly alarm system based on dual-channel detection and decision fusion includes: The image acquisition and preprocessing module is configured to: synchronously acquire optical and infrared image data during the 3D printing process, and preprocess the optical and infrared images respectively; The data augmentation module is configured to: use a data augmentation framework to perform multimodal augmentation on defects in preprocessed optical and infrared image data to obtain augmented optical and infrared datasets; The detection result output module is configured to: train a first YOLOv12-DAD model using the enhanced optical dataset, perform defect detection, and output a first detection result; train a second YOLOv12-DAD model using the enhanced infrared dataset, perform defect detection, and output a second detection result. The decision-level fusion module is configured to: perform decision-level fusion on the first detection result and the second detection result to generate a fused detection result; and trigger a graded early warning signal based on the fused detection result.

[0051] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0052] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in Example 1.

[0053] Example 4 The purpose of this embodiment is to provide an electronic device.

[0054] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in Embodiment 1.

[0055] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0056] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0057] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A 3D printing anomaly alarm method based on dual-channel detection and decision fusion, characterized in that, include: Optical and infrared image data are acquired simultaneously during the 3D printing process, and the optical and infrared images are preprocessed respectively. A data augmentation framework was used to perform multimodal augmentation on defects in preprocessed optical and infrared image data, resulting in augmented optical and infrared datasets. The first YOLOv12-DAD model is trained using the enhanced optical dataset and used for defect detection, outputting the first detection result; the second YOLOv12-DAD model is trained using the enhanced infrared dataset and used for defect detection, outputting the second detection result. The first and second detection results are fused at the decision level to generate a fused detection result. A graded early warning signal is triggered based on the fusion detection results.

2. The 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in claim 1, characterized in that, The simultaneous acquisition of optical and infrared image data during the 3D printing process, and the preprocessing of the optical and infrared images, include: Optical images are cropped to remove background noise, and CLAHE specular enhancement filtering technology is used to enhance image details; infrared images are radiometrically calibrated and dynamically emissivity compensated to ensure the accuracy of temperature data. Based on the printing bed corner calibration plate, the optical image and infrared image are aligned in spatial coordinates by feature point matching; and the image registration algorithm is used to register the images in the same coordinate system. The high-frequency features of the registered image are extracted using a high-pass filter and edge detection algorithm, and the image quality is optimized by combining histogram equalization, highlight suppression and color difference correction.

3. The 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in claim 1, characterized in that, The data augmentation framework is used to perform multimodal augmentation on defects in preprocessed optical and infrared image data, resulting in augmented optical and infrared datasets, including: The data augmentation framework is invoked to simulate different defect types through a generative adversarial network, and the augmented defects are added to the preprocessed optical and infrared image data to obtain augmented optical and infrared datasets.

4. The 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in claim 1, characterized in that, The first YOLOv12-DAD model and the second YOLOv12-DAD model include: The DAD inspection head module is used to output defect category, confidence level, and bounding box information. The SeverityNet regression network module is used to calculate the severity score of defects.

5. The 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in claim 1, characterized in that, The first detection result includes: a first defect category, a first confidence level, and a first bounding box; The second detection result includes a second defect category, a second confidence level, and a second bounding box.

6. The 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in claim 1, characterized in that, The first and second detection results are fused at the decision level to generate a fused detection result. A tiered early warning signal is triggered based on the fused detection result, including: The confidence levels of the first and second test results are calculated using a weighted average. A comprehensive assessment is conducted by combining the defect severity score and the frequency of consecutive occurrences. The final warning level is determined based on preset threshold rules.

7. The 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in claim 6, characterized in that, The warning levels include: Level I alert warning, Level II manual intervention warning, and Level III emergency shutdown warning.

8. A 3D printing anomaly alarm system based on dual-channel detection and decision fusion, characterized in that: include: The image acquisition and preprocessing module is configured to: synchronously acquire optical and infrared image data during the 3D printing process, and preprocess the optical and infrared images respectively; The data augmentation module is configured to: use a data augmentation framework to perform multimodal augmentation on defects in preprocessed optical and infrared image data to obtain augmented optical and infrared datasets; The detection result output module is configured to: train a first YOLOv12-DAD model using the enhanced optical dataset, perform defect detection, and output a first detection result; train a second YOLOv12-DAD model using the enhanced infrared dataset, perform defect detection, and output a second detection result. The decision-level fusion module is configured to: perform decision-level fusion on the first detection result and the second detection result to generate a fused detection result; A graded early warning signal is triggered based on the fusion detection results.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the 3D printing anomaly alarm method based on dual-channel detection and decision fusion as described in any one of claims 1-7.

Images