Defect real-time detection and compensation system and method for multi-axis FDM printing

Abstract

The invention relates to the field of FDM printing detection, in particular to a system and method for defect real-time detection and compensation for multi-axis FDM printing, and the system for defect real-time detection and compensation for multi-axis FDM printing comprises a multi-axis FDM printing system; a microscopic camera; a macroscopic camera; a processing module; the method comprises the following steps: S1, constructing a defect data set and training a model; s2, real-time defect macroscopic and microscopic detection; s3, defect position calculation; s4, microdefect quantitative analysis and printing parameter feedback correction; and S5, intelligently printing a start-stop decision. According to the invention, real-time detection can be realized, defect positions can be displayed, defects can be rapidly corrected, and the start-stop decision can be intelligently printed.

Description

Technical Field

[0001] The present invention relates to the field of FDM printing detection, and in particular to a system and method for real-time detection and compensation of defects in multi-axis FDM printing. Background Art

[0002] Multi-axis FDM printing has the advantages of multi-degree-of-freedom slicing printing and the realization of more complex structures. It is currently widely used in many fields such as aerospace, electronic manufacturing, and military equipment. As its research and application gradually deepen, the requirements for the accuracy, yield rate and reliability verification of multi-axis FDM printing are gradually increasing, which is mainly reflected in the real-time detection of multi-axis FDM printing defects.

[0003] Currently, there are two main methods for real-time defect detection in multi-axis FDM printing: real-time detection of macro defects and real-time detection of micro defects. The real-time detection of macro defects is more effective for detecting macro defects on the side surface of printed parts, but ordinary real-time detection of macro defects has the disadvantages of limited detection range and a limited number of detectable defects. The real-time detection of micro defects can achieve full coverage of the deposition process of printed parts and detect micro defects, but it cannot detect macro defects generated after deposition. Moreover, the current defect detection method lacks compensation after real-time detection, resulting in the application value of defect detection not being fully reflected. Summary of the Invention

[0004] The purpose of the present invention is to provide a system and method for real-time detection and compensation of defects in multi-axis FDM printing, which can detect in real time, display the defect location, quickly correct the defects and make intelligent printing start and stop decisions.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A system for real-time detection and compensation of defects in multi-axis FDM printing, comprising

[0007] Multi-axis FDM printing system for FDM printing;

[0008] A microscopic camera is used to capture the image of the material extruded by the nozzle of the FDM extrusion mechanism of the multi-axis FDM printing system in real time to obtain a microscopic image;

[0009] A macro camera is used to capture real-time images of the side surface of an FDM print produced by a multi-axis FDM printing system to obtain a macro image. The multi-axis FDM printing system can drive the FDM print to rotate, enabling the macro camera to capture images of the side surface of the FDM print without blind spots.

[0010] The processing module connects and controls the multi-axis FDM printing system, micro camera and macro camera, and processes and calculates the micro and macro images collected in real time to display the three-dimensional position of the defect. At the same time, it adjusts the extrusion amount of the FDM extrusion mechanism in real time to achieve rapid correction compensation or intelligent decision-making on printing start and stop.

[0011] The processing module is a high-performance computer.

[0012] The processing operations include qualitative defect detection of macroscopic images.

[0013] The processing operations include defect quantitative analysis of the microscopic images.

[0014] The multi-axis FDM printing system includes a grooved platform fixed horizontally on the ground, and a multi-axis robotic arm and a fixed base fixed on the grooved platform. The FDM extrusion mechanism is installed at the end of the multi-axis robotic arm. A high-precision turntable is fixedly installed on the fixed base via a positioning steel plate. A clamp is fixedly installed on the high-precision turntable via a clamp fixing flange. The clamp is used to clamp the FDM printed part.

[0015] The microscopic camera is fixed to the end of the multi-axis robotic arm through a microscopic camera bracket.

[0016] The macro camera is fixed on the grooved platform through a macro camera bracket.

[0017] The multi-axis robotic arm, the fixed base and the macro camera bracket are located on the same straight line.

[0018] A method for real-time defect detection and compensation for multi-axis FDM printing, comprising:

[0019] S1. Defect dataset construction and model training: Based on the collected macro and micro images of the printing process and the open source FDM printing defect dataset, a macro defect target detection dataset and a micro defect classification dataset are formed respectively, and then a macro visual qualitative detection model and a micro visual qualitative detection model are trained;

[0020] S2. Real-time macro and micro defect detection: Macro and micro images are acquired in real time. After preprocessing, they are input into the corresponding macro visual qualitative detection model and micro visual qualitative detection model for inference, and the macro visual qualitative detection results and micro visual qualitative detection results are output in real time.

[0021] S3. Defect Position Calculation: Based on the macro camera's internal and external parameters and the macroscopic visual qualitative inspection results, the 3D position of macro defects is calculated. The 3D position of micro defects is calculated by combining the micro camera's calibration parameters with the FDM extrusion mechanism's nozzle's real-time posture. The categories of macro and micro defects and their corresponding 3D coordinate data are visualized in real time in the form of a superimposed model.

[0022] S4. Quantitative analysis of micro defects and feedback correction of printing parameters: Based on the microscopic visual qualitative inspection results, the quantitative indicators of defects are calculated to obtain the quantitative analysis results of micro defects. At the same time, the deviation of the defect quantitative value calculated in real time is compared with the preset threshold, the parameter correction amount is calculated, and the corresponding relative extrusion amount is adjusted in real time;

[0023] S5. Intelligent printing start and stop decision-making: During the printing process, the system receives qualitative detection results and 3D positions of macro and micro defects in real time, and generates a print quality evaluation index through weighted defect calculation. When the index falls below the preset threshold, a "stop printing" command is sent to the multi-axis FDM printing system.

[0024] The training in step S1 is to train the Yolov8 and Yolov8-cls network models and adjust the parameters.

[0025] Beneficial effects of the present invention:

[0026] 1. Able to perform real-time macro and micro inspection of FDM printing defects simultaneously;

[0027] 2. Ability to visualize defect locations;

[0028] 3. Realize real-time closed-loop feedback correction of printing parameters and intelligent printing start and stop decision-making based on the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Flowchart of the system for real-time detection and compensation of defects in multi-axis FDM printing;

[0030] Figure 2 This is a diagram of the multi-axis FDM printing platform and visual hardware system;

[0031] Figure 3 Flowchart for qualitative detection of macro defects;

[0032] Figure 4 Flow chart for qualitative detection of micro defects;

[0033] Figure 5 Calculate the flow chart for defect location;

[0034] Figure 6 Flowchart for quantitative analysis of micro defects and correction of printing parameters feedback;

[0035] Figure 7 Provide a start-stop decision flow chart for intelligent printing;

[0036] In the picture:

[0037] Multi-axis robotic arm 1; micro camera bracket 2; micro camera 3; FDM extrusion mechanism 4; FDM printed part 5; clamp 6; clamp fixing flange 7; high-precision turntable 8; positioning steel plate 9; fixed base 10; macro camera 11; macro camera bracket 12; grooved platform 13. DETAILED DESCRIPTION

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0039] like Figure 1-2 As shown, the system for real-time detection and compensation of defects in multi-axis FDM printing is described in detail:

[0040] A system for real-time defect detection and compensation in multi-axis FDM printing includes the following parts:

[0041] Multi-axis FDM printing system, processing module, macro camera 11, macro camera bracket 12, micro camera 3 and micro camera bracket 2.

[0042] The multi-axis FDM printing system comprises a multi-axis robotic arm 1, an FDM extruder 4, a clamp 6, a clamp fixing flange 7, a high-precision turntable 8, a positioning steel plate 9, a fixed base 10, and a grooved platform 13. The multi-axis robotic arm 1 is connected to the FDM extruder 4 via two fixing screws; the clamp 6, the clamp fixing flange 7, the high-precision turntable 8, the positioning steel plate 9, and the fixed base 10 are each connected in sequence via screws; the multi-axis robotic arm 1 and the fixed base 10 are each connected to the grooved platform via fixing screws, with the distance between them being half the arm span of the multi-axis robotic arm 1.

[0043] The macro camera 11 and the macro camera support 12 are connected by two fixing screws, and the macro camera support 12 is fixed to the grooved platform 13 by bolts. The distance between the fixed base 10 and the macro camera support 12 is adjusted according to the size of the FDM print 5 and the field of view of the macro camera 11 to ensure that the FDM print 5 is completely covered by the field of view of the macro camera 11.

[0044] The micro camera 3 and the micro camera bracket 2 are connected by two fixing screws, and the micro camera bracket 2 is fixed to the flange at the end of the multi-axis robotic arm 1 by two screws.

[0045] The processing module is a high-performance computer and is connected to the macro camera 11 and the micro camera 3 via two data lines to control the macro camera 11 and the micro camera 3 and collect images. At the same time, it can generate scripts to control the multi-axis FDM printing process.

[0046] The multi-axis robotic arm 1, the fixed base 10 and the macro camera bracket 12 are positioned in a straight line.

[0047] The micro camera 3 is fixed to the end of the multi-axis robotic arm 1 via a micro camera bracket 2, capturing real-time images of the material being extruded from the nozzle of the FDM print head of the FDM extrusion mechanism 4. The macro camera 11 is fixed to the grooved platform 13 via a macro camera bracket 12, capturing real-time images of the FDM printed side surface. Simultaneously, during printing, the multi-axis FDM printing system's high-precision turntable 8 rotates the FDM printed part 5, enabling the macro camera 11 to capture a comprehensive view of the FDM printed part's side surface. This overcomes the limitation of a single macro camera in being unable to cover the entire side surface and avoids the high cost of using multiple cameras to cover the entire side surface.

[0048] Real-time acquisition of macro images and real-time acquisition of micro images are achieved through the macro camera 11 and the micro camera 3 respectively. Then, qualitative detection of defects in macro images and quantitative analysis of defects in micro images are achieved through a series of image preprocessing and model reasoning. Finally, three-dimensional position display of defects and rapid correction as well as intelligent printing start and stop decision are achieved based on the detection results. Among them, a series of image preprocessing and model reasoning include unified preprocessing of the images to be detected and the use of Yolov8 convolutional network to achieve qualitative analysis of defects and the use of improved gray-level co-occurrence matrix (GLCM) to achieve quantitative analysis based on qualitative analysis; qualitative analysis refers to the classification and determination of defect types; quantitative analysis refers to the numerical calculation of defective images based on qualitative analysis.

[0049] like Figure 1-7 As shown, the method of real-time detection and compensation of defects in multi-axis FDM printing is described in detail:

[0050] A method for real-time defect detection and compensation for multi-axis FDM printing, comprising the following steps:

[0051] S1. Defect dataset construction and model training: The images of each printing process are collected through the data acquisition script to form macro and micro unlabeled datasets. The macro unlabeled dataset and the corresponding macro open source FDM defect dataset are selected one by one according to the preset macro defect category using the LabelMe annotation tool. After unifying the image size, they are divided into training, validation and test sets at a ratio of 0.8:0.1:0.1 to form a macro defect target detection dataset; The micro unlabeled dataset and the corresponding micro open source FDM defect dataset are classified one by one according to the preset micro defect category using the LabelMe annotation tool. After unifying the image size, they are divided into training, validation and test sets at a ratio of 0.8:0.1:0.1 to form a micro defect classification dataset. After unified image preprocessing of the macro defect target detection dataset and the micro defect classification dataset, the Yolov8 and Yolov8-cls network models are trained and the parameters are adjusted. The Yolov8 and Yolov8-cls models with the best performance in the test set are used as the macro and micro visual qualitative detection models;

[0052] Among them, image preprocessing includes pixel value normalization, geometric transformation, and color dithering;

[0053] S2. Real-time macro and micro defect detection: During the printing process, frame images of the macro camera 11 and the micro camera 3 are acquired in real time. After image preprocessing in step S1, the images are input into the corresponding macro visual qualitative detection model and micro visual qualitative detection model for inference, and the macro visual qualitative detection results and the micro visual qualitative detection results are output in real time;

[0054] Among them, the macroscopic visual qualitative inspection results include the category and frame position of the defect in the macroscopic image; the microscopic visual qualitative inspection results are the category of the defect in the microscopic image;

[0055] S3. Defect Position Calculation: Based on the intrinsic and extrinsic parameters of the macro camera 11 and the results of the macroscopic visual qualitative inspection, the pixel coordinates are projected onto the 3D model coordinate system of the printing platform through a spatial camera projection matrix geometric transformation to achieve the 3D position calculation of the macro defect. For the micro camera 3, its calibration parameters are combined with the real-time position of the FDM print head nozzle of the FDM extruder 4. Similarly, the nozzle position coordinates are mapped to the model coordinate system through a spatial geometric transformation to obtain the 3D position of the micro defect. Finally, the categories of macro and micro defects and the corresponding 3D coordinate data are sent to a high-performance computer via a computer script and displayed in real time as an overlay model in the user interface.

[0056] The internal parameters of the macro camera 11 are the camera's own "lens parameters", including focal length, principal point position and distortion, which are used to project real-world points onto the image. The external parameters of the macro camera 11 are the camera's "position and orientation" in space, which are used to convert the coordinates on the image back to real-world three-dimensional coordinates. The real-time nozzle pose is provided by the multi-axis FDM printer control system.

[0057] S4. Quantitative Analysis of Micro-Defects and Feedback Correction of Printing Parameters: Based on the microscopic visual qualitative inspection results, improved gray-level co-occurrence matrix (GLCM) features are extracted, and quantitative indicators of defects are calculated based on these features to obtain quantitative analysis results of micro-defects. The defect quantification values ​​and relative extrusion amounts calculated in real time are input into a closed-loop system based on PID control. By comparing the deviation between the quantified values ​​and the preset thresholds, the parameter correction amount is calculated according to the proportional, integral, and differential terms, and the corresponding relative extrusion amount of the FDM extruder 4 is adjusted in real time, thereby alleviating micro-defects caused by environmental interference, equipment fluctuations, etc. and maintaining the stability of the printing process.

[0058] Among them, the improved gray-level co-occurrence matrix (GLCM) features include contrast, energy, entropy, homogeneity, etc.

[0059] S5. Intelligent printing start-stop decision-making: During the printing process, the system receives qualitative detection results and three-dimensional positions of macro and micro defects in real time, and generates a print quality evaluation index through weighted defect calculation. When the index falls below the preset threshold, it automatically sends a "stop printing" command to the multi-axis FDM printing system to realize intelligent start-stop decision-making of the printing process.

[0060] Of course, the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention also fall within the scope of protection of the present invention.

Claims

1. A system for real-time defect detection and compensation in multi-axis FDM printing, characterized by: Includes a multi-axis FDM printing system for FDM printing; A microscopic camera (3) is used to capture in real time an image of a material being extruded from a nozzle of an FDM extrusion mechanism (4) of a multi-axis FDM printing system to obtain a microscopic image; A macro camera (11) is used to capture in real time a side surface formed image of an FDM printed part (5) printed by a multi-axis FDM printing system to obtain a macro image. The multi-axis FDM printing system can drive the FDM printed part (5) to rotate, so that the macro camera (11) can capture an image of the side surface of the FDM printed part (5) without blind spots. The processing module connects and controls the multi-axis FDM printing system, the microscopic camera (3) and the macroscopic camera (11), processes and calculates the microscopic images and macroscopic images collected in real time, displays the three-dimensional position of the defect, and adjusts the extrusion amount of the FDM extrusion mechanism (4) in real time to achieve rapid correction compensation or intelligent decision-making for printing start and stop.

2. The system according to claim 1, wherein: The processing module is a high-performance computer.

3. The system according to claim 1, wherein: The processing operations include qualitative defect detection of macroscopic images.

4. The system according to claim 3, wherein: The processing operations include defect quantitative analysis of the microscopic images.

5. The system according to claim 1, wherein: The multi-axis FDM printing system comprises a grooved platform (13) fixed horizontally on the ground, and a multi-axis robotic arm (1) and a fixed base (10) fixed on the grooved platform (13); the FDM extrusion mechanism (4) is mounted at the end of the multi-axis robotic arm (1); a high-precision turntable (8) is fixedly mounted on the fixed base (10) via a positioning steel plate (9); a clamp (6) is fixedly mounted on the high-precision turntable (8) via a clamp fixing flange (7); and the clamp (6) is used to clamp the FDM printed part (5).

6. The system according to claim 5, characterized in that: The microscopic camera (3) is fixed to the end of the multi-axis mechanical arm (1) via a microscopic camera bracket (2).

7. The system according to claim 5, characterized in that: The macro camera (11) is fixed on a grooved platform (13) via a macro camera bracket (12).

8. The system according to claim 7, characterized in that: The multi-axis robotic arm (1), the fixed base (10) and the macro camera support (12) are located on the same straight line.

9. A method for real-time defect detection and compensation in multi-axis FDM printing, characterized by: include S1. Defect dataset construction and model training: Based on the collected macro and micro images of the printing process and the open source FDM printing defect dataset, a macro defect target detection dataset and a micro defect classification dataset are formed respectively, and then a macro visual qualitative detection model and a micro visual qualitative detection model are trained; S2. Real-time macro and micro defect detection: Macro and micro images are acquired in real time. After preprocessing, they are input into the corresponding macro and micro visual qualitative detection models for inference, and the macro and micro visual qualitative detection results are output in real time. S3. Defect position calculation: Based on the internal parameters, external parameters and macroscopic visual qualitative detection results of the macro camera (11), the three-dimensional position calculation of the macro defect is completed; combined with the calibration parameters of the micro camera (3) and the real-time position of the nozzle of the FDM extrusion mechanism (4), the three-dimensional position calculation of the micro defect is completed, and the categories of macro and micro defects and the corresponding three-dimensional coordinate data are visualized in real time in the form of a superimposed model; S4. Quantitative analysis of micro defects and feedback correction of printing parameters: Based on the microscopic visual qualitative inspection results, the quantitative indicators of defects are calculated to obtain the quantitative analysis results of micro defects. At the same time, the deviation of the defect quantitative value calculated in real time is compared with the preset threshold, the parameter correction amount is calculated, and the corresponding relative extrusion amount is adjusted in real time; S5. Intelligent printing start and stop decision-making: During the printing process, the system receives qualitative detection results and 3D positions of macro and micro defects in real time, and generates a print quality evaluation index through weighted defect calculation. When the index falls below a preset threshold, a "stop printing" command is sent to the multi-axis FDM printing system.

10. The method according to claim 9, characterized in that: The training in step S1 is to train the Yolov8 and Yolov8-cls network models and adjust the parameters.

Images