Real-time monitoring device and method for fused deposition modeling based on artificial intelligence

Abstract

A real-time monitoring device and method for fused deposition modeling based on artificial intelligence. In the device, a real-time video channel is established between a control panel and a server to intercept frame data of the image information to generate an image. The four printing parameters at the current moment are acquired in real time by the octoprint software and transmitted to the server by the controller. The control panel transmits the G code to the FDM 3D printer to adjust the printing parameters so that the camera collects image data sets under different parameters for training the artificial intelligence model unit. The image data sets are randomly divided into a training set and a test set. The training set data is used to train the artificial intelligence model, and the accuracy of the artificial intelligence model is verified on the test set. The image data to be predicted is used as input to judge the status and class label of the current printing parameters through the trained artificial intelligence model. The Python program unit generates G code based on the status of the current printing parameters, and the generated G code is transmitted to the printer through the control panel.

Description

Technical Field

[0001] The present invention relates to the field of additive manufacturing technology, and in particular to an artificial intelligence-based fused deposition modeling real-time monitoring device and method. Background Art

[0002] 3D printing technology, with its advantages of customization, rapid prototyping, and high efficiency, has found widespread application in fields such as medicine, aerospace, automotive, energy, and tools. FDM printing, a key branch of 3D printing, involves melting thermoplastic materials like PLA and ABS and depositing them layer by layer through a computer-controlled nozzle to create three-dimensional objects.

[0003] Correctly setting process parameters during the printing process is crucial to print quality. However, due to improperly set process parameters (nozzle Z-axis height, nozzle temperature, nozzle movement speed, and nozzle material extrusion rate) and the influence of random factors during the printing process, defects often occur in printed parts. This not only affects part print quality and manufacturing efficiency, but also results in waste of material and time during the printing process. Therefore, real-time monitoring of part print quality and real-time feedback control of process parameters during the printing process are crucial to ensuring part print quality.

[0004] Currently available monitoring methods mostly use non-contact sensors, such as camera monitoring (Chinese patent: 202211170763.1, titled: "Continuous fiber 3D printing process monitoring method based on artificial intelligence image recognition") and acoustic wave sensors (Chinese patent: CN109187755B, titled: "A method for online ultrasonic non-destructive testing based on 3D printing"). However, these methods can only achieve real-time monitoring of the printing process and cannot use the monitoring results to feedback control the printing process. The patent still under review: CN116442529A also proposes the need for feedback control of the printing process, but this method only addresses the problem of uneven printing temperature distribution and does not provide sufficiently detailed and executable methods, such as which process parameters to adjust and how to combine process monitoring results with real-time feedback control of process parameters. In addition, the data set used by this method is completely manually annotated, lacking an automatic data annotation method, making it difficult to construct a large data set.

[0005] In summary, there is currently a lack of methods to monitor the FDM printing process in real time, determine the quality of current process parameters, and provide real-time feedback control for process parameters that may cause part defects. This results in the printer operating under incorrect printing parameters, further causing part quality defects. This severely limits part printing quality and manufacturing efficiency, and also results in waste of materials and time during the printing process.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0007] To address the challenges of existing technologies, this paper proposes an artificial intelligence-based real-time monitoring device and method for fused deposition modeling (FDM). This device monitors print quality during FDM printing and determines the quality of current printing parameters. Furthermore, it enables feedback control of four process parameters during the printing process: nozzle Z-axis height, nozzle temperature, nozzle movement speed, and nozzle material extrusion rate. Furthermore, during the dataset construction phase, this paper provides an automated data collection and labeling method, providing a foundation for constructing diverse large datasets.

[0008] The present invention is achieved through the following technical solutions:

[0009] A real-time monitoring device for fused deposition modeling based on artificial intelligence includes:

[0010] A 3D printing platform comprising:

[0011] FDM 3D printer, which is used for fused deposition modeling,

[0012] A camera is fixed to one side of the nozzle of the FDM 3D printer to capture image information at the nozzle during the printing process.

[0013] The server includes a Python program unit and an artificial intelligence model unit that executes an artificial intelligence algorithm based on computer vision. The Python program unit automatically randomly generates a new set of printing parameters from a preset uniform distribution of printing parameters and generates G code.

[0014] The control board connects the camera, server and FDM 3D printer to serve as a gateway for information interaction and transmission between the three. The control board includes the OctoPrint software unit, which

[0015] A real-time video channel is constructed between the control board and the server to intercept the frame data of the image information to generate an image. The four printing parameters at the current moment are acquired in real time by the octoprint software and transmitted to the server by the controller. The printing parameters are divided into three categories: too high, normal, and too low according to the preset range and used as the class label of the image at the corresponding moment. The control board transmits the G code to the FDM 3D printer to adjust the printing parameters so that the camera collects image data sets under different parameters for training the artificial intelligence model unit. The image data sets are randomly divided into training sets and test sets. The training set data is used to train the artificial intelligence model, and the accuracy of the artificial intelligence model is verified on the test set. The image data to be predicted is used as input to judge the status and class label of the current printing parameters through the trained artificial intelligence model. The python program unit generates G code based on the status of the current printing parameters. The generated G code is transmitted to the printer through the control board to perform the adjustment operation.

[0016] In the artificial intelligence-based fused deposition modeling real-time monitoring device, the camera is a high-resolution camera, which is fixed to the nozzle of the FDM 3D printer using a fixed fixture to collect image information at the nozzle during the printing process in real time.

[0017] In the artificial intelligence-based fused deposition modeling real-time monitoring device, the control panel is connected to the camera and printer via data cables, and the control panel is connected to the server via a network to realize the transmission of image information and control information between the printer and the server.

[0018] In the artificial intelligence-based fused deposition modeling real-time monitoring device, the printing parameters include nozzle Z-axis height, nozzle temperature, nozzle movement speed, and nozzle material extrusion rate.

[0019] In the artificial intelligence-based real-time monitoring device for fused deposition modeling, the artificial intelligence model unit includes a convolutional neural network, a deep residual network or an Inception image classification model.

[0020] A method for real-time control of fused deposition modeling process parameters based on an artificial intelligence-based fused deposition modeling real-time monitoring device comprises the following steps:

[0021] Step 1: Build a 3D printing platform: A camera is fixed to one side of the nozzle of the FDM 3D printer to capture image information at the nozzle during the printing process. The server includes a Python program unit and an artificial intelligence model unit that executes an artificial intelligence algorithm based on computer vision. The Python program unit automatically randomly generates a new set of printing parameters from a preset uniform distribution of printing parameters and generates G-code. A control board connects the camera, server, and FDM 3D printer to serve as a gateway for information exchange and transmission among the three.

[0022] Step 2: Automated diverse data set construction: The control board collects image information at the nozzle through the camera and builds a real-time video channel between the control board and the server based on this information, intercepting frame data to generate images; the four printing parameters at the current moment are acquired in real time by the OctoPrint software unit and transmitted to the server by the controller. The printing parameters are divided into three categories according to the preset range: too high, normal, and too low, and used as the class label of the image at the corresponding moment; the Python program unit running on the server automatically randomly generates a new set of printing parameters from the preset uniform distribution of the four parameters, and generates the relevant G code and transmits it to the printer via the control board, adjusting the printing parameters so that the camera collects diverse image data printed under different parameters for subsequent artificial intelligence model training. The endpoint values ​​of the uniform distribution are preset according to the characteristics of the printer, the properties of the material, and the requirements of the printing process;

[0023] Step 3: AI model training: The AI ​​model unit monitors the current printing status and judges the quality of four types of printing parameters. First, data preparation is performed using the diverse image dataset constructed in step 2. The image dataset is randomly divided into training and test sets in an 8:2 ratio. The image data is standardized using the Z-score standardization method to convert the data to a common scale to accelerate the convergence of the model training process.

[0024] Then, the model structure is designed. A convolutional neural network is used to build the model to extract image features. The final model output is the judgment result of the quality of the four types of process parameters corresponding to the current image. The Pytorch framework is used to define the model layer structure. The cross entropy loss function is selected as the loss function for this classification task. The adaptive optimizer is selected for parameter optimization. Iterative training of no less than 5000 steps is carried out until the model converges. If the model does not converge, the learning rate, batch size or the number of training rounds are adjusted until the model converges.

[0025] Step 4: Real-time image acquisition: The control board is connected to the camera on one hand and to the FDM 3D printer and server on the other. The camera collects image information at the nozzle. The control board and the OctoPrint software unit establish a real-time video channel between the server and the control board. The server runs the Python program unit, which is responsible for intercepting frame data from the real-time video channel to generate images, thereby realizing real-time monitoring of the printing process.

[0026] Step 5: Real-time judgment of the status of printing process parameters: The trained artificial intelligence model is used to predict and judge the status of the four types of printing parameters. The artificial intelligence model uses the collected image data as input and outputs the quality of the status of the four types of printing parameters, including too high, normal, and too low.

[0027] Step 6: Feedback control: The artificial intelligence model classifies the real-time image to obtain the current printing status and the quality classification results of the four types of process parameters. Based on the characteristics of the printer, the material properties and the requirements of the printing process, the step value for each adjustment of each process parameter is preset in advance. When generating the G code, it is ensured that the adjustment of each process parameter is limited by the preset step value to ensure that each adjustment is within a reasonable range. The generated G code is transmitted to the FDM 3D printer through the control board to perform the adjustment operation. The FDM 3D printer parses the G code and gradually adjusts the process parameters according to the step value.

[0028] In the method for real-time control of fused deposition modeling process parameters, the control board includes a Raspberry Pi.

[0029] In the real-time control method of fused deposition modeling process parameters, the preset ranges of the printing parameters are: the normal range of the nozzle Z-axis height [-4.25mm, -3.85mm], the normal range of the nozzle temperature [185°C, 220°C], the normal range of the nozzle movement speed [80%, 120%], and the normal range of the nozzle material extrusion rate [85%, 120%].

[0030] In the real-time control method of fused deposition modeling process parameters, the four printing parameters are evenly distributed as follows: nozzle Z-axis height ~U(-4.25,-3.85), nozzle temperature ~U(185,220), nozzle movement speed ~U(70%,130%), and nozzle material extrusion rate ~U(60%,140%).

[0031] In the method for real-time control of fused deposition modeling process parameters, the data set is randomly divided into a training set and a test set in a ratio of 8:2.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] This invention introduces a real-time monitoring system that, through automated diverse image data acquisition and a computer vision-based artificial intelligence model, enables real-time monitoring of the fused deposition modeling (FDM) process, thereby improving printing quality and efficiency. By incorporating a computer vision-based artificial intelligence model, the invention intelligently determines the current printing status and the quality of four types of printing parameters. A feedback control module generates corresponding G-code commands for real-time correction, enhancing the intelligence and automation of the printing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Various other advantages and benefits of the present invention will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are intended only to illustrate preferred embodiments and are not to be construed as limiting the present invention. It should be understood that the drawings described below are merely examples of the present invention, and that those skilled in the art will be able to derive other drawings from these drawings without inventive effort. Throughout the drawings, identical reference numerals are used to denote identical components.

[0035] In the attached figure:

[0036] Figure 1 A schematic structural diagram of the real-time monitoring device provided by the present invention;

[0037] Figure 2 This is a schematic diagram of the process of offline data set construction and online printing parameter real-time correction in this embodiment;

[0038] Figure 3 Schematic diagram of the monitoring method in this embodiment;

[0039] Figure 4 Schematic diagram of the structure of the camera fixing fixture in this embodiment;

[0040] Figure 5 This is a schematic diagram of the printer adjustment printing process in this embodiment.

[0041] Figure 6 Schematic diagram of the artificial intelligence model based on the residual neural network built in this embodiment;

[0042] Figure 7 This is a schematic diagram of the residual block structure built in this embodiment;

[0043] Figure 8 This is the multi-head convolutional neural network training process built in this embodiment.

[0044] The present invention will be further explained below with reference to the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0045] Specific embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although specific embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0046] It should be noted that certain words are used in the specification and claims to refer to specific components. Those skilled in the art should understand that technicians may use different nouns to refer to the same component. This specification and claims do not use the difference in nouns as a way to distinguish components, but use the difference in the functions of the components as the criterion for distinction. As mentioned throughout the specification and claims, "including" or "comprising" is an open term, so it should be interpreted as "including but not limited to". The subsequent description of the specification is a preferred embodiment of the present invention, but the description is based on the general principles of the specification and is not intended to limit the scope of the invention. The scope of protection of the present invention shall be as defined in the attached claims.

[0047] To facilitate understanding of the embodiments of the present invention, further explanation will be given below using specific embodiments as examples in conjunction with the accompanying drawings, and the accompanying drawings do not constitute a limitation on the embodiments of the present invention.

[0048] like Figures 1 to 8 As shown, the artificial intelligence-based fused deposition modeling real-time monitoring device includes:

[0049] A 3D printing platform comprising:

[0050] FDM 3D printer, which is used for fused deposition modeling,

[0051] A camera is fixed to one side of the nozzle of the FDM 3D printer to capture image information at the nozzle during the printing process.

[0052] The server includes a Python program unit and an artificial intelligence model unit that executes an artificial intelligence algorithm based on computer vision. The Python program unit automatically randomly generates a new set of printing parameters from a preset uniform distribution of printing parameters and generates G code.

[0053] The control board connects the camera, server and FDM 3D printer to serve as a gateway for information interaction and transmission between the three. The control board includes the OctoPrint software unit, which

[0054] A real-time video channel is constructed between the control board and the server to intercept the frame data of the image information to generate an image. The four printing parameters at the current moment are acquired in real time by the octoprint software and transmitted to the server by the controller. The printing parameters are divided into three categories: too high, normal, and too low according to the preset range and used as the class label of the image at the corresponding moment. The control board transmits the G code to the FDM 3D printer to adjust the printing parameters so that the camera collects image data sets under different parameters for training the artificial intelligence model unit. The image data sets are randomly divided into training sets and test sets. The training set data is used to train the artificial intelligence model, and the accuracy of the artificial intelligence model is verified on the test set. The image data to be predicted is used as input to judge the status and class label of the current printing parameters through the trained artificial intelligence model. The python program unit generates G code based on the status of the current printing parameters. The generated G code is transmitted to the printer through the control board to perform the adjustment operation.

[0055] In a preferred embodiment of the artificial intelligence-based fused deposition modeling real-time monitoring device, the camera is a high-resolution camera, which is fixed to the nozzle of the FDM 3D printer using a fixed fixture to collect image information at the nozzle during the printing process in real time.

[0056] In a preferred embodiment of the artificial intelligence-based fused deposition modeling real-time monitoring device, the control panel is connected to the camera and the printer via data cables, and the control panel is connected to the server via a network to realize the transmission of image information and control information between the printer and the server.

[0057] In a preferred embodiment of the artificial intelligence-based fused deposition modeling real-time monitoring device, the printing parameters include nozzle Z-axis height, nozzle temperature, nozzle movement speed, and nozzle material extrusion rate.

[0058] In a preferred embodiment of the artificial intelligence-based fused deposition modeling real-time monitoring device, the artificial intelligence model unit includes a convolutional neural network, a deep residual network or an Inception image classification model.

[0059] A method for real-time control of fused deposition modeling process parameters based on an artificial intelligence-based fused deposition modeling real-time monitoring device comprises the following steps:

[0060] Step 1: Build a 3D printing platform: A camera is fixed to one side of the nozzle of the FDM 3D printer to capture image information at the nozzle during the printing process. The server includes a Python program unit and an artificial intelligence model unit that executes an artificial intelligence algorithm based on computer vision. The Python program unit automatically randomly generates a new set of printing parameters from a preset uniform distribution of printing parameters and generates G-code. A control board connects the camera, server, and FDM 3D printer to serve as a gateway for information exchange and transmission among the three.

[0061] Step 2: Automated diverse data set construction: The control board collects image information at the nozzle through the camera and builds a real-time video channel between the control board and the server based on this information, intercepting frame data to generate images; the four printing parameters at the current moment are acquired in real time by the OctoPrint software unit and transmitted to the server by the controller. The printing parameters are divided into three categories according to the preset range: too high, normal, and too low, and used as the class label of the image at the corresponding moment; the Python program unit running on the server automatically randomly generates a new set of printing parameters from the preset uniform distribution of the four parameters, and generates the relevant G code and transmits it to the printer via the control board, adjusting the printing parameters so that the camera collects diverse image data printed under different parameters for subsequent artificial intelligence model training. The endpoint values ​​of the uniform distribution are preset according to the characteristics of the printer, the properties of the material, and the requirements of the printing process;

[0062] Step 3: AI model training: The AI ​​model unit monitors the current printing status and judges the quality of four types of printing parameters. First, data preparation is performed using the diverse image dataset constructed in step 2. The image dataset is randomly divided into training and test sets in an 8:2 ratio. The image data is standardized using the Z-score standardization method to convert the data to a common scale to accelerate the convergence of the model training process.

[0063] Then, the model structure is designed. A convolutional neural network is used to build the model to extract image features. The final model output is the judgment result of the quality of the four types of process parameters corresponding to the current image. The Pytorch framework is used to define the model layer structure. The cross entropy loss function is selected as the loss function for this classification task. The adaptive optimizer is selected for parameter optimization. Iterative training of no less than 5000 steps is carried out until the model converges. If the model does not converge, the learning rate, batch size or the number of training rounds are adjusted until the model converges.

[0064] Step 4: Real-time image acquisition: The control board is connected to the camera on one hand and to the FDM 3D printer and server on the other. The camera collects image information at the nozzle. The control board and the OctoPrint software unit establish a real-time video channel between the server and the control board. The server runs the Python program unit, which is responsible for intercepting frame data from the real-time video channel to generate images, thereby realizing real-time monitoring of the printing process.

[0065] Step 5: Real-time judgment of the status of printing process parameters: The trained artificial intelligence model is used to predict and judge the status of the four types of printing parameters. The artificial intelligence model uses the collected image data as input and outputs the quality of the status of the four types of printing parameters, including too high, normal, and too low.

[0066] Step 6: Feedback control: The artificial intelligence model classifies the real-time image to obtain the current printing status and the quality classification results of the four types of process parameters. Based on the characteristics of the printer, the material properties and the requirements of the printing process, the step value for each adjustment of each process parameter is preset in advance. When generating the G code, it is ensured that the adjustment of each process parameter is limited by the preset step value to ensure that each adjustment is within a reasonable range. The generated G code is transmitted to the FDM 3D printer through the control board to perform the adjustment operation. The FDM 3D printer parses the G code and gradually adjusts the process parameters according to the step value.

[0067] In a preferred embodiment of the method for real-time control of fused deposition modeling process parameters, the control board includes a Raspberry Pi.

[0068] In a preferred embodiment of the method for real-time control of fused deposition modeling process parameters, the preset ranges of the printing parameters are: normal range of nozzle Z-axis height [-0.1mm, +0.1mm], normal range of nozzle temperature [200°C, 210°C], normal range of nozzle movement speed [80%, 120%], and normal range of nozzle material extrusion rate [85%, 120%].

[0069] In a preferred embodiment of the method for real-time control of fused deposition modeling process parameters, the four printing parameters are evenly distributed as nozzle Z-axis height ~U(-4.25,-3.85), nozzle temperature ~U(185,220), nozzle movement speed ~U(70%,130%), and nozzle material extrusion rate ~U(60%,140%).

[0070] In a preferred embodiment of the method for real-time control of fused deposition modeling process parameters, the data set is randomly divided into a training set and a test set in a ratio of 8:2.

[0071] In one embodiment, the method comprises the following steps:

[0072] Step 1: Build a 3D printing monitoring platform: The platform built in this example is as follows: Figure 1 The FDM 3D printer used is Ender3 S1 printer 300, and the high-resolution camera is Logitech C270 camera 100. Figure 4 The fixture shown is fixed to the nozzle 200, and the server 500 is configured to include Intel(R) Core TMThe i7-9700K CPU, NVIDIA GeForce RTX 2080Ti GPU, and 16GB of RAM are used to train and deploy the required AI models. The control board uses a Raspberry Pi 4B model 400, which has a local area network connection and serves as a gateway for information exchange and transmission between the camera, server, and printer.

[0073] Step 2: Automated diverse data set construction: Refer to the attached Figure 2 and attached Figure 5 In this example, Octopi and OctoPrint (both common open-source FDM printer software) are installed on a Raspberry Pi. The Raspberry Pi uses a camera to capture image information at the nozzle site. Based on this information, a real-time video link is established between the Pi and a server, capturing frame data and generating images. Four printing parameters at the current moment are captured in real time by the OctoPrint software and transmitted from the Pi to the server. Printing parameters are classified as excessive, normal, or low within preset ranges (normal range for nozzle Z-axis height [-0.1mm, +0.1mm], normal range for nozzle temperature [200°C, 210°C], normal range for nozzle speed [80%, 120%], and normal range for nozzle material extrusion rate [85%, 120%]). These parameters are then used as class labels for the corresponding images. This example uses automated data annotation, eliminating the need for manual labeling and thus enabling the collection of large datasets for training. Large amounts of data and diverse data help improve model generalization and accuracy.

[0074] To obtain diverse image data when printing under different process parameters, every time the server collects and prints 150 photos during the printing process, a Python program running on the server automatically randomly generates a new set of printing parameters from a preset uniform distribution of four parameters (nozzle Z-axis height ~U(-4.25,-3.85), nozzle temperature ~U(185,220), nozzle movement speed ~U(70%,130%), and nozzle material extrusion rate ~U(60%,140%)). This program then generates the relevant G-code and transmits it to the printer via the Raspberry Pi. This adjusts the printing parameters so that the camera can capture diverse image data from prints under different parameters for subsequent AI model training. The endpoint values ​​of this uniform distribution are preset based on the printer's characteristics, material properties, and printing process requirements.

[0075] Step 3: AI model training: This example uses a deep residual network (ResNet) model to monitor the current printing status and determine the quality of four types of printing parameters. The training data is constructed using the automated dataset construction method demonstrated in Step 2. The dataset used in this example consists of nearly 1,000,000 photos generated by printing 192 parts. This dataset is randomly divided into a training set and a test set in an 8:2 ratio. The residual neural network model is trained using the training set data, and the model accuracy is verified on the test set.

[0076] Step 4: Real-time image acquisition: refer to the attached online monitoring and feedback control process. Figure 2 The Raspberry Pi control board is connected to the camera on one hand, and to the printer and server on the other hand. The image information at the nozzle is collected through the Logitech C270 camera. Using the Raspberry Pi 4B control board and the Octopi and OctoPrint software, a real-time video channel is established between the server and the Raspberry Pi. The server runs Python code, which is responsible for intercepting frame data from the video at a frequency of 2.5HZ to generate images, thereby realizing real-time monitoring of the printing process.

[0077] Step 5: Real-time judgment of printing process parameter status: For the real-time control stage, refer to the attached Figure 3 , use the method in step 4 to realize real-time image data acquisition, and use the trained artificial intelligence model to make predictions and judge the current status of the four types of printing parameters; the model uses the image data collected by the image acquisition module as input and outputs the quality (too high, normal, too low) of the four categories (nozzle Z-axis height, nozzle temperature, nozzle movement speed, nozzle material extrusion rate) displayed by the current printing status;

[0078] Step 6: Feedback control: refer to the attached Figure 3 In step five, the AI ​​model is used to classify the real-time image, obtaining the current printing status and the quality classification results of the four categories of process parameters. Based on the printer's characteristics, material properties, and printing process requirements, the step value for each process parameter adjustment is preset (in this example, the nozzle Z-axis height step value is ±0.04mm, the nozzle temperature is ±2°C, the nozzle movement speed is ±10%, and the nozzle material extrusion rate is ±8%). When generating G-code, ensure that the adjustment of each process parameter is limited by the preset step value, ensuring that each adjustment is within a reasonable range. The generated G-code is transmitted to the printer via the Raspberry Pi to perform the adjustment operation. The printer interprets the G-code and gradually adjusts the process parameters according to the step value. After this adjustment, new image data is collected and further evaluated by the AI ​​model, forming a feedback loop to ensure that the adjustment is based on the actual printing status and maintains stability with each adjustment.

[0079] In one embodiment, a real-time monitoring device for fused deposition modeling (FDM) based on artificial intelligence image classification comprises a pre-assembled 3D printing platform for monitoring; an image acquisition module that uses a camera to capture image information at the nozzle during the printing process; a process monitoring module that automatically monitors the printing process image information based on a computer vision artificial intelligence model and automatically determines the quality of four current printing parameters; and a feedback control module that generates G-code to adjust the four process parameters based on a control algorithm, enabling real-time correction of the process parameters. The 3D printing platform device for monitoring includes at least the following components: an FDM 3D printer for manufacturing parts; a high-resolution, follow-up camera fixed to one side of the printer nozzle for capturing images of the nozzle during printing; a server for executing a computer vision-based artificial intelligence algorithm; and a control board for transmitting image and control information between the printer and the server. The control board includes at least two USB ports, one connected to the printer control board and the other to the camera. The control board also has network connectivity, acting as a gateway to efficiently transmit image data and parameter control codes between the printer and the server.

[0080] The image acquisition module includes a camera, a control board, and a server. The control board is connected to the camera on one side and to the printer and server on the other. This allows for a real-time video link between the printer and server to be established based on the control board. The server runs Python code that captures frames of video at a 2.5Hz frequency to generate images. The image acquisition module has two main functions: first, it collects a large number of photos printed under different parameter combinations as a training dataset for the artificial intelligence classification model; second, it is used to capture photos of the printing process in real time during the real-time control phase.

[0081] During the dataset construction phase, an automated data collection and labeling method was used to generate diverse data. After collecting every 150 photos, the server program automatically adjusted the four printing parameters and continued collecting data under the new settings. Each data item's label was composed of the four printing parameters in their current state. These four parameters were categorized as excessive, low, and normal according to pre-set standards. The labels and data were mapped one-to-one and stored on the server for subsequent model training.

[0082] During the real-time control stage, the image acquisition module will collect image data in real time according to the above method, but will not collect the current status of the four types of parameters. The collected image data will be passed to the trained model for prediction to monitor the current printing status and automatically judge the pros and cons of the current four types of printing parameters.

[0083] The process monitoring module mainly relies on a trained artificial intelligence model based on computer vision. The model takes the image data collected by the image acquisition module as input and outputs the quality of four types of printing parameters (too high, normal, too low) displayed by the current printing status.

[0084] The feedback control module's primary function is to generate G-code commands to adjust the corresponding printing parameters based on the AI ​​model's judgment of the current printing parameters. If a printing parameter is judged to be too high, the feedback control module generates G-code to decrease it; if a printing parameter is judged to be too low, the feedback control module generates G-code to increase it; if a printing parameter is judged to be normal, the feedback control module considers the current parameter to be optimal and does not modify it. This feedback control is a step-by-step adjustment, with each parameter adjustment stepping by a preset adjustable step value.

[0085] In one embodiment, a method for real-time monitoring of fused deposition modeling (FDM) based on artificial intelligence image classification includes the following steps.

[0086] 1) Build a 3D printing monitoring platform: The device includes an FDM 3D printer, a high-resolution camera, a server, and a control panel.

[0087] 2) Automated diverse dataset construction: Under different parameter combinations, the system automatically adjusts four types of printing parameters and collects image data to construct diverse datasets for subsequent AI model training.

[0088] 3) Artificial Intelligence Model Training: Use models such as Convolutional Neural Networks (CNN), Deep Residual Networks (ResNet), and Inception to train computer vision-based artificial intelligence models to monitor the current printing status and judge the pros and cons of four types of printing parameters.

[0089] 4) Real-time image acquisition: The image information at the nozzle is collected through the camera, a real-time video channel is established using the control board and server, and frame data is intercepted to generate images.

[0090] 5) Real-time monitoring: In the real-time control stage, real-time image data is collected and predicted through the trained artificial intelligence model to determine the status of the current four types of printing parameters and achieve real-time monitoring.

[0091] 6) Feedback control: Based on the results of the artificial intelligence model, corresponding G-code commands are generated to achieve real-time correction of process parameters. Each adjustment is limited by a preset step value.

[0092] The artificial intelligence model for detecting the forming process is a residual neural network, which consists of a shared backbone network and four output heads, which correspond to four process parameters, namely material extrusion rate, nozzle movement speed, nozzle temperature, and Z-axis offset distance. The network structure diagram is shown in the figure. Figure 6 shown.

[0093] The shared backbone network used to extract image features consists of six residual blocks and three attention models. Each residual block includes a convolutional layer, batch normalization, activation function activation, and pooling layer. The residual block structure diagram is shown in the figure below. Figure 7 As shown in Figure 2, the ReLu function is used to activate the output of the convolutional layer after batch normalization, and the activation value is sent to the next layer.

[0094] After extracting feature information from the forming process image using a shared backbone network consisting of six serially connected residual blocks, its output is flattened into a column vector. A fully connected layer then maps the image features to the four process parameter states. This means that the four output heads share the feature information extracted by the shared backbone. During training, the weights of the shared backbone network are updated using the total loss of the four output heads. This means that the shared backbone network can learn the interactions between process parameters and the relationship between each process parameter and the forming state image, allowing it to propose multiple process parameter adjustment methods for a single undesirable forming state.

[0095] In order to obtain an effective and stable multi-head convolutional network to predict accurate process parameter states, the training process of the multi-head convolutional network is divided into three parts, such as Figure 8 As shown in the figure, the network is first trained using images from the training set that display distinct process parameter state features. Due to these distinct state features, the multi-head convolutional neural network can effectively and quickly capture image information and detect important features. Next, the pre-trained multi-head convolutional neural network model is fine-tuned on the full training set to improve its state recognition capabilities for images with less distinct features. Finally, considering the imbalance of the training set samples—that is, due to the influence of forming characteristics, excessively low Z-axis offsets can lead to nozzle-heatbed collisions and forming failures—the number of samples with excessively high Z-axis offsets far exceeds those with excessively low offsets. To mitigate the impact of sample imbalance on the predictive performance of the multi-head convolutional neural network, a transfer learning approach is used in the final stage of training to freeze the shared backbone network weights and fine-tune the fully connected layer weights using balanced samples, resulting in improved process parameter state prediction results. Finally, the accuracy of the trained model is verified on the test set, where the classification accuracy for each parameter is 87.1% for flow rate, 86.4% for lateral velocity, 85.5% for Z-offset, and 78.3% for hot-end temperature.

[0096] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments and application fields. The above-mentioned specific embodiments are merely illustrative and instructive, and are not restrictive. A person skilled in the art, guided by this specification and without departing from the scope of protection of the claims of the present invention, may also devise various forms, all of which fall within the scope of protection of the present invention.

Claims

1. A real-time monitoring device for fused deposition modeling based on artificial intelligence, characterized in that: These include, A 3D printing platform comprising: FDM 3D printer, which is used for fused deposition modeling, A camera is fixed to one side of the nozzle of the FDM 3D printer to capture image information at the nozzle during the printing process. The server includes a Python program unit and an artificial intelligence model unit that executes an artificial intelligence algorithm based on computer vision. The Python program unit automatically randomly generates a new set of printing parameters from a preset uniform distribution of printing parameters and generates G code. The control board connects the camera, server and FDM 3D printer to serve as a gateway for information interaction and transmission between the three. The control board includes the OctoPrint software unit, which A real-time video channel is constructed between the control board and the server to intercept the frame data of the image information to generate an image. The four printing parameters at the current moment are acquired in real time by the octoprint software and transmitted to the server by the controller. The printing parameters are divided into three categories: too high, normal, and too low according to the preset range and used as the class label of the image at the corresponding moment. The control board transmits the G code to the FDM 3D printer to adjust the printing parameters so that the camera collects image data sets under different parameters for training the artificial intelligence model unit. The image data sets are randomly divided into training sets and test sets. The training set data is used to train the artificial intelligence model, and the accuracy of the artificial intelligence model is verified on the test set. The image data to be predicted is used as input to judge the status and class label of the current printing parameters through the trained artificial intelligence model. The python program unit generates G code based on the status of the current printing parameters. The generated G code is transmitted to the printer through the control board to perform the adjustment operation.

2. The artificial intelligence-based fused deposition modeling real-time monitoring device according to claim 1, characterized in that: The camera is a high-resolution camera that is fixed to the nozzle of the FDM 3D printer using a fixed fixture to collect image information at the nozzle in real time during the printing process.

3. The artificial intelligence-based fused deposition modeling real-time monitoring device according to claim 1, characterized in that: The control panel is connected to the camera and printer via data cables, and the control panel is connected to the server via a network to realize the transmission of image information and control information between the printer and the server.

4. The artificial intelligence-based real-time monitoring device for fused deposition modeling according to claim 1, characterized in that: Printing parameters include nozzle Z-axis height, nozzle temperature, nozzle movement speed, and nozzle material extrusion rate.

5. The artificial intelligence-based fused deposition modeling real-time monitoring device according to claim 1, characterized in that: Artificial intelligence model units include convolutional neural networks, deep residual networks, or Inception image classification models.

6. A method for real-time control of fused deposition modeling process parameters according to the artificial intelligence-based fused deposition modeling real-time monitoring device according to any one of claims 1 to 5, characterized in that: The steps include: Step 1: Build a 3D printing platform: A camera is fixed to one side of the nozzle of the FDM 3D printer to capture image information at the nozzle during the printing process. The server includes a Python program unit and an artificial intelligence model unit that executes an artificial intelligence algorithm based on computer vision. The Python program unit automatically randomly generates a new set of printing parameters from a preset uniform distribution of printing parameters and generates G-code. A control board connects the camera, server, and FDM 3D printer to serve as a gateway for information exchange and transmission among the three. Step 2: Automated diverse data set construction: The control board collects image information from the nozzle through a camera and builds a real-time video channel between the control board and the server based on this information, intercepting frame data to generate images; The four current printing parameters are acquired in real time by the OctoPrint software unit and transmitted to the server by the controller. The printing parameters are classified into three categories according to the preset range: too high, normal, and too low, and used as the class label for the image at the corresponding moment. The Python program unit running on the server automatically randomly generates a new set of printing parameters from the preset uniform distribution of the four parameters, and generates the relevant G code, which is transmitted to the printer via the control board. The printing parameters are adjusted so that the camera can collect diverse image data printed under different parameters for subsequent artificial intelligence model training. The endpoint values ​​of the uniform distribution are preset according to the characteristics of the printer, the properties of the material, and the requirements of the printing process. Step 3: AI model training: The AI ​​model unit monitors the current printing status and judges the quality of four types of printing parameters. First, data preparation is performed using the diverse image dataset constructed in step 2. The image dataset is randomly divided into training and test sets in an 8:2 ratio. The image data is standardized using the Z-score standardization method to convert the data to a common scale to accelerate the convergence of the model training process. Then, the model structure is designed. A convolutional neural network is used to build the model to extract image features. The final model output is the judgment result of the quality of the four types of process parameters corresponding to the current image. The Pytorch framework is used to define the model layer structure. The cross entropy loss function is selected as the loss function for this classification task. The adaptive optimizer is selected for parameter optimization. Iterative training of no less than 5000 steps is carried out until the model converges. If the model does not converge, the learning rate, batch size or the number of training rounds are adjusted until the model converges. Step 4: Real-time image acquisition: The control board is connected to the camera on one hand and to the FDM 3D printer and server on the other. The camera collects image information at the nozzle. The control board and the OctoPrint software unit establish a real-time video channel between the server and the control board. The server runs the Python program unit, which is responsible for intercepting frame data from the real-time video channel to generate images, thereby realizing real-time monitoring of the printing process. Step 5: Real-time judgment of the status of printing process parameters: The trained artificial intelligence model is used to predict and judge the status of the four types of printing parameters. The artificial intelligence model uses the collected image data as input and outputs the quality of the status of the four types of printing parameters, including too high, normal, and too low. Step 6: Feedback control: The artificial intelligence model classifies the real-time image to obtain the current printing status and the quality classification results of the four types of process parameters. Based on the characteristics of the printer, the material properties and the requirements of the printing process, the step value for each adjustment of each process parameter is preset in advance. When generating the G code, it is ensured that the adjustment of each process parameter is limited by the preset step value to ensure that each adjustment is within a reasonable range. The generated G code is transmitted to the FDM 3D printer through the control board to perform the adjustment operation. The FDM 3D printer parses the G code and gradually adjusts the process parameters according to the step value.

7. The method for real-time control of fused deposition modeling process parameters according to claim 6, characterized in that: The control board includes a Raspberry Pi.

8. The method for real-time control of fused deposition modeling process parameters according to claim 6, characterized in that: The preset range of printing parameters is: normal range of nozzle Z-axis height , normal range of nozzle temperature , normal range of nozzle movement speed , nozzle material extrusion rate normal range .

9. The method for real-time control of fused deposition modeling process parameters according to claim 6, characterized in that: The four printing parameters are evenly distributed as follows: nozzle Z-axis height , nozzle temperature , nozzle moving speed , nozzle material extrusion rate ).

10. The method for real-time control of fused deposition modeling process parameters according to claim 6, characterized in that: The dataset is randomly divided into training and test sets in a ratio of 8:2.

Images