A method for predicting optimal process parameters for fused deposition modeling based on residual neural networks

By using residual neural networks to monitor and predict fused deposition modeling (FDM) parameters in real time, the problem of difficult quality control of FDM parts has been solved, achieving efficient online quality monitoring and parameter optimization, and improving the quality of molded parts and production efficiency.

CN118683059BActive Publication Date: 2026-11-17XI AN JIAOTONG UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410906791.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2026-11-17
Estimated Expiration
2044-07-08

AI Technical Summary

Technical Problem

In existing technologies, the quality control of fused deposition modeling parts is difficult, and defects occur frequently, which limits their application in key fields.

Method used

A residual neural network-based method is used to monitor and predict parameters such as nozzle flow rate, nozzle lateral movement speed, nozzle temperature, and Z-axis offset in real time during the fused deposition modeling process. Online control is achieved through image acquisition and deep learning models to improve quality control and efficiency.

Benefits of technology

It enables real-time quality monitoring and parameter optimization in the fused deposition modeling process, significantly improving the quality stability and production efficiency of the molded parts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118683059B_ABST
    Figure CN118683059B_ABST
Patent Text Reader

Abstract

A method for predicting optimal process parameters in fused deposition modeling based on residual neural networks is disclosed. In this method, an image acquisition device is connected to the nozzle of a 3D printer to acquire RGB images P1, P2…Pn of the fused deposition process at the nozzle, marking the current nozzle tip coordinates and printing parameters. The RGB images P1, P2…Pn are preprocessed by random rotation, scaling, and cropping to obtain preprocessed images N1, N2…Nn. The images N1, N2…Nn are then processed using a residual neural network. Images are acquired in real-time during the printing process of a randomly downloaded model, and these real-time images are preprocessed by random rotation, scaling, and cropping. The preprocessed images are input into a pre-trained residual neural network, which outputs prediction results for four types of printing parameters.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of real-time monitoring technology in additive manufacturing, and in particular to a method for predicting optimal process parameters for fused deposition modeling based on residual neural networks. Background Technology

[0002] Additive manufacturing, also known as 3D printing, is a novel manufacturing technology different from traditional subtractive and equal-material manufacturing. It rapidly shapes materials by layer-by-layer deposition. Fused deposition modeling (FDM) is a branch of additive manufacturing and is another widely used 3D printing technology following photopolymerization and multilayer solid prototyping. It has advantages such as low cost, simple operation, and rapid forming, and is widely used in various fields. Furthermore, FDM is widely used in the production of desktop 3D printing equipment due to its low noise, non-toxicity, and pollution-free characteristics.

[0003] Despite the aforementioned advantages, concerns about quality control of fused deposition modeling (FDM) parts limit its application. FDM parts often hinder critical applications in certain fields due to internal and surface defects. Therefore, developing an online monitoring technology for real-time prediction of process parameters is crucial for controlling the quality of fused deposition modeled parts. However, current research on combining optimal process parameter prediction with image processing is limited.

[0004] The information disclosed in the background section is only for enhancing the understanding of the background of this invention, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes a method for predicting optimal process parameters for fused deposition modeling based on residual neural networks. The online monitoring technology is used to control the quality of fused deposition modeled parts, ensuring the stability and consistency of product quality.

[0006] This invention is achieved through the following technical solution:

[0007] A method for predicting optimal process parameters for fused deposition modeling based on residual neural networks includes:

[0008] S1: The image acquisition device is connected to the nozzle of the 3D printer to acquire RGB images P1, P2...Pn of the molten deposit at the nozzle of the 3D printer and marks the current coordinates of the nozzle tip and printing parameters.

[0009] S2: The RGB images P1, P2...Pn are randomly rotated, scaled and cropped to obtain the preprocessed images N1, N2...Nn;

[0010] S3: The images N1, N2…Nn are processed based on a residual neural network, wherein…

[0011] S3.1: The residual neural network takes the images N1, N2…Nn as input and outputs classification predictions for four printing parameters: nozzle flow rate, nozzle lateral velocity, nozzle temperature, and Z-axis offset for the images N1, N2…Nn. These predictions are categorized into three classes: high, normal, and low. The four output heads for nozzle flow rate, nozzle lateral velocity, nozzle temperature, and Z-axis offset share feature extraction with the shared backbone. The input data passes through the backbone network, passing through multiple convolutional and pooling layers to extract features layer by layer. All feature extraction operations in this process are shared among all output heads. The features extracted by the backbone network generate a high-dimensional feature representation in the last fully connected layer. This feature representation contains all the information from the input data. The shared features are then passed in parallel to each output head. The weights of the shared backbone are updated during backpropagation of training using the sum of the losses of each individual output head.

[0012] S3.2: Input the complete material deposition image of the first layer in contact with the substrate during printing into the residual neural network to obtain the training result.

[0013] S3.3: Input all image acquisition data during the material printing process into the residual neural network to obtain the training results.

[0014] S3.4: Balance each of the 81 parameter classification combinations in the dataset to the same number, obtain a new dataset, input it into the residual neural network, and obtain the training results;

[0015] S3.5: The training process is divided into three stages: In the first stage, the network is trained on a subset of the dataset containing only printed images of the first layer with a 100% padding rate, making the features of each parameter more obvious and easier to learn; In the second stage, using transfer learning, the best-performing model from the first stage is retrained on a complete dataset containing all 3D geometric images, allowing the network to apply the features learned from the simple dataset to a more complex complete dataset. This process is repeated three times; In the third stage, transfer learning is used again, but only the fully connected layers of each output head are retrained using a balanced subset of the dataset, which contains an equal number of samples for each of the 81 possible parameter combinations. The weights of the feature extraction network backbone are frozen to preserve the learned general features. When the validation loss does not decrease significantly over several epochs, the model is considered to have converged.

[0016] S4: Acquire images in real time during the printing process to be optimized, and preprocess the acquired images by randomly rotating, scaling and cropping them.

[0017] S5: The preprocessed image is input into the trained residual neural network, which outputs the prediction results for four printing parameter classifications.

[0018] In the aforementioned method for predicting optimal process parameters for fused deposition modeling based on residual neural networks, step S2, which involves preprocessing the RGB images P1, P2…Pn by random rotation, scaling, and random cropping, includes:

[0019] S2.1: The acquired RGB images P1, P2...Pn are randomly rotated by 10° in any direction, and then a small perspective transformation with a probability of 0.1 is applied. The nozzle tip coordinates saved during data acquisition are then used to automatically crop the image to a 320×320 pixel square area focused on the nozzle tip.

[0020] S2.2: Crop random square portions with an area ratio between 0.9 and 1.0 from the 320×320 image and resize them to 224×224 pixels as the input size for the neural network. Then apply a horizontal flip with a probability of 0.5 to the image and apply ±10% color jitter to the image's brightness, contrast, hue, and saturation.

[0021] S2.3: Normalize the channels in the transformed image using the pixel mean and standard deviation of each channel of all images in the dataset.

[0022] In the method for predicting optimal process parameters for fused deposition modeling based on residual neural networks, the printing parameters include nozzle moving speed, z-axis height, nozzle flow rate, and nozzle temperature.

[0023] In the aforementioned method for predicting optimal process parameters for fused deposition modeling based on residual neural networks, the shared network backbone comprises 3 attention modules and 6 residual modules.

[0024] In the aforementioned method for predicting optimal process parameters for fused deposition modeling based on residual neural networks, the attention module consists of two branches: a mask and a backbone. The backbone branch is constructed from residual blocks to perform feature processing of the network. The mask branch performs downsampling and then upsampling to learn the attention mask used for the output features of the weighting module.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] This invention uses real-time images of material deposition at the nozzle during the printing process to upload them online to a trained deep network model. The model outputs real-time classification predictions of four printing parameters, providing data support for online adjustment of printing process parameters, improving the adaptability and flexibility of the monitoring system, and significantly enhancing the quality control and efficiency of the FDM printing process. Attached Figure Description

[0027] Various other advantages and benefits of the present invention will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0028] In the attached diagram:

[0029] Figure 1 This is a schematic flowchart of a method for predicting optimal process parameters for fused deposition modeling based on residual neural networks provided by the present invention.

[0030] Figure 2 This is a schematic diagram of a neural network provided in one embodiment of the present invention;

[0031] Figure 3 Schematic diagram of residual block structure;

[0032] Figure 4 Training process of multi-head convolutional neural networks.

[0033] The present invention will be further explained below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0034] Specific embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While specific embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0035] It should be noted that certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that different terms may be used to refer to the same component. This specification and claims do not distinguish components based on differences in terminology, but rather on differences in function. The terms "comprising" or "including" used throughout the specification and claims are open-ended and should be interpreted as "comprising but not limited to." The following descriptions are preferred embodiments for carrying out the invention; however, these descriptions are for the purpose of understanding the general principles of the specification and are not intended to limit the scope of the invention. The scope of protection of this invention is determined by the appended claims.

[0036] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. The accompanying drawings do not constitute a limitation on the embodiments of the present invention.

[0037] like Figures 1 to 4 As shown, the optimal process parameter prediction method for fused deposition modeling based on residual neural networks includes the following steps:

[0038] S1: The image acquisition device is connected to the nozzle of the 3D printer to acquire RGB images P1, P2...Pn of the molten deposit at the nozzle of the 3D printer and marks the current coordinates of the nozzle tip and printing parameters.

[0039] S2: The RGB images P1, P2...Pn are randomly rotated, scaled and cropped to obtain the preprocessed images N1, N2...Nn;

[0040] S3: The images N1, N2…Nn are processed based on a residual neural network, wherein…

[0041] S3.1: The residual neural network takes the images N1, N2…Nn as input and outputs classification predictions for four printing parameters (nozzle flow rate, nozzle lateral velocity, nozzle temperature, and Z-axis offset) for the images N1, N2…Nn. These predictions are categorized into three classes: high, normal, and low. The four output heads (nozzle flow rate, nozzle lateral velocity, nozzle temperature, and Z-axis offset) share feature extraction with the backbone. The input data first passes through the backbone network, then through multiple convolutional and pooling layers, extracting and abstracting high-level features layer by layer. All feature extraction operations in this process are shared among all output heads. The features extracted by the backbone network generate a high-dimensional feature representation in the last layer (fully connected layer). This representation contains all the important information of the input data, and these shared features are then passed in parallel to each output head. The weights of the shared backbone are updated during the backpropagation of training by summing the losses of each individual output head.

[0042] S3.2: Input the complete material deposition image of the first layer in contact with the substrate during printing into the residual neural network to obtain the training result.

[0043] S3.3: Input all image acquisition data during the material printing process into the residual neural network to obtain the training results.

[0044] S3.4: Balance each of the 81 parameter classification combinations in the dataset to the same number, obtain a new dataset, input it into the residual neural network, and obtain the training results;

[0045] S3.5: The training process is divided into three main stages: In the first stage, the network is trained on a subset of data containing only printed images of the first layer with 100% padding, making the features of each parameter more obvious and easier to learn. In the second stage, using transfer learning, the best-performing model from the first stage is retrained on a complete dataset containing all 3D geometric images. This allows the network to apply features learned from the simple dataset to a more complex complete dataset; this process is repeated three times. In the third stage, transfer learning is used again, but only the fully connected layers of each output head are retrained using a balanced subset of data containing equal samples of each of the 81 possible parameter combinations (four parameters, each of which can be low, medium, or high). The weights of the feature extraction network backbone are frozen to preserve the learned general features.

[0046] S4: Acquire images in real time during the printing process to be optimized, and preprocess the acquired images by randomly rotating, scaling and cropping them.

[0047] S5: The preprocessed image is input into the trained residual neural network, which outputs the prediction results for four printing parameter classifications.

[0048] In a preferred embodiment of the method for predicting optimal process parameters for fused deposition modeling based on residual neural networks, step S2, which involves preprocessing the RGB images P1, P2…Pn by random rotation, scaling, and random cropping, includes:

[0049] S2.1: The acquired RGB images P1, P2...Pn are randomly rotated by 10° in any direction, and then a small perspective transformation with a probability of 0.1 is applied. The nozzle tip coordinates saved during data acquisition are then used to automatically crop the image to a 320×320 pixel square area focused on the nozzle tip.

[0050] S2.2: Crop random square portions with an area ratio between 0.9 and 1.0 from the 320×320 image and resize them to 224×224 pixels as the input size for the neural network. Then apply a horizontal flip with a probability of 0.5 to the image and apply ±10% color jitter to the image's brightness, contrast, hue, and saturation.

[0051] S2.3: Normalize the channels in the transformed image using the pixel mean and standard deviation of each channel of all images in the dataset.

[0052] In a preferred embodiment of the method for predicting optimal process parameters for fused deposition modeling based on residual neural networks, the printing parameters include nozzle flow rate, nozzle lateral movement speed, nozzle temperature, and Z-axis offset.

[0053] In a preferred embodiment of the method for predicting optimal process parameters for fused deposition modeling based on residual neural networks, the shared network backbone includes 3 attention modules and 6 residual modules.

[0054] In a preferred embodiment of the method for predicting optimal process parameters for fused deposition modeling based on residual neural networks, the attention module consists of two branches: a mask and a backbone. The backbone is constructed from residual blocks to perform feature processing of the network. The mask branch performs downsampling and then upsampling to learn an attention mask for the output features of the weighting module.

[0055] In one embodiment, the printer nozzle continuously prints, and a camera fixed to the printer nozzle takes a picture of the material deposition at the nozzle every 0.5 seconds. Sensors arranged in the printer simultaneously collect the coordinates of the printer nozzle and four printing parameters (nozzle movement speed, z-axis height, nozzle flow rate, and nozzle temperature) at the time of the picture, and store these parameters as image tags.

[0056] The residual neural network for detecting the forming process consists of a shared backbone network and four output heads, which correspond to four process parameters: material extrusion rate, nozzle movement speed, nozzle temperature, and Z-axis offset distance. A schematic diagram of the network structure is shown below. Figure 2 As shown.

[0057] The shared backbone network for extracting image features consists of six residual blocks and three attention models. Each residual block includes a convolutional layer, batch normalization, activation function activation, and pooling layer. A schematic diagram of the residual block structure is shown below. Figure 3 As shown, the ReLU function is used to activate the output of the batch-normalized convolutional layer, and the activation value is sent to the next layer.

[0058] After extracting feature information from the forming process image using a shared backbone network composed of six cascaded residual blocks, its output is flattened into a column vector. Then, a fully connected layer is used to map the image features to four process parameter states. This means that the four output heads share the feature information extracted by the shared backbone. During training, the weights of the shared backbone network are updated using the total loss of the four output heads. This means that the shared backbone network can learn the interaction relationships between process parameters, as well as the correlation between each process parameter and the forming state image, enabling it to propose multiple process parameter adjustment methods for a given defective forming state.

[0059] To obtain an effective and stable multi-head convolutional network for accurately predicting process parameter states, the training process of the multi-head convolutional network is divided into three parts, such as... Figure 4As shown, the network is first trained using images with obvious process parameter state features from the training set. Due to these obvious state features, the multi-head convolutional neural network (MCN) can effectively and quickly capture image information and detect important features. Next, the pre-trained MCN model is fine-tuned on the complete training set to improve its ability to recognize states in images with inconspicuous features. Finally, considering the imbalance of the training set samples—that is, due to the influence of forming characteristics, excessively low Z-axis offsets will lead to nozzle-hotbed collisions and forming failures—the number of samples with excessively high Z-axis offsets far exceeds those with excessively low offsets. To mitigate the interference of sample imbalance on the prediction performance of the MCN, a transfer learning method is used in the final stage of the training process. The network weights sharing the backbone are frozen, and the weights of the fully connected layers are fine-tuned using balanced samples to obtain good process parameter state prediction results. Finally, the accuracy of the trained model is tested on the test set. The classification accuracy for each parameter on the test set is as follows: flow rate 87.1%, lateral velocity 86.4%, Z-offset 85.5%, and hot end temperature 78.3%.

[0060] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1. A method for predicting optimal process parameters for fused deposition modeling based on residual neural networks, characterized in that, Includes the following steps: S1: The image acquisition device is connected to the nozzle of the 3D printer to acquire RGB images P1, P2...Pn of the molten deposit at the nozzle of the 3D printer and marks the current coordinates of the nozzle tip and printing parameters. S2: The RGB images P1, P2...Pn are randomly rotated, scaled and cropped to obtain the preprocessed images N1, N2...Nn; S3: The images N1, N2…Nn are processed based on a residual neural network, wherein… S3.1: The residual neural network takes the images N1, N2…Nn as input and outputs classification predictions for four printing parameters: nozzle flow rate, nozzle lateral velocity, nozzle temperature, and Z-axis offset for the images N1, N2…Nn. These predictions are categorized into three classes: high, normal, and low. The four output heads for nozzle flow rate, nozzle lateral velocity, nozzle temperature, and Z-axis offset share feature extraction with the shared backbone. The input data passes through the backbone network, passing through multiple convolutional and pooling layers to extract features layer by layer. All feature extraction operations in this process are shared among all output heads. The features extracted by the backbone network generate a high-dimensional feature representation in the last fully connected layer. This feature representation contains all the information from the input data. The shared features are then passed in parallel to each output head. The weights of the shared backbone are updated during backpropagation of training using the sum of the losses of each individual output head. S3.2: Input the complete material deposition image of the first layer in contact with the substrate during printing into the residual neural network to obtain the training result. S3.3: Input all image acquisition data during the material printing process into the residual neural network to obtain the training results. S3.4: Balance each of the 81 parameter classification combinations in the dataset to the same number, obtain a new dataset, input it into the residual neural network, and obtain the training results; S3.5: The training process is divided into three stages: In the first stage, the network is trained on a subset of the dataset containing only printed images of the first layer with a 100% fill rate, making the features of each parameter more obvious and easier to learn; In the second stage, using transfer learning, the best-performing model from the first stage is retrained on a complete dataset containing all 3D geometric images, allowing the network to apply the features learned from the simple dataset to a more complex complete dataset. This process is repeated three times; In the third stage, transfer learning is used again, but only the fully connected layers of each output head are retrained using a balanced subset of the dataset, which contains an equal number of samples for each of the 81 possible parameter combinations. The weights of the feature extraction network backbone are frozen to preserve the learned general features. When the validation loss does not decrease significantly over several epochs, the model is considered to have converged. S4: Acquire images in real time during the printing process to be optimized, and preprocess the acquired images by randomly rotating, scaling and cropping them. S5: The preprocessed image is input into the trained residual neural network, which outputs the prediction results for four printing parameter classifications.

2. The method for predicting optimal process parameters for fused deposition modeling based on residual neural networks according to claim 1, characterized in that, Step S2, the preprocessing of the RGB images P1, P2…Pn by random rotation, scaling, and random cropping includes: S2.1: The acquired RGB images P1, P2...Pn are randomly rotated by 10° in any direction, and then a small perspective transformation with a probability of 0.1 is applied. The nozzle tip coordinates saved during data acquisition are then used to automatically crop the image to a 320×320 pixel square area focused on the nozzle tip. S2.2: Crop random square portions with an area ratio between 0.9 and 1.0 from the 320×320 image and resize them to 224×224 pixels as the input size for the neural network. Then apply a horizontal flip with a probability of 0.5 to the image and apply ±10% color jitter to the image's brightness, contrast, hue, and saturation. S2.3: Normalize the channels in the transformed image using the pixel mean and standard deviation of each channel of all images in the dataset.

3. The method of claim 1, wherein, The printing parameters include printhead movement speed, z-axis height, printhead flow rate, and printhead temperature.

4. The method of claim 1, wherein, The shared network backbone consists of 3 attention modules and 6 residual modules.

5. The method of claim 4, wherein, The attention module consists of two branches: a mask and a backbone. The backbone is constructed from residual blocks to perform feature processing for the network. The mask branch performs downsampling and then upsampling to learn an attention mask for the output features of the weighting module.

Citation Information

Patent Citations

  • Numerical control device, additive manufacturing device, and control method for additive manufacturing device

    CN114599475A

  • Laser additive manufacturing defect monitoring method based on spatio-temporal information fusion

    CN117593255A