Binder spraying additive manufacturing monitoring method and device based on machine vision and electronic equipment

Abstract

The invention provides a binder spraying additive manufacturing monitoring method and device based on machine vision and electronic equipment, in the manufacturing process, the level image of each level of a printed piece is collected in real time, the level images comprise images of laid powder and sprayed binders, and feature vectors of the level images are extracted. And performing defect identification by using an identification model obtained by pre-training and based on the feature vector to obtain a defect identification result. And under the condition that the defect recognition result does not meet the preset requirement, a preset optimization algorithm is adopted for optimization on the basis of the current printing parameters, and the optimal printing parameters are obtained. And adjusting information is generated based on the optimal printing parameters, and additive manufacturing of the next level is executed based on the adjusting information. According to the scheme, defect analysis processing is conducted on each layer of image in real time based on the powder information in the image and the binder spraying condition, problems can be found in time, printing parameters can be adjusted in time, and the quality and the yield of printed pieces are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of additive manufacturing, and in particular to a binder jetting additive manufacturing monitoring method, device and electronic equipment based on machine vision. Background Art

[0002] Binder Jetting Additive Manufacturing (BJAM), an additive manufacturing technology, is widely used for the rapid prototyping of metal and ceramic parts by applying powder layer by layer and selectively spraying a binder to solidify the powder material. However, this process faces several quality control challenges, such as powder placement uniformity, binder jetting accuracy, and interlayer bond strength.

[0003] Current binder jetting additive manufacturing quality monitoring relies primarily on offline detection methods (such as CT scanning and metallographic analysis), but these methods have significant technical limitations. Specifically, existing offline detection technologies suffer from significant feedback delays, and quality assessment can only be performed after printing is complete, resulting in defects being discovered beyond repair. This lag results in a high scrap rate for each batch of products and makes it impossible to dynamically compensate for printing parameters. Traditional image processing algorithms have excessively long response times, making it difficult to meet the real-time requirements of high-speed printing.

[0004] Existing technologies struggle to effectively identify defects in the powder layer, such as uneven powder accumulation, cracks, and delamination. Manual inspection is time-consuming, labor-intensive, and inconsistent. Manual spot checks are not only inefficient and ineffective, but traditional image processing methods also miss up to 42% of microcracks.

[0005] Traditional monitoring methods have significant limitations in capturing the dynamic behavior of adhesive droplets, especially during high-speed spraying. Problems such as droplet splashing and uneven diffusion are difficult to perceive and quantify in real time, leading to frequent quality problems such as uneven bonding and porosity defects.

[0006] In the field of metal additive manufacturing, traditional monitoring systems are often limited to data collection and lack the ability to conduct in-depth analysis of detection results and provide real-time feedback. This leads to serious lags in process optimization and makes it difficult to adapt to complex and changing material properties and printing environments. Summary of the Invention

[0007] The purpose of the embodiments of the present invention is to provide a binder jetting additive manufacturing monitoring method, device and electronic equipment based on machine vision, so as to timely detect problems and adjust printing parameters, thereby effectively improving the quality and yield of printed parts.

[0008] In a first aspect, the present invention provides a binder jetting additive manufacturing monitoring method based on machine vision, the method comprising:

[0009] Performing binder jetting additive manufacturing based on the constructed three-dimensional model, and acquiring layer images of each layer of the printed part in real time during the manufacturing process, wherein the layer images include images of the laid powder and the sprayed binder;

[0010] For the currently acquired hierarchical image, extracting a feature vector of the hierarchical image;

[0011] Using the pre-trained recognition model and based on the feature vector to perform defect recognition to obtain a defect recognition result;

[0012] When the defect recognition result does not meet the preset requirements, a preset optimization algorithm is used to optimize the current printing parameters to obtain the optimal printing parameters;

[0013] Adjustment information is generated based on the optimal printing parameters, and additive manufacturing of the next level is performed based on the adjustment information.

[0014] In an optional embodiment, the step of extracting the feature vector of the hierarchical image includes:

[0015] A threshold segmentation algorithm is used to segment the powder area and the non-powder area in the hierarchical image to obtain segmentation information;

[0016] detecting the spraying trajectory and diffusion boundary of the adhesive in the hierarchical image based on an edge detection algorithm to obtain edge information;

[0017] Using a convolutional neural network to extract features from the hierarchical image to obtain feature information;

[0018] The segmentation information, edge information and feature information are combined to obtain a feature vector.

[0019] In an optional embodiment, the method further includes a step of pre-training to obtain the recognition model, including:

[0020] Collecting a plurality of sample images, each of the sample images having defect annotation information, wherein the defect annotation information includes the position, shape, size and category of the defect;

[0021] The constructed improved Unet++ network is trained using the multiple sample images until the training is stopped when a preset condition is met, thereby obtaining a trained recognition model.

[0022] In an optional embodiment, the step of optimizing the current printing parameters using a preset optimization algorithm to obtain the optimal printing parameters includes:

[0023] Randomly generating a plurality of printing parameters including a current printing parameter, and optimizing the plurality of printing parameters based on a preset optimization algorithm;

[0024] During the optimization process, corresponding print quality scores are obtained based on the print parameters, and the print quality scores are used as fitness values ​​to guide the optimization until the print parameters with the best fitness values ​​are found as the optimal print parameters.

[0025] In an optional embodiment, the step of obtaining a corresponding print quality score based on the printing parameters includes:

[0026] The printing parameters are imported into a pre-trained evaluation model for evaluation to obtain a printing quality score.

[0027] In an optional embodiment, the evaluation model is obtained by:

[0028] Collecting historical printing parameters and historical images obtained under each of the historical printing parameters, each of the historical images having a print quality label;

[0029] Importing each of the historical printing parameters and the corresponding historical images into the constructed regression model, and outputting the corresponding print quality prediction results;

[0030] A loss function is constructed based on the print quality label and the print quality prediction result, and the iterative training of the regression model is performed under the guidance of the loss function until a preset stopping condition is met, thereby obtaining a trained evaluation model.

[0031] In an optional embodiment, the method further comprises:

[0032] Obtaining an acoustic emission signal detected by an acoustic emission sensor, performing frequency, amplitude, and duration analysis on the acoustic emission signal, and obtaining an acoustic emission signal analysis result;

[0033] Obtaining pressure information of the printing device detected by the pressure sensor and deformation information of the printing device detected by the strain sensor;

[0034] The step of generating adjustment information based on the optimal printing parameters includes:

[0035] Adjustment information is generated by combining the optimal printing parameters, the acoustic emission signal analysis results, the pressure information, and the deformation information.

[0036] In an optional embodiment, the method further comprises:

[0037] Obtaining distance information detected by a laser ranging sensor, and obtaining geometric size information of the printed work based on the distance information;

[0038] The step of generating adjustment information based on the optimal printing parameters includes:

[0039] Adjustment information is generated by combining the optimal printing parameters and the geometric dimension information.

[0040] In a second aspect, the present invention provides a binder jetting additive manufacturing monitoring device based on machine vision, the device comprising:

[0041] an acquisition module, configured to perform binder jetting additive manufacturing based on the constructed three-dimensional model, and to acquire a layer image of each layer of the printed part in real time during the manufacturing process, the layer image including images of the laid powder and the sprayed binder;

[0042] An extraction module, configured to extract a feature vector of a currently acquired hierarchical image;

[0043] An identification module, configured to use a pre-trained identification model and perform defect identification based on the feature vector to obtain a defect identification result;

[0044] An optimization module is used to optimize the current printing parameters using a preset optimization algorithm to obtain the optimal printing parameters if the defect recognition result does not meet the preset requirements;

[0045] An adjustment module is configured to generate adjustment information based on the optimal printing parameters and perform additive manufacturing at a next level based on the adjustment information.

[0046] In a third aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement any of the methods described in the foregoing embodiments.

[0047] The present invention provides a method, device and electronic equipment for monitoring binder jetting additive manufacturing based on machine vision, which collects hierarchical images of each layer of the printed part in real time during the manufacturing process, wherein the hierarchical images include images of laid powder and sprayed binder, and extracts feature vectors of the hierarchical images. Defect recognition is performed based on the feature vector using a pre-trained recognition model to obtain a defect recognition result. In the case where the defect recognition result does not meet the preset requirements, a preset optimization algorithm is used to perform optimization based on the current printing parameters to obtain the optimal printing parameters. Adjustment information is generated based on the optimal printing parameters, and the next level of additive manufacturing is performed based on the adjustment information. In this solution, defect analysis and processing are performed on each level image in real time based on the powder information and binder spraying conditions in the image, so that problems can be discovered in a timely manner and printing parameters can be adjusted, effectively improving the quality of printed parts and the yield rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A schematic diagram of the combined architecture of a machine vision-based binder jetting additive manufacturing monitoring system provided in an embodiment of the present invention;

[0050] Figure 2 A flowchart of a binder jetting additive manufacturing monitoring method based on machine vision provided by an embodiment of the present invention;

[0051] Figure 3 for Figure 2 Flowchart of the sub-steps included in step S12;

[0052] Figure 4 A flowchart of a recognition model training method in a monitoring method provided in an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of the model structure of the recognition model provided by an embodiment of the present invention;

[0054] Figure 6 for Figure 2 Flowchart of the sub-steps included in step S14;

[0055] Figure 7 A flowchart of an evaluation model training method in a monitoring method provided in an embodiment of the present invention;

[0056] Figure 8 A functional module block diagram of a machine vision-based binder jetting additive manufacturing monitoring system according to an embodiment of the present invention;

[0057] Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0058] Icons: 1-forming cylinder; 2-printing nozzle; 3-roller system; 4-powder hopper; 5-industrial camera; 6-lighting device; 7-control system; 8-monitoring system. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

[0060] See also Figure 1The binder jetting additive manufacturing monitoring system provided in an embodiment of the present invention includes a build cylinder 1, a print head 2, a roller system 3, a powder hopper 4, an industrial camera 5, a lighting device 6, a control system 7, and a monitoring system 8. The industrial camera 5 and lighting device 6 are connected to the monitoring system 8 for transmitting image data. The monitoring system 8 analyzes the collected image data to generate feedback signals, and the control system 7 adjusts the printing system based on the feedback signals.

[0061] Please refer to Figure 2 , which is a flow chart of a machine vision-based binder jetting additive manufacturing monitoring method provided in an embodiment of the present invention. The monitoring method can be applied to electronic devices, which can be devices equipped with the above-mentioned control system 7 and monitoring system 8, such as computer devices, servers, etc. The monitoring method may specifically include the following steps:

[0062] S11, performing binder jetting additive manufacturing based on the constructed three-dimensional model, and acquiring a layer image of each layer of the printed part in real time during the manufacturing process, the layer image including images of the laid powder and the sprayed binder.

[0063] S12, extracting a feature vector of the hierarchical image for the currently acquired hierarchical image.

[0064] S13, using the pre-trained recognition model and performing defect recognition based on the feature vector to obtain a defect recognition result.

[0065] S14, when the defect recognition result does not meet the preset requirements, based on the current printing parameters, a preset optimization algorithm is used to perform optimization to obtain the optimal printing parameters.

[0066] S15, generating adjustment information based on the optimal printing parameters, and executing the next level of additive manufacturing based on the adjustment information.

[0067] During the binder jetting additive manufacturing process, the metal powder can first be dried and sieved. Specifically, the metal powder can be placed in a drying oven for drying, and then placed in an automatic sieving machine to screen out qualified metal powder. The drying temperature for the metal powder can be 150°C, the drying time can be at least 360 minutes, and the sieve size can be 300 mesh.

[0068] After the 3D model of the part to be printed is constructed based on the requirements, the 3D model is imported into the electronic device and printing is started based on the control system 7. Powder is laid on the substrate, and the print head 2 sprays the binder, and the layers are stacked to complete the green body.

[0069] During the binder jetting additive manufacturing process, industrial camera 5 captures the powder layering and binder spraying at a high frame rate, acquiring layer-by-layer images of each printed part. Each layer-by-layer image contains image information about the powder and binder during the printing process, demonstrating the distribution of the powder and the state of the binder spraying, providing essential data for subsequent monitoring and analysis.

[0070] At least one industrial camera5 can be deployed within the print chamber to capture the powder layer and binder spraying area in real time from various angles, such as top and side, to obtain high-resolution image data. This multi-angle imaging method captures comprehensive details of the printing process, providing a rich data foundation for subsequent image analysis and a prerequisite for the implementation of the entire monitoring system.

[0071] In this embodiment, lighting device 6 provides an appropriate light source for industrial camera 5 according to a preset lighting pattern, ensuring the quality of the captured image. By properly selecting and adjusting the light source, optical interference can be effectively reduced, improving image clarity and contrast, making features in the image more distinct and facilitating subsequent processing and analysis.

[0072] In this embodiment, each captured layer image can be analyzed and processed in real time to obtain a feature vector for the layer image. This feature vector contains the distribution characteristics of the powder and the adhesive spraying conditions within the layer image. A pre-trained recognition model is used to process the feature vector to obtain defect recognition results for the layer image. These defect recognition results may include defects such as powder layer deficiency, cracks in the printed pattern, clogged printheads, and adhesive droplet splashing.

[0073] Because the image quality of each layer is determined by printing parameters, such as powder spreading speed, roller rotation speed, drying power, drying time, and binder jet head reference voltage, the current defect recognition results can be used to determine whether they meet the preset requirements. If not, the printing parameters need to be adjusted to ensure subsequent printing quality.

[0074] In this embodiment, if the defect identification results obtained under the current printing parameters do not meet the preset requirements, a preset optimization algorithm can be used to optimize the current printing parameters to obtain the optimal printing parameters. Adjustment information can then be generated based on the optimal printing parameters. The electronic device can then adjust various devices in the system based on this adjustment information, and then use the adjusted system to perform the next level of additive manufacturing.

[0075] The following describes the specific implementation methods of each of the above steps.

[0076] In this embodiment, the collected images of each layer include images of the laid powder and sprayed binder. TensorFlow can be used to read the image data, convert the image data into tensors, and then perform grayscale processing to convert the color layer images into grayscale images, thereby reducing the data dimension and the complexity of subsequent processing.

[0077] In addition, a Gaussian filter algorithm can be used to smooth the image, effectively removing Gaussian noise and preventing it from interfering with subsequent feature extraction. For example, a 3x3 or 5x5 Gaussian kernel is used. Based on the noise level of the image, the appropriate kernel size is selected and a weighted average is performed on each pixel in the image to smooth the image.

[0078] See also Figure 3 On this basis, the feature vector of the hierarchical image is extracted in real time. This step can be achieved by the following methods:

[0079] S121 , using a threshold segmentation algorithm to segment the powder area and the non-powder area in the hierarchical image to obtain segmentation information.

[0080] S122 , detecting the spraying trajectory and diffusion boundary of the adhesive in the hierarchical image based on an edge detection algorithm to obtain edge information.

[0081] S123, using a convolutional neural network to extract features from the hierarchical image to obtain feature information.

[0082] S124: Combine the segmentation information, edge information, and feature information to obtain a feature vector.

[0083] In this embodiment, the analysis and processing of the hierarchical image is mainly carried out from three aspects, including threshold segmentation processing for powder distribution, edge detection processing for adhesive laying, and convolutional neural network processing for the overall characteristics of the image.

[0084] Specifically, a threshold segmentation algorithm can be used to segment the hierarchical image into foreground and background images by setting an appropriate threshold based on the grayscale characteristics of the hierarchical image. The foreground image represents the powdery area, while the background image represents the non-powdered area. This threshold segmentation process clearly separates the powdery and non-powdered areas, facilitating subsequent analysis of the powder distribution.

[0085] In this embodiment, the Otsu algorithm may be used to automatically calculate the optimal threshold. The algorithm may determine the threshold by maximizing the inter-class variance based on the grayscale histogram of the image, and may adaptively adapt to the characteristics of different images.

[0086] Additionally, edge detection algorithms, such as the Canny edge detection algorithm, can be used to extract edge information from hierarchical images. For adhesive spraying area images, edge detection can accurately delineate the adhesive spray trajectory and diffusion boundary. The Canny edge detection algorithm uses Gaussian filtering for noise reduction, gradient magnitude and direction calculation, non-maximum suppression, dual-threshold detection, and edge connection to effectively detect edges in images and demonstrate a certain degree of robustness to noise.

[0087] In addition, a convolutional neural network (CNN) can be used to extract features from hierarchical images. In this embodiment, a CNN model can be constructed that includes multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer uses convolution kernels of different sizes to slide on the hierarchical image to extract local features of the hierarchical image, such as the texture of powder particles and the spraying morphology of the binder. The pooling layer can downsample the feature map output by the convolutional layer to reduce the amount of data while retaining the main features. The fully connected layer integrates the features output by the pooling layer and outputs the final feature information.

[0088] In the early stage of training the CNN model, a large amount of labeled printing process image data can be used for training, and the model parameters can be continuously adjusted through the back propagation algorithm so that the model can accurately extract features related to printing quality.

[0089] On the basis of obtaining the segmentation information, edge information and feature information, the feature splicing method can be used to connect the segmentation information, edge information and feature information in sequence to form a comprehensive feature vector.

[0090] On this basis, the principal component analysis (PCA) method can be used to reduce the dimension of the spliced ​​feature vectors to remove redundant information while retaining the main feature-related information.

[0091] PCA calculates the covariance matrix of the eigenvectors to determine the principal component directions of the data and projects the data onto these principal components to achieve dimensionality reduction. This implementation can also incorporate a channel attention mechanism. In addition to obtaining global channel information through global average pooling and generating channel weights through fully connected layers, residual connections can also be introduced to enhance information transfer. By weighting each element in the eigenvector—for example, increasing the weight of segmentation information or edge information—it is possible to more accurately extract the integrity, uniformity, and defect characteristics of the powder layer, as well as key information such as the binder spray trajectory and diffusion range. For example, the uniformity of the powder layer can be measured by calculating the standard deviation of the eigenvalues ​​related to the powder distribution in the eigenvector. For analysis of the binder spray trajectory, parameters such as the trajectory length and curvature can be calculated based on the edge information obtained through edge detection.

[0092] Based on the feature vector of the hierarchical image, the defect recognition result is obtained by using the pre-trained recognition model and performing defect recognition based on the feature vector. Figure 4 , the recognition model can be trained in advance by the following methods:

[0093] S21 , collecting a plurality of sample images, each sample image having defect annotation information, the defect annotation information including the position, shape, size and category of the defect.

[0094] S22, using multiple sample images to train the constructed improved Unet++ network until the preset conditions are met and the training is stopped to obtain a trained recognition model.

[0095] To accurately detect printing defects during binder jetting additive manufacturing, this example introduces an improved Unet++ network. This improved Unet++ network incorporates an advanced attention mechanism into the encoder portion of the traditional Unet++ architecture to significantly improve its ability to capture complex defect features.

[0096] Please refer to Figure 5 ,The overall architecture of the Unet++ network includes multiple ,structures such as feature extraction, feature classification, decoder, jump connection, and segmentation extraction, which ,collaboratively complete defect detection and identification.

[0097] Specifically, a jet monitoring sensor, such as an industrial camera 5, is used to capture images of the internal structure during the printing process as sample images. These sample images are converted into C-scan images suitable for network processing. A dataset is constructed based on these C-scan images, which can be larger than 10,000 images. Furthermore, defect information (location, shape, size, and category) in the sample images is accurately annotated as defect annotation information.

[0098] The dataset can be divided into a training set and a test set. During the training phase, the Unet++ network incorporating an attention mechanism is trained for deep learning using sample images from the training set. The validation set is then used to verify the trained network's detection performance. Preliminary experimental results demonstrate that the improved Unet++ network can quickly and accurately identify and segment defects such as powder shortages, cracks, and nozzle clogs in complex printing environments. Recognition accuracy remains stable at over 95%, and signal processing time is effectively controlled within 200ms. This significantly improves the efficiency and accuracy of online defect monitoring for binder jetting additive manufacturing, providing a strong guarantee for real-time control of print quality.

[0099] The trained recognition model can be applied to printing defect recognition and detection in the actual monitoring stage. Specifically, the recognition model is used to process the feature vector of the real-time hierarchical image obtained above to obtain the defect recognition result of the hierarchical image.

[0100] If the defect recognition result does not meet the preset requirements, for example, there are many defects or the defects are serious, it means that the current printing parameters are not suitable, which leads to many defects in the layered image under the printing parameters.

[0101] Therefore, it is necessary to adjust the current printing parameters. In this embodiment, based on the current printing parameters, a preset optimization algorithm is used to optimize and obtain the optimal printing parameters. For details, please refer to Figure 6 , this step can be achieved by:

[0102] S141 , randomly generating multiple printing parameters including the current printing parameters, and performing optimization among the multiple printing parameters based on a preset optimization algorithm.

[0103] S142 , during the optimization process, a corresponding print quality score is obtained based on the print parameters, and the optimization is guided by the print quality score as the fitness value, until the print parameters with the best fitness value are found as the optimal print parameters.

[0104] In this embodiment, the printing parameters include multiple parameters, such as powder spreading speed, roller rotation speed, drying power, drying time, binder injection head reference voltage, etc. The combined effect of these multiple parameters will affect the final printing quality.

[0105] In this embodiment, multiple printing parameters including the current printing parameters can be generated within a certain range, and a preset optimization algorithm, such as a particle swarm optimization algorithm (PSO) or a genetic algorithm, can be used to find the optimal printing parameters.

[0106] Taking the PSO algorithm as an example, it simulates the foraging behavior of a flock of birds, treating each set of printing parameters as a particle in the search space, with two properties: position and velocity. In each iteration, a particle updates its velocity and position based on its own historical optimal position and the global optimal position of the swarm. For the binder jet additive manufacturing process, the print quality score is used as the fitness function, and through continuous iteration, particles are directed toward the direction of optimal print quality. In the PSO algorithm, appropriate learning factors and inertia weights are set. The learning factor controls the step size of the particle's flight toward its historical optimal position and the swarm's global optimal position, while the inertia weight controls the degree to which the particle maintains its current velocity. By adjusting these parameters, the algorithm's global and local search capabilities can be balanced, improving its convergence speed and search accuracy.

[0107] Taking into account actual engineering requirements, a penalty function is added to the optimization algorithm. For printing parameter combinations that don't meet actual production conditions, such as excessively high drying power, which can lead to energy waste and equipment damage, or excessively low binder head reference voltage, which can result in weak bonding, the penalty function reduces the fitness value of these combinations, making the optimization algorithm more inclined to search for parameter combinations that meet actual engineering requirements.

[0108] In this embodiment, the print quality scores under each set of printing parameters can be obtained by importing the printing parameters into a pre-trained evaluation model for evaluation.

[0109] See also Figure 7 , the evaluation model can be pre-trained in the following way:

[0110] S31 , collecting historical printing parameters and historical images obtained under each historical printing parameter, each historical image having a print quality label.

[0111] S32, importing each historical printing parameter and the corresponding historical image into the constructed regression model, and outputting the corresponding printing quality prediction result.

[0112] S33, constructing a loss function based on the print quality label and the print quality prediction result, and performing iterative training of the regression model guided by the loss function until a preset stopping condition is met, thereby obtaining a trained evaluation model.

[0113] In this embodiment, a large number of historical printing parameters can be collected, including powder spreading speed, roller rotation speed, drying power, drying time, binder injection head reference voltage, etc., and corresponding historical images monitored are obtained, wherein each historical image has a print quality label, which is the actual print quality of the historical image. The print quality can be comprehensively evaluated based on whether there are defects in the final product, the type and quantity of defects, etc.

[0114] Historical images can be used to train a machine learning-based evaluation model, such as a random forest regression model or a support vector regression model. Taking the random forest regression model as an example, multiple decision trees are constructed to learn from the input printing parameters and historical images. Each decision tree splits based on the data characteristics, and the print quality prediction result is ultimately obtained by averaging the predictions of multiple decision trees.

[0115] During training, a loss function is constructed based on the print quality labels of each historical image and the print quality prediction results obtained by the model to minimize the loss. Using this loss function as a guide, optimal model parameters, such as the number of decision trees and maximum depth, are selected through cross-validation to improve the model's generalization capabilities. Training stops when a preset stopping condition is reached, such as the number of iterations reaching a preset number, the iteration duration reaching a preset length, or the loss function reaching convergence and no longer changing, resulting in a completed evaluation model.

[0116] The evaluation model obtained in the above manner can be used to evaluate the pros and cons of printing parameters during the printing parameter optimization process, so as to ultimately find the optimal printing parameters in the global scope.

[0117] When the optimal printing parameters are obtained, adjustment information can be obtained, and each device in the printing system can be adjusted and controlled based on the adjustment information, and then subsequent additive manufacturing can be performed to improve the quality of the hierarchical image in the subsequent additive manufacturing process.

[0118] In this embodiment, it is considered that the printing parameters obtained by a single optimization may not fully guarantee the subsequent printing quality, or the difference between theoretical optimization and actual application may not guarantee the theoretical high-quality effect of the optimal printing parameters.

[0119] Therefore, based on the above, when performing subsequent additive manufacturing based on the adjustment information, an actual printing test can be performed, and a new hierarchical image can be acquired using the industrial camera 5. The same image analysis and feature extraction and other processing as described above are performed on the newly acquired hierarchical image, and then the defect recognition result is obtained using the evaluation model. The newly acquired defect recognition result is compared with the previously acquired defect recognition result to verify whether the printing quality has been improved. If the printing quality is improved and achieves the expected effect, subsequent processing can be performed based on the current evaluation model and recognition model. If the printing quality has not been improved, for example, there are still large defects, the new printing parameters, hierarchical images, and print quality obtained can be added to the data set to retrain the evaluation model, recognition model, etc., thereby improving the evaluation effect and recognition effect of the evaluation model and recognition model, and thereby improving the subsequent printing quality.

[0120] In this embodiment, the image processing unit uses algorithms such as background subtraction and convolutional neural networks (CNNs) to preprocess and analyze the collected image data, accurately identifying powder spreading uniformity, binder diffusion, and defect areas. Furthermore, the monitoring system incorporates deep learning capabilities, enabling continuous optimization of the evaluation model as monitoring data accumulates, improving monitoring accuracy and reliability. This is a core technical approach for achieving accurate defect identification and process optimization.

[0121] In this embodiment, in addition to mainly using machine vision to monitor binder jetting additive manufacturing, a monitoring solution of multiple sensor fusion can also be used to achieve comprehensive monitoring of binder jetting additive manufacturing.

[0122] In a possible implementation, the monitoring method provided in this embodiment may further include the following steps:

[0123] Acquire the acoustic emission signal detected by the acoustic emission sensor, perform frequency, amplitude and duration analysis on the acoustic emission signal, and obtain the acoustic emission signal analysis result; obtain the pressure information of the printing device detected by the pressure sensor, and the deformation information of the printing device detected by the strain sensor.

[0124] In this case, in the step of generating the adjustment information based on the optimal printing parameters, the adjustment information may be generated by combining the optimal printing parameters, the acoustic emission signal analysis results, the pressure information, and the deformation information.

[0125] Specifically, acoustic emission sensors, pressure sensors, and strain sensors can be combined to capture information about the printing process from different physical dimensions. Acoustic emission sensors can detect acoustic emission signals generated by powder particle collisions, binder jet impact, and other factors during the printing process. By analyzing characteristics such as the frequency, amplitude, and duration of these signals, abnormalities in the printing process, such as nozzle blockage and insufficient powder supply, can be identified.

[0126] Pressure sensors and strain sensors can be installed on key components such as the print head 2 and roller to monitor their stress and deformation during operation in real time, indirectly reflecting print quality and process stability. By integrating and analyzing the data acquired by these sensors with image data collected by the industrial camera 5, a more comprehensive and accurate assessment of the printing process can be achieved, improving the reliability and accuracy of defect detection.

[0127] In addition, in a possible implementation, the monitoring method provided in this embodiment may further include the following steps:

[0128] The distance information detected by the laser distance measuring sensor is obtained, and the geometric size information of the printed part is obtained based on the distance information.

[0129] In this case, in the step of generating the adjustment information based on the optimal printing parameters, the adjustment information may be generated by combining the optimal printing parameters and the geometric size information.

[0130] Specifically, introducing a monitoring solution combining a laser ranging sensor and an industrial camera 5 into the monitoring system can achieve accurate measurement of the geometric dimensions and surface topography of printed parts. The laser ranging sensor emits a laser beam and receives reflected light, measuring the distance between the laser beam and the surface of the object being measured, thereby obtaining geometric dimensional information such as the height and thickness of the printed part. At the same time, the industrial camera 5 captures surface images of the printed part for analysis of characteristics such as surface quality and binder distribution uniformity. Fusion of the data obtained by these two monitoring methods can achieve a comprehensive assessment of the internal and external quality of the printed part. For example, when testing the dimensional accuracy of a printed part, the laser ranging sensor provides accurate dimensional measurement data, while the industrial camera image can assist in determining whether dimensional deviations are caused by uneven binder injection or problems with the powder layer laying, thereby providing richer information for process optimization.

[0131] In this embodiment, the generated adjustment information is transmitted to each printing device, actively acquiring real-time adjustment information. Based on the relationship between the classified signals and part quality, dynamic adjustments are made to process parameters such as powder spreading speed, roller rotation speed, drying power, drying time, binder injection head reference voltage, and cleaning frequency. This real-time feedback and dynamic adjustment mechanism ensures the printing process always operates at optimal conditions, effectively improving print quality and process stability. It is a key link in the transition from monitoring to actual process optimization.

[0132] In this embodiment, the monitoring system primarily uses an industrial camera 5 for image acquisition. Industrial camera 5 is the primary device used for image acquisition in the current technical solution. However, in actual applications, other imaging devices can be considered to replace industrial camera 5. For example, a laser scanner can be used to obtain three-dimensional topographic information of the powder layer and the binder spraying area. By emitting a laser beam and receiving reflected light, a laser scanner can accurately measure the shape and size of an object's surface and has high accuracy for detecting the thickness and uniformity of the powder layer and the distribution of the binder. However, compared to industrial camera 5, the acquisition speed of a laser scanner may be slower and the equipment cost is relatively high.

[0133] In addition to traditional industrial cameras based on visible light imaging, cameras based on other imaging principles can also be used to achieve image acquisition. For example, infrared thermal imagers can detect the temperature distribution on the surface of an object, which has certain advantages for monitoring the drying condition of the binder during spraying and the thermal stress distribution of the powder layer. Through infrared thermal imagers, the temperature changes of the powder layer after the binder is sprayed can be observed in real time, thereby indirectly judging the degree of curing of the binder and the stability of the powder layer. However, the image resolution of infrared thermal imagers is relatively low, and it is impossible to directly obtain information such as the surface morphology of the powder layer and the spray trajectory of the binder. Therefore, in actual applications, it is necessary to select a camera based on specific monitoring needs.

[0134] In the image processing unit, the current technical solution uses algorithms such as background subtraction and convolutional neural networks (CNN) for image analysis and defect recognition. However, other image processing algorithms can also be considered to replace or supplement these algorithms. For example, operations based on mathematical morphology can be used to preprocess the image and enhance specific features in the image, such as the edges of powder particles and the diffusion area of ​​the binder. In addition, dimensionality reduction algorithms such as principal component analysis (PCA) can be used to extract the main feature information in the image, reduce the amount of data and improve processing efficiency. These algorithms have their own advantages and limitations in different application scenarios and need to be comprehensively considered and selected based on the actual image data characteristics and defect recognition requirements.

[0135] In addition to using convolutional neural networks (CNNs), other machine learning models can also be used for image classification and defect recognition. For example, support vector machines (SVMs) are a classification algorithm based on statistical learning theory that has good classification performance for small sample data sets. In binder jetting additive manufacturing defect recognition, the extracted image features can be used as input vectors, and the support vector machine model can be trained to classify the defect types. In addition, ensemble learning algorithms such as random forests can also be used for image classification tasks, improving classification accuracy and stability by constructing multiple decision trees and combining their classification results. These machine learning models exhibit different performance characteristics when processing different types of image data and defect features, and need to be selected and optimized according to specific circumstances.

[0136] In different application scenarios and technical conditions, the above-mentioned technical means can be reasonably selected and combined based on needs to help promote the diversified development and innovation of binder jetting additive manufacturing technology.

[0137] In summary, the monitoring method provided in this embodiment improves print quality through real-time monitoring and feedback. Specifically, an image acquisition device is positioned at a preset position in the print chamber to capture real-time image data of the powder layer and the binder injection area. Compared to traditional offline detection methods, this online monitoring method can promptly capture the printing status of each layer. For example, after the roller evenly spreads each layer of powder, an image of the powder layer surface is immediately captured. Using image processing algorithms, the uniformity and coverage of the powder layer are assessed in real time. Problems can be identified promptly and print parameters can be adjusted or printing can be paused for correction, effectively preventing defects from expanding and improving print quality and yield.

[0138] Furthermore, multi-angle and multi-spectral imaging is used to enhance defect recognition capabilities. Specifically, image acquisition devices, such as industrial camera 5, employ multi-spectral or high dynamic range (HDR) imaging technology and are equipped with polarizing filters to suppress interference from powder reflections. In complex optical environments, multi-spectral imaging captures image information from different wavelengths. Combined with polarizing filters, this effectively suppresses reflections, allowing industrial camera 5 to clearly capture the surface morphology of the powder layer and the surface after binder injection. Multi-angle and multi-spectral imaging comprehensively and clearly presents print details, improving defect recognition and monitoring accuracy.

[0139] Furthermore, intelligent image processing is combined with deep learning to achieve process optimization. Specifically, the image processing unit is equipped with advanced image processing algorithms such as background subtraction and convolutional neural networks (CNNs) to accurately identify and analyze image features and defects, overcoming the misjudgment problem of traditional algorithms. The monitoring system has deep learning capabilities, and as monitoring data accumulates, it optimizes the evaluation model, improving monitoring accuracy and intelligence. For example, historical inspection data is linked to printing parameters to generate a process optimization suggestion library, providing reasonable process parameter references for subsequent printing tasks, realizing intelligent monitoring and control, improving printing efficiency and quality, and reducing scrap rates.

[0140] Furthermore, adaptive control enables dynamic adjustment of process parameters. Specifically, an adaptive control module is set up to communicate with the machine tool to obtain real-time information. Based on the relationship between the classified signal and part quality, the machine tool is controlled and process parameters are adjusted. During the printing process, if the monitoring system detects an abnormality in powder spreading or binder injection, the adaptive control module can promptly receive feedback and dynamically adjust process parameters such as powder spreading speed, roller rotation speed, drying power, drying time, binder injection head reference voltage, and number of cleaning cycles. This ensures that the printing process remains optimal, improves the process stability and print quality consistency of binder jetting additive manufacturing, and reduces printing defects and quality issues.

[0141] The solution provided in this embodiment utilizes a deep learning algorithm and process optimization feedback mechanism to continuously optimize the evaluation model based on historical data, enabling intelligent monitoring and adaptive control, optimizing printing parameters, and improving process stability. An innovative three-level real-time closed-loop system, "industrial camera-edge computing-process execution," is constructed. This system utilizes a high-frame-rate industrial camera 5 to capture micron-level defects and combines it with a lightweight convolutional neural network to determine interlayer quality. A specially designed parameter compensation algorithm sends process adjustment instructions to the actuator before the next layer begins printing, forming a dynamic process optimization closed-loop, significantly reducing scrap rates and saving production costs.

[0142] A multispectral industrial camera 5 was developed, combined with a neural network feature selection algorithm to establish a print quality evaluation system. The innovative Unet++ network architecture simultaneously processes high-resolution images, enabling real-time detection of powder particle size distribution, cracks, and delamination. By integrating with the powder spreading and injection systems, adaptive adjustment of parameters such as spreading speed, reference voltage, and drying temperature is achieved.

[0143] By innovatively combining the multi-view industrial camera 5 with dynamic image analysis technology, the bottleneck of traditional monitoring methods has been broken through. The industrial camera 5 uses high-precision imaging, which can capture the droplet shape, speed and trajectory during the binder spraying process in real time, providing comprehensive data support for dynamic behavior. At the same time, the dynamic image analysis model based on machine learning performs intelligent processing on the collected image data, accurately identifies the diffusion boundary of the droplet, and quantifies its dynamic changes through algorithms to achieve real-time feedback and optimization of the spraying process. Through this combination of technologies, the present invention can effectively solve the problems of splashing and uneven diffusion during the binder spraying process, ensure that the binder penetrates evenly into the material, and significantly improve the density and mechanical properties of the parts. This solution not only improves manufacturing accuracy, but also provides an intelligent solution for quality control in industrial production, and promotes the development of additive manufacturing technology to a higher level.

[0144] In summary, this solution innovatively proposes an intelligent monitoring and feedback system that integrates deep learning models with multi-view industrial cameras. Through the closed-loop control mechanism of "perception-analysis-optimization", it completely overcomes the technical limitations of traditional systems.

[0145] Based on the same inventive concept, please refer to Figure 8 , an embodiment of the present invention further provides a functional module diagram of a binder jetting additive manufacturing monitoring device based on machine vision. This embodiment can divide the functional modules of the binder jetting additive manufacturing monitoring device based on machine vision according to the above-mentioned method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0146] For example, when each functional module is divided into corresponding functional modules, Figure 8The machine vision-based binder jetting AM monitoring device shown is only a schematic diagram. The machine vision-based binder jetting AM monitoring device can include an acquisition module, an extraction module, a recognition module, an optimization module, and an adjustment module. The functions of each module of the machine vision-based binder jetting AM monitoring device are described in detail below.

[0147] an acquisition module, configured to perform binder jetting additive manufacturing based on the constructed three-dimensional model, and to acquire layer images of each layer of the printed part in real time during the manufacturing process, the layer images including images of the laid powder and the sprayed binder;

[0148] An extraction module, for extracting a feature vector of a hierarchical image from a currently acquired hierarchical image;

[0149] The recognition module is used to use the pre-trained recognition model and perform defect recognition based on the feature vector to obtain a defect recognition result;

[0150] The optimization module is used to use a preset optimization algorithm to optimize the current printing parameters to obtain the optimal printing parameters when the defect recognition results do not meet the preset requirements;

[0151] The adjustment module is used to generate adjustment information based on the optimal printing parameters and perform the next level of additive manufacturing based on the adjustment information.

[0152] The machine vision-based binder jetting additive manufacturing monitoring device provided in this embodiment can be used to execute the machine vision-based binder jetting additive manufacturing monitoring method under any implementation method of the above embodiments. For any details not provided in this embodiment, please refer to the corresponding description of the above embodiments, and this embodiment will not be repeated here.

[0153] See also Figure 9 , is a block diagram of the structure of an electronic device provided in an embodiment of the present invention. This electronic device may be a computer device, server, or the like that integrates monitoring system 8 and control system 7. The electronic device includes a memory, a processor, and a communication module. The memory, processor, and communication module are electrically connected to each other, directly or indirectly, to enable data transmission or exchange. For example, these components may be electrically connected via one or more communication buses or signal lines.

[0154] Memory is used to store computer programs or data. Memory can be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM).

[0155] The processor is used to read / write data or programs stored in the memory and execute the machine vision-based binder jetting additive manufacturing monitoring method provided by any embodiment of the present invention.

[0156] The communication module is used to establish a communication connection between the electronic device and other communication terminals through the network, and is used to send and receive data through the network.

[0157] It should be understood that Figure 9 The structure shown is only a schematic diagram of the structure of the electronic device. The electronic device may also include Figure 9 More or fewer components than shown, or with Figure 9 Different configurations shown.

[0158] Furthermore, an embodiment of the present invention also provides a computer-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are executed, the machine vision-based binder jetting additive manufacturing monitoring method provided in the above embodiment is implemented.

[0159] Specifically, the computer-readable storage medium can be a general-purpose storage medium, such as a removable disk or hard disk. When the computer program on the computer-readable storage medium is executed, the aforementioned machine vision-based binder jetting additive manufacturing monitoring method can be executed. Regarding the processes involved in executing the computer-readable storage medium and its executable instructions, please refer to the relevant description in the aforementioned method embodiment and will not be further described here.

[0160] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms.

[0161] In addition, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] Furthermore, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0163] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0164] The above are merely embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A binder jetting additive manufacturing monitoring method based on machine vision, characterized in that: The method comprises: Performing binder jetting additive manufacturing based on the constructed three-dimensional model, and acquiring layer images of each layer of the printed part in real time during the manufacturing process, wherein the layer images include images of the laid powder and the sprayed binder; For the currently acquired hierarchical image, extracting a feature vector of the hierarchical image; Using the pre-trained recognition model and based on the feature vector to perform defect recognition to obtain a defect recognition result; When the defect recognition result does not meet the preset requirements, a preset optimization algorithm is used to optimize the current printing parameters to obtain the optimal printing parameters; Adjustment information is generated based on the optimal printing parameters, and additive manufacturing of the next level is performed based on the adjustment information.

2. The binder jetting additive manufacturing monitoring method based on machine vision according to claim 1, characterized in that: The step of extracting the feature vector of the hierarchical image includes: A threshold segmentation algorithm is used to segment the powder area and the non-powder area in the hierarchical image to obtain segmentation information; detecting the spraying trajectory and diffusion boundary of the adhesive in the hierarchical image based on an edge detection algorithm to obtain edge information; Using a convolutional neural network to extract features from the hierarchical image to obtain feature information; The segmentation information, edge information and feature information are combined to obtain a feature vector.

3. The binder jetting additive manufacturing monitoring method based on machine vision according to claim 1, characterized in that: The method further includes the step of pre-training to obtain the recognition model, including: Collecting a plurality of sample images, each of the sample images having defect annotation information, wherein the defect annotation information includes the position, shape, size and category of the defect; The constructed improved Unet++ network is trained using the multiple sample images until the training is stopped when a preset condition is met, thereby obtaining a trained recognition model.

4. The binder jetting additive manufacturing monitoring method based on machine vision according to claim 1, characterized in that: The step of using a preset optimization algorithm to search for the optimal printing parameters based on the current printing parameters to obtain the optimal printing parameters includes: Randomly generating a plurality of printing parameters including a current printing parameter, and optimizing the plurality of printing parameters based on a preset optimization algorithm; During the optimization process, corresponding print quality scores are obtained based on the print parameters, and the print quality scores are used as fitness values ​​to guide the optimization until the print parameters with the best fitness values ​​are found as the optimal print parameters.

5. The binder jetting additive manufacturing monitoring method based on machine vision according to claim 4, characterized in that: The step of obtaining a corresponding print quality score based on the print parameters includes: The printing parameters are imported into a pre-trained evaluation model for evaluation to obtain a printing quality score.

6. The binder jetting additive manufacturing monitoring method based on machine vision according to claim 5, characterized in that: The evaluation model is obtained in the following way: Collecting historical printing parameters and historical images obtained under each of the historical printing parameters, each of the historical images having a print quality label; Importing each of the historical printing parameters and the corresponding historical images into the constructed regression model, and outputting the corresponding print quality prediction results; A loss function is constructed based on the print quality label and the print quality prediction result, and the iterative training of the regression model is performed under the guidance of the loss function until a preset stopping condition is met, thereby obtaining a trained evaluation model.

7. The binder jetting additive manufacturing monitoring method based on machine vision according to claim 1, characterized in that: The method further comprises: Obtaining an acoustic emission signal detected by an acoustic emission sensor, performing frequency, amplitude, and duration analysis on the acoustic emission signal, and obtaining an acoustic emission signal analysis result; Obtaining pressure information of the printing device detected by the pressure sensor and deformation information of the printing device detected by the strain sensor; The step of generating adjustment information based on the optimal printing parameters includes: Adjustment information is generated by combining the optimal printing parameters, the acoustic emission signal analysis results, the pressure information, and the deformation information.

8. The binder jetting additive manufacturing monitoring method based on machine vision according to claim 1, characterized in that: The method further comprises: Obtaining distance information detected by a laser ranging sensor, and obtaining geometric size information of the printed work based on the distance information; The step of generating adjustment information based on the optimal printing parameters includes: Adjustment information is generated by combining the optimal printing parameters and the geometric dimension information.

9. A binder jetting additive manufacturing monitoring device based on machine vision, characterized in that: The device comprises: an acquisition module, configured to perform binder jetting additive manufacturing based on the constructed three-dimensional model, and to acquire a layer image of each layer of the printed part in real time during the manufacturing process, the layer image including images of the laid powder and the sprayed binder; An extraction module, configured to extract a feature vector of a currently acquired hierarchical image; An identification module, configured to use a pre-trained identification model and perform defect identification based on the feature vector to obtain a defect identification result; An optimization module is used to optimize the current printing parameters using a preset optimization algorithm to obtain the optimal printing parameters if the defect recognition result does not meet the preset requirements; An adjustment module is configured to generate adjustment information based on the optimal printing parameters and perform additive manufacturing at a next level based on the adjustment information.

10. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the method according to any one of claims 1 to 8.

Images