Molten pool and spatter monitoring method and application in selective laser melting based on target detection

Abstract

The present invention discloses a method and application for monitoring the molten pool and spatter during the selective laser melting process based on target detection. The method uses a deep learning-based target detection network to identify and capture the positions of spatter and molten pool. The steps include: 1. constructing a melting processing image dataset; 2. building a target detection model, including a feature extraction module, a feature fusion module, and an output module; 3. training the target detection model; and 4. obtaining the near-infrared processing image detection results of the selected laser melting to be monitored through the target detection model. The present invention can accurately extract the spatter and molten pool position information in the near-infrared image of the selective laser melting process, thereby realizing real-time process monitoring of the molten pool and spatter during the processing. The proposed target detection model has the characteristics of a small number of parameters, a small amount of calculation, a fast speed, and a high detection accuracy, thereby providing important assistance for real-time detection of the processing process.

Description

Technical Field

[0001] The present invention relates to the field of selective laser melting process monitoring, and in particular to a method for monitoring the molten pool and spatter of a selective laser melting process based on target detection, thereby completing the molten pool and spatter monitoring in near-infrared images of the selective laser melting process. Background Art

[0002] Selective laser melting is one of the important technologies in the field of additive manufacturing. With the development of the times, it has been widely applied in various fields. How to better control the processing process of selective laser melting and improve the quality of its products has been one of the key issues in recent years. How to achieve better process monitoring is one of the very important technical difficulties in studying the selective laser melting process. During the processing, near-infrared images can be collected by high-speed cameras. These images contain information about the molten pool and spatter during the processing. Through these image signals, the molten pool and spatter are extracted, and corresponding feature analysis is performed, which is of great help for the monitoring of the selective laser melting process.

[0003] In current near-infrared image-based selective laser melting process monitoring, the method for extracting spatter and melt pools mostly uses a relatively simple threshold-based binarization method. This method is generally unable to detect spatter with low brightness, and the detection effect is also very dependent on the accuracy of the threshold selection. At the same time, it is unable to adaptively adjust the threshold for different images and relies heavily on manual experience. Moreover, because the grayscale value distributions of spatter, plume, and melt pool often overlap in near-infrared images, the detection results of spatter and melt pool based on threshold segmentation have high missed detection and false detection rates, and the location of the melt pool and spatter is also inaccurate. Furthermore, after completing the threshold segmentation, the location of the melt pool and spatter must be located and classified, making it impossible to capture the melt pool and spatter from the original near-infrared image in a single step. Summary of the Invention

[0004] The present invention aims to address the deficiencies of the above-mentioned prior art and proposes a method and application for monitoring the molten pool and spatter during the selective laser melting process based on target detection, in order to solve the problems of high missed detection rate and false detection rate of the molten pool and spatter, as well as inaccurate positioning in the molten pool and spatter detection of the processing image of the selective laser melting process, thereby achieving a monitoring effect with high precision, small number of parameters, low missed detection rate and low false detection rate.

[0005] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0006] The present invention provides a method for monitoring the molten pool and spatter during selective laser melting based on target detection, which comprises the following steps:

[0007] Step 1: Construct an image dataset for input network:

[0008] The near-infrared images of the selected laser melting process are collected and cropped to obtain the melting process images. The positions and categories of the spatter and molten pool in the melting process images are annotated to obtain an image dataset containing the annotated information, which is recorded as ,in, represents the i-th melt-processed image, C, H, and W are the number of channels, height, and length of the image, respectively; Indicates the annotation information of the nth object in the i-th melt processing image, Represents the i-th melt processing image The number of labeled objects in ; and = ,in, Represents the i-th melt processing image The category of the nth object in , Represents the i-th melt processing image The position information of the nth object in is the width and height of the real frame where the nth object is located, is the coordinate of the center point of the real frame where the nth object is located; the object includes the molten pool and spatter;

[0009] Step 2: Using the Yolov5s network as the basic network model, build a target detection model, including: feature extraction module, feature fusion module, and output module;

[0010] Step 2.1: The feature extraction module consists of a SCBA module, three CBA modules, It consists of three CSP_x modules, two SCunit modules, and one SPPF module;

[0011] Step 2.1.1: The SCBA module consists of a convolution layer, a normalization layer, and an activation function layer in sequence; and the convolution kernel size of the convolution layer in the SCBA module is ;

[0012] Any CBA module consists of a convolution layer, a normalization layer, and an activation function layer in sequence; and the convolution kernel size of the convolution layer in each CBA module is ;

[0013] Any CSP_x module consists of two branches and a CBA module, where one branch includes a CBA module and x residual structures, and the other branch includes another CBA module; the outputs of the two branches are connected through a stacking operation and then connected to the input of the third CBA module;

[0014] Any SCunit module consists of a self-correcting convolution unit and an adder, and the output of the self-correcting convolution unit is connected to the input of the SCunit module through the adder; wherein the self-correcting convolution unit is composed of four convolution layers, a Sigmoid activation function, an average pooling layer, an upsampling layer, an adder, and a multiplier; the input of the self-correcting convolution unit is divided into two parts according to the number of channels, the first part of the input is output after passing through an average pooling layer, a first convolution layer, and an upsampling layer in sequence, the output of the upsampling layer and the first part of the input are respectively connected to the adder and then to the input of the Sigmoid activation function; the second part of the input is output after passing through the second convolution layer, the output of the second convolution layer and the output of the Sigmoid activation function are respectively connected to the multiplier and then to the input of the third convolution layer; the second part of the input is output after passing through the fourth convolution layer, and the output of the fourth convolution layer and the output of the third convolution layer are stacked to obtain the final output of the self-correcting convolution unit;

[0015] The SPPF module consists of two convolutional layers and three maximum pooling layers; the input of the SPPF module passes through the first convolutional layer, the first maximum pooling layer, the second maximum pooling layer, and the third maximum pooling layer in sequence; wherein, the output of the first convolutional layer is connected with the output of the first maximum pooling layer, the output of the second maximum pooling layer, and the output of the third maximum pooling layer through a stacking operation, and the output after the stacking operation passes through the second convolutional layer to obtain the final output of the SPPF module;

[0016] Step 2.1.2: The i-th melt-processed image Input into the feature extraction module, and pass through an SCBA module, the first CBA module and After processing by the CSP_x module, a feature map containing shallow semantic information is obtained. ; Then go through the second CBA module and After processing by the CSP_x module, a feature map containing mid-level semantic information is obtained. ; Then go through the first SCunit module, After processing by a CSP_x, a SPPF module, a third CBA module, and a second SCunit module, a feature map containing high-level semantic information is obtained. ;

[0017] The feature extraction module will represent different depth semantic information , , Output to the feature fusion module;

[0018] Step 2.2: The feature fusion module consists of two CBA modules and three attention mechanism CBAM modules;

[0019] Step 2.2.1: Any convolutional attention CBAM module includes a channel attention module and a spatial attention module; wherein the channel attention module includes: a maximum pooling layer, an average pooling layer, two multi-layer perceptron layers, an adder, a multiplier, and a sigmoid activation function; the spatial attention module includes: a maximum pooling layer, an average pooling layer, a multiplier, and a convolution layer;

[0020] The input of the convolutional attention CBAM module passes through the channel attention module and the spatial attention module in sequence before output;

[0021] The input of the channel attention module is divided into two branches. The first branch includes the first maximum pooling layer and the first multi-layer perceptron layer in sequence, and the second branch includes the first average pooling layer and the second multi-layer perceptron layer in sequence.

[0022] The output of the first multi-layer perceptron layer and the output of the second multi-layer perceptron layer are respectively connected to the adder and then to the input of the Sigmoid activation function. The output of the Sigmoid activation function is connected to the input of the channel attention module through the first multiplier and then output. The output of the first multiplier is used as the output of the channel attention module and input into the spatial attention module. The input of the spatial attention module is processed by the second maximum pooling layer and the second average pooling layer and then connected through the stacking operation Concat, and then connected to the input of the convolution layer. The output of the convolution layer and the input of the spatial attention module are respectively connected to the input of the second multiplier. The output of the second multiplier is the final output of the attention mechanism CBAM module;

[0023] Step 2.2.2: , , Input into the feature fusion module, After processing by the first CBAM module, the output low-dimensional feature map is obtained ;right After upsampling, it is input into the fourth CBA module and the second CBAM module for processing, and the medium-dimensional feature map is output. ;right After upsampling, it is input into the fifth CBA module and the third CBAM module for processing, and then the high-dimensional feature map is output. ;Will 、 、 After connecting through the stacking operation Concat, we get the feature stacking graph ;

[0024] Step 2.3: The output module consists of two CBA modules, one SCunit module and a Yolo Head layer;

[0025] The Yolo Head layer is the decoding layer of the yolov5s network. After being processed by the sixth CBA module, the seventh CBA module, and the third SCunit module, the feature fusion map is obtained. , the Yolo Head layer fuses the feature map Decoding conversion to target detection model output feature output map ;

[0026] Step 3: Use CIOU_loss loss function to calculate position loss, and use binary cross entropy loss function to calculate target confidence loss and classification loss; the total loss function Loss composed of the position loss, target confidence loss, and classification loss;

[0027] Based on the image dataset containing the labeled information, the target detection model is trained using a gradient descent method, and the total loss function Loss is calculated to update the model parameters. When the total loss function Loss converges, a trained target detection model is obtained;

[0028] Step 4: Input the near-infrared processed image of the selected area to be monitored into the trained target detection model to obtain several detection results, and use the non-maximum suppression method to filter the several detection results to obtain the final target detection result; the final result includes: the category of the object identified in the near-infrared processed image to be monitored and the location information of the detection frame where the object is located;

[0029] Step 5: Based on the final result of target detection, draw each detection frame on the processed image, and use different annotations to distinguish detection frames of different categories.

[0030] The present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the method for monitoring the molten pool and spatter during the selective laser melting process, and the processor is configured to execute the program stored in the memory.

[0031] The present invention provides a computer-readable storage medium having a computer program stored thereon. The computer-readable storage medium is characterized in that when the computer program is executed by a processor, the computer program executes the steps of the method for monitoring the molten pool and spatter during selective laser melting.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. Based on Yolov5s, this paper designs a new feature extraction module and feature fusion module, and builds a brand-new target detection model with the characteristics of small number of parameters, low computational complexity, and high detection accuracy. The self-correction and CBAM attention mechanisms are introduced. The multi-scale encoding capability of the self-correction convolutional unit enhances the feature extraction capability of the feature extraction module. At the same time, through the combined application of the attention mechanism CBAM and the self-correction convolutional unit, a new feature fusion strategy is proposed. Compared with other current methods for detecting the position of the molten pool and spatter, the target detection model of the present invention has unique advantages in extracting information from the original process image of the selective laser melting.

[0034] 2. The present invention can automatically detect the positions of the molten pool and spatter from the near-infrared image of the selective laser melting process and classify them. It can effectively extract the molten pool and spatter information in the original image, realize image-based feature capture, and thus contribute to subsequent feature extraction and processing process monitoring based on selective laser melting. It has practical significance in the image monitoring of selective laser melting. The target detection model proposed in the present invention has a low false detection rate for spatter and molten pool, accurate position positioning, and effectively improves the accuracy of selective laser melting process monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the overall implementation process of the present invention;

[0036] Figure 2 This is a schematic diagram of a high-speed camera process monitoring device used in a specific implementation method of the present invention;

[0037] Figure 3 It is a partial display diagram of the data set used in the specific implementation method of the present invention;

[0038] Figure 4 This is the SCBA module structure diagram;

[0039] Figure 5 This is the CBA module structure diagram;

[0040] Figure 6 This is the CSP_x module structure diagram;

[0041] Figure 7 This is the SCunit module structure diagram;

[0042] Figure 8 It is a diagram of the self-correcting convolution unit structure;

[0043] Figure 9 This is the SPPF module structure diagram;

[0044] Figure 10 This is the structure diagram of the attention mechanism CBAM module;

[0045] Figure 11 is a structural diagram of a target detection model used in a specific implementation method of the present invention;

[0046] Figure 12 It is a diagram showing the target detection effect of the molten pool and spatter according to the specific implementation method of the present invention. DETAILED DESCRIPTION

[0047] In this embodiment, the overall implementation process is as follows: Figure 1 As shown, it includes two parts: model preparation part and method application part. First, collect near-infrared images of the selective laser processing process, complete image preprocessing and label production, and divide the processed data set into training set, test set, and validation set. Use the training set to train the target detection model network proposed in this application, and save the final model training results. Afterwards, collect near-infrared images of the selective laser melting process to be monitored, input the pictures into the trained target detection model for target detection, and after completing the detection task, extract the corresponding molten pool and spatter position information. Specifically, the method for monitoring the molten pool and spatter in the selective laser melting process based on target detection includes the following steps:

[0048] Step 1: Construct an image dataset for input network:

[0049] In this example, the image dataset used was acquired using off-axis monitoring using a high-speed camera equipped with a wavelength-cutoff filter. During the selective laser melting process in this example, off-axis signal acquisition and monitoring was performed using a near-infrared camera within the selective laser melting system. The camera employed a high-speed FASTCAM Mini AX200type 200K-M-16GB near-infrared camera, with the angle between the camera and the substrate set at 45°. To prevent excessive plasma radiation from obscuring the state of the melt pool in the image, an external bandpass filter was used to filter the plasma radiation, thereby reducing the brightness of the plume.

[0050] In this specific example, an external bandpass filter with a wavelength of 350 nm to 800 nm is set to capture the image. During the processing, the processing material used is 316L stainless steel powder with a particle diameter of 10 μm to 45 μm. The schematic diagram of the high-speed camera process monitoring used in this example is shown in the figure. Figure 2As shown. The image part of the data set collected in this specific example is shown in Figure 3 ,The image used in this example is 3×250×515 in size.

[0051] The near-infrared images collected during the selective laser melting process are cropped to obtain the melting process images, and the positions and categories of the spatter and molten pool in the melting process images are annotated to obtain an image dataset containing the annotated information, which is recorded as ,in, represents the i-th melt-processed image, C, H, and W are the number of channels, height, and length of the image, respectively. In this embodiment, C, H, and W are 3, 250, and 515, respectively; Indicates the annotation information of the nth object in the i-th melt processing image, Represents the i-th melt processing image The number of labeled objects in ; and = ,in, Represents the i-th melt processing image The category of the nth object in , Represents the i-th melt processing image The position information of the nth object in is the width and height of the real frame where the nth object is located, The coordinates of the center point of the real frame where the nth object is located; objects include melt pools and spatters;

[0052] Step 2: Using the Yolov5s network as the basic network model, build a target detection model, including: feature extraction module, feature fusion module, and output module;

[0053] Step 2.1: In this embodiment, the feature extraction module consists of one SCBA module, three CBA modules, two CSP_1 modules, one CSP_2 module, two SCunit modules, and one SPPF module;

[0054] Step 2.1.1: The SCBA module consists of a convolution layer, a normalization layer, and an activation function layer in sequence; and the convolution kernel size of the convolution layer in the SCBA module is ;like Figure 4 As shown in the SCBA module.

[0055] Any CBA module consists of a convolution layer, a normalization layer, and an activation function layer in sequence; and the convolution kernel size of the convolution layer in each CBA module is ;like Figure 5 As shown in the CBA module.

[0056] Any CSP_x module consists of two branches and a CBA module, where one branch includes a CBA module and x residual structures, and the other branch includes another CBA module; the outputs of the two branches are connected through a stacking operation and then connected to the input of the third CBA module; Figure 6 As shown in the CSP_x module.

[0057] Any SCunit module consists of a self-correcting convolution unit and an adder. The output of the self-correcting convolution unit is connected to the input of the SCunit module through the adder. The self-correcting convolution unit consists of four convolution layers, a Sigmoid activation function, an average pooling layer, an upsampling layer, an adder, and a multiplier. The input of the self-correcting convolution unit is divided into two parts according to the number of channels. The first part of the input passes through an average pooling layer, the first convolution layer, and an upsampling layer in sequence and is output. The output of the upsampling layer and the first part of the input are respectively connected to the adder and then to the input of the Sigmoid activation function. The second part of the input passes through the second convolution layer and is output. The output of the second convolution layer and the output of the Sigmoid activation function are respectively connected to the multiplier and then to the input of the third convolution layer. The second part of the input passes through the fourth convolution layer and is output. The output of the fourth convolution layer and the output of the third convolution layer are stacked to obtain the final output of the self-correcting convolution unit. Figure 7 SCunit module and Figure 8 As shown in the self-correcting convolution unit in Figure 2.

[0058] The SPPF module consists of two convolutional layers and three maximum pooling layers. The input of the SPPF module passes through the first convolutional layer, the first maximum pooling layer, the second maximum pooling layer, and the third maximum pooling layer in sequence. The output of the first convolutional layer is connected with the output of the first maximum pooling layer, the output of the second maximum pooling layer, and the output of the third maximum pooling layer through a stacking operation. The output after the stacking operation passes through the second convolutional layer to obtain the final output of the SPPF module. Figure 9 As shown in the SPPF module.

[0059] Step 2.1.2: The i-th melt-processed image After inputting into the feature extraction module and being processed by an SCBA module, the first CBA module and the first CSP_1 module, a feature map containing shallow semantic information is obtained. ; After being processed by the second CBA module and the first CSP_2 module, a feature map containing mid-level semantic information is obtained. ; After being processed by the first SCunit module, the second CSP_1, an SPPF module, the third CBA module, and the second SCunit module, a feature map containing high-level semantic information is obtained. In the feature extraction process of the original Yolov5s network, the information of small targets will be gradually compressed, resulting in the lack of shallow information of spatter and molten pool in the deep network, which is not conducive to the detection of spatter and molten pool. The present invention designs a new feature extraction module to adjust the network depth and downsampling ratio, which can effectively improve the detection accuracy of the network model. At the same time, a self-correcting convolution module is introduced. The advantage of self-correcting convolution is that it can complete the modeling of channel dependencies, encode multi-scale information, and enrich the pertinence of the network output features.

[0060] The feature extraction module will represent different depth semantic information , , Output to the feature fusion module;

[0061] Step 2.2: The feature fusion module consists of two CBA modules and three attention mechanism CBAM modules;

[0062] Step 2.2.1: Any convolutional attention CBAM module includes a channel attention module and a spatial attention module. The channel attention module includes: a maximum pooling layer, an average pooling layer, two multi-layer perceptron layers, an adder, a multiplier, and a sigmoid activation function; the spatial attention module includes: a maximum pooling layer, an average pooling layer, a multiplier, and a convolution layer.

[0063] The input of the convolutional attention CBAM module passes through the channel attention module and the spatial attention module in sequence before output;

[0064] The input of the channel attention module is divided into two branches. The first branch includes the first maximum pooling layer and the first multi-layer perceptron layer in sequence, and the second branch includes the first average pooling layer and the second multi-layer perceptron layer in sequence.

[0065] The output of the first multi-layer perceptron layer and the output of the second multi-layer perceptron layer are respectively connected to the adder and then connected to the input of the Sigmoid activation function. The output of the Sigmoid activation function is connected to the input of the channel attention module through the first multiplier and then output. The output of the first multiplier is used as the output of the channel attention module and input into the spatial attention module. The input of the spatial attention module is processed by the second maximum pooling layer and the second average pooling layer respectively and then connected through the stacking operation Concat, and then connected to the input of the convolution layer. The output of the convolution layer and the input of the spatial attention module are respectively connected to the input of the second multiplier. The output of the second multiplier is the final output of the attention mechanism CBAM module; Figure 10 The attention mechanism CBAM module is shown.

[0066] Step 2.2.2: , , In the input feature fusion module, After processing by the first CBAM module, the output low-dimensional feature map is obtained ;right After upsampling, it is input into the fourth CBA module and the second CBAM module for processing, and the medium-dimensional feature map is output. ;right After upsampling, it is input into the fifth CBA module and the third CBAM module for processing, and then the high-dimensional feature map is output. ;Will 、 、 After connecting through the stacking operation Concat, we get the feature stacking graph In the feature fusion module, the attention mechanism module CBAM is introduced to enhance the representation of important information of different feature maps. In this way, different feature maps can obtain different enhancement effects, so that each feature map can "look" at the detected target more carefully from different perspectives and in different ways, making each feature map responsible for a unique function.

[0067] Step 2.3: The output module consists of two CBA modules, one SCunit module and the Yolo Head layer;

[0068] The Yolo Head layer is the decoding layer of the yolov5s network. After being processed by the sixth CBA module, the seventh CBA module, and the third SCunit module, the feature fusion map is obtained. , the Yolo Head layer fuses the features into a graph Decoding conversion to target detection model output feature output map Finally, after completing the channel stacking integration of the feature output layer, self-correcting convolution is introduced in the output module. Using self-correcting convolution, the features are once again encoded from a multi-scale perspective. From the three integrated feature stacking maps, it is adaptively learned and mined which feature layer contains more sufficient potential information, thereby enriching the information of the feature fusion map in a targeted manner.

[0069] Step 3: Through step 2, we get the target detection model used in this example. Its specific structure is as follows: Figure 11 As shown. The CIOU_loss loss function is used to calculate the position loss, and the binary cross entropy loss function is used to calculate the target confidence loss and classification loss; the total loss function Loss consists of the position loss, target confidence loss, and classification loss;

[0070] Based on the image dataset containing labeled information, the target detection model is trained using the gradient descent method, and the total loss function Loss is calculated to update the model parameters. When the total loss function Loss converges, the trained target detection model is obtained.

[0071] Step 4: Input the near-infrared processed image of the selected area to be monitored into the trained target detection model to obtain several detection results. The non-maximum suppression method is used to filter the several detection results to obtain the final target detection result. The final result includes: the category of the object identified in the near-infrared processed image to be monitored and the location information of the detection frame where the object is located;

[0072] Step 5: Based on the final result of target detection, draw each detection frame on the processed image, and use different annotations to distinguish detection frames of different categories.

[0073] Through the above steps, the target detection task of the molten pool and spatter in this example is completed. Some pictures of the target detection results obtained in this example are shown in Figure 12 , Figure 12 The class 0 labeled in the figure represents the target detection result of spatter, and the class 1 labeled in the figure represents the target detection result of the molten pool.

[0074] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the method for monitoring the molten pool and spatter during the selective laser melting process. The processor is configured to execute the program stored in the memory.

[0075] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for monitoring the molten pool and spatter during the selective laser melting process are executed.

[0076] In summary, this application addresses the issue of low melt pool and spatter identification accuracy in near-infrared image-based selective laser melting process monitoring. The proposed method uses a target detection model to identify and extract the type, location, and size of spatter and melt pool from raw near-infrared images. This method offers the advantages of high detection accuracy, good practicality, and a low missed detection rate.

[0077] At the same time, other target detection networks were adopted and trained using the same data set, and comparative analysis was performed to demonstrate the advantages of the target detection network of the present invention over other target detection networks.

[0078] Since this model is a lightweight network improved from Yolov5s, in addition to comparing it with Yolov5s itself, this application also uses several other popular lightweight object detection networks for comparison. The following three models were selected: 1) the SSD object detection network based on the mobilenetV2 backbone network; 2) the Yolov3-Tiny lightweight model based on Yolov3; and 3) the Yolox-s and Yolo-nano models proposed by Zheng Ge et al. The evaluation metrics used for comparison are the commonly used detection accuracy mAP@0.5, mAP@0.7, and the average detection accuracy mAP@0.5:0.95. The specific experimental results are shown in Table 1.

[0079] Table 1 Comparison of detection performance of different networks

[0080] Network Model mAP@0.5 mAP@0.7 mAP@0.5:0.95 SSD-Mobilenetv2 0.727 0.193 0.250 Yolov5s 0.873 0.476 0.396 Yolov3-Tiny 0.750 0.324 0.310 Yolox-s 0.878 0.289 0.335 Yolox-nano 0.845 0.240 0.307 Target detection model of the present invention 0.913 0.587 0.466

[0081] From the above experimental results comparison table, it can be seen that the target detection model proposed in this application has significantly better detection accuracy than other traditional target detection networks after targeted improvement based on the near-infrared image of the selective laser melting process as the detection task.

Claims

1. A method for monitoring the molten pool and spatter during selective laser melting based on target detection, characterized in that: The steps include: Step 1: Construct an image dataset for input network: The near-infrared images of the selected laser melting process are collected and cropped to obtain the melting process images. The positions and categories of the spatter and molten pool in the melting process images are annotated to obtain an image dataset containing the annotated information, which is recorded as ,in, represents the i-th melt-processed image, C, H, and W are the number of channels, height, and length of the image, respectively; Indicates the annotation information of the nth object in the i-th melt processing image, Represents the i-th melt processing image The number of labeled objects in ; and = ,in, Represents the i-th melt processing image The category of the nth object in , Represents the i-th melt processing image The position information of the nth object in is the width and height of the real frame where the nth object is located, is the coordinate of the center point of the real frame where the nth object is located; the object includes the molten pool and spatter; Step 2: Using the Yolov5s network as the basic network model, build a target detection model, including: feature extraction module, feature fusion module, and output module; Step 2.1: The feature extraction module consists of a SCBA module, three CBA modules, It consists of three CSP_x modules, two SCunit modules, and one SPPF module; Step 2.1.1: The SCBA module consists of a convolution layer, a normalization layer, and an activation function layer in sequence; and the convolution kernel size of the convolution layer in the SCBA module is ; Any CBA module consists of a convolution layer, a normalization layer, and an activation function layer in sequence; and the convolution kernel size of the convolution layer in each CBA module is ; Any CSP_x module consists of two branches and a CBA module, where one branch includes a CBA module and x residual structures, and the other branch includes another CBA module; the outputs of the two branches are connected through a stacking operation and then connected to the input of the third CBA module; Any SCunit module consists of a self-correcting convolution unit and an adder, and the output of the self-correcting convolution unit is connected to the input of the SCunit module through the adder; wherein the self-correcting convolution unit is composed of four convolution layers, a Sigmoid activation function, an average pooling layer, an upsampling layer, an adder, and a multiplier; the input of the self-correcting convolution unit is divided into two parts according to the number of channels, the first part of the input is output after passing through an average pooling layer, a first convolution layer, and an upsampling layer in sequence, the output of the upsampling layer and the first part of the input are respectively connected to the adder and then to the input of the Sigmoid activation function; the second part of the input is output after passing through the second convolution layer, the output of the second convolution layer and the output of the Sigmoid activation function are respectively connected to the multiplier and then to the input of the third convolution layer; the second part of the input is output after passing through the fourth convolution layer, and the output of the fourth convolution layer and the output of the third convolution layer are stacked to obtain the final output of the self-correcting convolution unit; The SPPF module consists of two convolutional layers and three maximum pooling layers; the input of the SPPF module passes through the first convolutional layer, the first maximum pooling layer, the second maximum pooling layer, and the third maximum pooling layer in sequence; wherein, the output of the first convolutional layer is connected with the output of the first maximum pooling layer, the output of the second maximum pooling layer, and the output of the third maximum pooling layer through a stacking operation, and the output after the stacking operation passes through the second convolutional layer to obtain the final output of the SPPF module; Step 2.1.2: The i-th melt-processed image Input into the feature extraction module, and pass through an SCBA module, the first CBA module and After processing by the CSP_x module, a feature map containing shallow semantic information is obtained. ; Then go through the second CBA module and After processing by the CSP_x module, a feature map containing mid-level semantic information is obtained. ; Then go through the first SCunit module, After processing by a CSP_x, a SPPF module, a third CBA module, and a second SCunit module, a feature map containing high-level semantic information is obtained. ; The feature extraction module will represent different depth semantic information , , Output to the feature fusion module; Step 2.2: The feature fusion module consists of two CBA modules and three attention mechanism CBAM modules; Step 2.2.1: Any convolutional attention CBAM module includes a channel attention module and a spatial attention module; wherein the channel attention module includes: a maximum pooling layer, an average pooling layer, two multi-layer perceptron layers, an adder, a multiplier, and a sigmoid activation function; the spatial attention module includes: a maximum pooling layer, an average pooling layer, a multiplier, and a convolution layer; The input of the convolutional attention CBAM module passes through the channel attention module and the spatial attention module in sequence before output; The input of the channel attention module is divided into two branches. The first branch includes the first maximum pooling layer and the first multi-layer perceptron layer in sequence, and the second branch includes the first average pooling layer and the second multi-layer perceptron layer in sequence. The output of the first multi-layer perceptron layer and the output of the second multi-layer perceptron layer are respectively connected to the adder and then to the input of the Sigmoid activation function. The output of the Sigmoid activation function is connected to the input of the channel attention module through the first multiplier and then output. The output of the first multiplier is used as the output of the channel attention module and input into the spatial attention module. The input of the spatial attention module is processed by the second maximum pooling layer and the second average pooling layer and then connected through the stacking operation Concat, and then connected to the input of the convolution layer. The output of the convolution layer and the input of the spatial attention module are respectively connected to the input of the second multiplier. The output of the second multiplier is the final output of the attention mechanism CBAM module; Step 2.2.2: , , Input into the feature fusion module, After processing by the first CBAM module, the output low-dimensional feature map is obtained ;right After upsampling, it is input into the fourth CBA module and the second CBAM module for processing, and the medium-dimensional feature map is output. ;right After upsampling, it is input into the fifth CBA module and the third CBAM module for processing, and then the high-dimensional feature map is output. ;Will 、 、 After connecting through the stacking operation Concat, we get the feature stacking graph ; Step 2.3: The output module consists of two CBA modules, one SCunit module and a Yolo Head layer; The Yolo Head layer is the decoding layer of the yolov5s network. After being processed by the sixth CBA module, the seventh CBA module, and the third SCunit module, the feature fusion map is obtained. , the Yolo Head layer fuses the feature map Decoding conversion to target detection model output feature output map ; Step 3: Use CIOU_loss loss function to calculate position loss, and use binary cross entropy loss function to calculate target confidence loss and classification loss; the total loss function Loss composed of the position loss, target confidence loss, and classification loss; Based on the image dataset containing the labeled information, the target detection model is trained using a gradient descent method, and the total loss function Loss is calculated to update the model parameters. When the total loss function Loss converges, a trained target detection model is obtained; Step 4: Input the near-infrared processed image of the selected area to be monitored into the trained target detection model to obtain several detection results, and use the non-maximum suppression method to filter the several detection results to obtain the final target detection result; the final result includes: the category of the object identified in the near-infrared processed image to be monitored and the location information of the detection frame where the object is located; Step 5: Based on the final result of target detection, draw each detection frame on the processed image, and use different annotations to distinguish detection frames of different categories.

2. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the method for monitoring the molten pool and spatter during the selective laser melting process according to claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for monitoring the molten pool and spatter during selective laser melting according to claim 1 are executed.

Images