A laser welding state identification method based on convolutional neural network

Abstract

A laser welding state discrimination method based on a convolutional neural network relates to the field of welding technology. In order to solve the defects of the existing laser welding state discrimination method that it relies on single information for discrimination when facing complex welding states, has low accuracy, low precision, requires manual participation, and has a high defective rate of welds, the laser plume and molten pool morphology in the laser welding process are collected through preliminary experiments; the laser plume and molten pool morphology are processed separately by a three-dimensional convolutional neural network to obtain feature vectors; the feature vectors corresponding to the laser plume and molten pool morphology are fused through weight fusion, and the welding state is distinguished by a fully connected layer; the three-dimensional convolutional neural network is trained to obtain the output result of each fully connected layer, and each output result is transformed, and the welding state corresponding to the maximum value after the transformation is used as the final diagnosis result. The present invention is mainly used to discriminate the laser welding state.

Description

Technical Field

[0001] The invention relates to the technical field of welding, and in particular to a method for distinguishing a laser welding state. Background Art

[0002] As an advanced and efficient material connection method, laser welding has been gradually applied to hull subassemblies, mid-assembly components, marine exploration platforms, submersibles and other equipment, effectively driving the development of the marine economy. For example, on a certain type of transport ship, the length of the laser weld of a subassembly structural unit can reach more than 20 meters. Subassembly is the production process of processing the simplest hull structural components of a ship in a fixed site, which is to combine two or more parts into components, such as assembling T-shaped materials, installing reinforcement materials on ribs, and opening holes for reinforcement ribs. Therefore, maintaining a good laser welding state is very critical to the reliability of the hull structure.

[0003] At present, there are more and more automated laser welding equipment in the shipbuilding industry, and welding robots can automatically complete welding operations through the "teaching-reproduction" method. However, current laser welding equipment does not have the function of weld state detection, and cannot detect and adjust bad welding conditions in time. Therefore, a large number of manual inspections are still required after welding to find unqualified welds caused by bad welding conditions.

[0004] Advanced foreign laser welding equipment can distinguish the state based on a single welding process information, such as collecting digital signals such as current and voltage, molten pool morphology, etc. during the welding process, and comparing them with the predetermined reasonable range. When the signal strength is within the normal range, the state is judged to be normal; when the signal strength deviates from the normal range, the state is judged to be poor, and welding defects may occur. Although these technologies can provide a reference for the weld state, in actual applications, the welding state changes are complex, and it is difficult to achieve high-precision diagnosis of complex states through a single monitoring information. For example, the lock bottom joint is widely used in lightweight alloy welding structures such as aluminum and titanium. The lock bottom design can effectively ensure the back of the weld to form and prevent the molten pool from collapsing, but the lock bottom joint has a more complex welding state than the conventional butt joint. The welding state of the lock bottom joint can be divided into 5 categories, namely, insufficient penetration, butt penetration, good penetration, full penetration and over penetration; the welding state of the conventional butt joint has only 3 types: insufficient penetration, full penetration and over penetration. The existing welding state discrimination method is difficult to achieve high-precision discrimination of the complex 5 types of welding states.

[0005] Therefore, there is a need for a laser welding state identification method that can achieve high-precision diagnosis of welding states without the need for human intervention when facing complex welding states. Summary of the invention

[0006] The purpose of the present invention is to solve the defects of the existing laser welding state judgment method that, when faced with complex welding states, it relies on single information for judgment, has low accuracy, low precision, requires manual participation, and has a high weld defect rate. A laser welding state judgment method based on a convolutional neural network is provided, which has high accuracy, high welding precision, no manual participation, and a low weld defect rate when faced with complex welding states.

[0007] The laser welding state identification method based on convolutional neural network described in the present invention comprises the following steps:

[0008] S1: Collect the laser plume and molten pool morphology during laser welding through preliminary experiments;

[0009] S2: The laser plume and melt pool morphology are processed separately by a three-dimensional convolutional neural network to obtain feature vectors;

[0010] S3: The feature vectors corresponding to the laser plume and the molten pool morphology are fused by weight fusion to obtain a fusion vector for classification, and the welding state is distinguished by a fully connected layer;

[0011] S4: The images collected in the pre-experiment are used to train the three-dimensional convolutional neural network to obtain the output results of each fully connected layer, and each output result is transformed. The welding state corresponding to the maximum value after transformation is used as the final diagnosis result.

[0012] Further: in S2, two independent three-dimensional convolutional neural networks are used to process the laser plume and the molten pool morphology respectively, and the processing of the laser plume and the molten pool morphology specifically includes the following steps:

[0013] S21: transforming the laser plume and molten pool morphology obtained in the pre-experiment into images of uniform size; that is, the size of the transformed image is C*H*W, where C represents the number of image channels, H represents the pixel height of the image, and W represents the pixel width of the image;

[0014] S22: stacking pictures within any time length to form a picture sequence of size C*T*H*W, where T represents the number of pictures;

[0015] S23: inputting the image sequence C1*T1*H1*W1 of the laser plume and the image sequence C2*T2*H2*W2 of the molten pool morphology into two independent three-dimensional convolutional neural networks respectively, and extracting the temporal information and spatial information of the laser plume and the molten pool morphology at the same time;

[0016] S24: Each of the three-dimensional convolutional neural networks performs convolution on three scales H, W, and T simultaneously, and the input image sequence increases on the scale of image channel C, and decreases on the scales of image height H, image width W, and image number T, until the size of the image sequence becomes C'*1*1*1, thereby obtaining two feature vectors a and b with a size of C'*1.

[0017] Further: in S24, the three-dimensional convolutional neural network includes N layers of convolution, each layer of convolution is followed by a batch normalization layer, a nonlinear activation layer and a maximum pooling layer.

[0018] Further: in S3, the specific process of fusing the feature vectors corresponding to the laser plume and the molten pool morphology by weight fusion includes:

[0019] S31: Eigenvectors a and b are respectively combined with two weight coefficients and After multiplication and addition, we get the fusion vector c for classification. The result is as follows:

[0020] ;

[0021] The weight coefficient and The value range of is [0,1], and the sum of the two is 1, as follows:

[0022] , ;

[0023] S32: If the welding state needs to be distinguished into n categories, n pairs of weight coefficients are used. , get n fusion vectors , and the fusion vector Input an independent fully connected layer to obtain n 1*1 output results, and then concatenate the n output results to form a result vector of size n*1.

[0024] Further: in S3, the welding state includes insufficient penetration, butt penetration, good penetration, full penetration and over penetration.

[0025] Further: in S4, the three-dimensional convolutional neural network is trained using the pictures collected in the pre-experiment, specifically including the following steps:

[0026] S41: using the loss function to perform gradient descent solutions on n pairs of weight coefficients respectively, so that the weight parameters corresponding to different welding states converge to the optimal values ​​independently;

[0027] S42: The final output result vector is transformed into the range of [0,1] through the Sigmoid activation function, and then the welding state corresponding to the maximum value in the transformation result is taken as the final diagnosis result.

[0028] Furthermore: the loss function adopts a binary cross entropy function.

[0029] Further: In S1, the preliminary experiment collects the laser plume and molten pool morphology during the laser welding process using two industrial cameras; specifically including industrial camera No. 1, industrial camera No. 2, and a focusing device, wherein the industrial camera No. 1 is fixed paraxially to one side of the laser focusing device and is perpendicular to the horizontal plane, and is used to collect the laser plume; the industrial camera No. 2 is coaxially fixed to the laser focusing device, and the optical axes of the two coincide at the welding point, and is used to photograph the molten pool morphology.

[0030] Furthermore: when collecting the laser plume and the molten pool morphology, the two industrial cameras are set to be triggered synchronously, and the collection results at any time correspond to each other.

[0031] Furthermore: the acquisition speeds of the two cameras in the acquisition process are both greater than 100 Hz.

[0032] The beneficial effects of the present invention are:

[0033] The present invention is based on multi-visual information fusion, and simultaneously extracts the changing characteristics of laser plume and molten pool morphology on the time scale and spatial scale through a three-dimensional convolutional neural network, and realizes multi-information fusion based on weight parameters, which can reflect the importance of different information in the judgment of different welding states, improve the automatic laser welding system's ability to distinguish welding states, improve the accuracy of state judgment, reduce the workload of manual inspection, and reduce defective welds. The welding state detection method based on the weight fusion of laser plume and molten pool morphology proposed in the present invention can not only realize non-destructive detection of welds, but also achieve a detection error of no more than 2%. It is convenient and fast, suitable for automated, information-based and intelligent online detection applications, and has broad application prospects. The present invention realizes welding state judgment of the automated welding process by performing feature processing on different sensor information of the welding process, which has higher accuracy than single information judgment, thereby reducing the time of manual inspection after welding and reducing the welding scrap rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of a device for collecting the laser plume and molten pool morphology during laser welding;

[0035] Figure 2 It is a schematic diagram of the classification of welding states of butt joints;

[0036] Figure 3 It is a schematic diagram of the classification of welding status of the lock bottom joint;

[0037] Figure 4 It is a schematic diagram of the weight fusion three-dimensional convolutional neural network structure;

[0038] Figure 5 It is a schematic diagram of the comparison between the application effect and the common single information diagnosis method;

[0039] Among them, 1 is industrial camera No. 1, 2 is industrial camera No. 2, 3 is laser focusing device, and 4 is the workpiece to be welded. DETAILED DESCRIPTION

[0040] The following are only preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a technician familiar with the technical field within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. The embodiments described below are only used to explain the present invention and cannot be interpreted as limitations on the present invention. The protection scope of the present invention should be based on the protection scope of the claims. The embodiments of the present invention are described in detail below. In order to facilitate the description of the present invention and simplify the description, the technical terms used in the specification of the present invention should be interpreted in a broad sense, including but not limited to conventional replacement schemes not mentioned in this application, and also including direct implementation methods and indirect implementation methods.

[0041] Example 1

[0042] Combination Figure 1-Figure 5 This embodiment describes a laser welding state discrimination method based on a convolutional neural network, which mainly utilizes the fusion of visual monitoring information and a three-dimensional convolutional neural network to achieve multi-information fusion of laser welding plume and molten pool morphology through weight fusion, thereby achieving discrimination of complex welding states. Figure 1 As described above, the workpiece 4 to be welded is an aluminum alloy with a thickness of 3.0 mm and a bottom-locked structure. The laser power is set to 4.0 kW, the welding speed is 1.5 m / min, and the defocus is 0 mm. Due to the processing errors of the assembly and the plate, Figure 2 As shown in the figure, there are only three welding states for conventional butt joints: insufficient penetration, full penetration and excessive penetration. Figure 3 As shown, during welding, the welding state will randomly fluctuate among insufficient penetration, butt penetration, good penetration, full penetration and excessive penetration. The detection method comprises the following steps:

[0043] S1: The laser plume and molten pool morphology during laser welding are collected through preliminary experiments; two industrial cameras are used to collect welding process signals, such as Figure 1As shown, it includes an industrial camera No. 1 1, an industrial camera No. 2 2 and a laser focusing device 3, wherein the industrial camera No. 1 is paraxially fixed on one side of the laser focusing device 3; the paraxial fixation means that the industrial camera No. 1 is not fixed on the laser focusing device 3, but is fixed above the laser focusing device 3, with an angle of nearly 90° to the horizontal, and the industrial camera No. 1 is used to collect laser plume; the industrial camera No. 2 is coaxially fixed on the laser focusing device 3, and the coaxial fixation means that the industrial camera No. 2 is fixed on the left side of the workpiece 4 to be welded through the connecting axis of the laser focusing device 3, and the optical axes of the industrial camera No. 1 and the industrial camera No. 2 coincide at the welding point of the workpiece 4 to be welded, and the industrial camera No. 2 is used to photograph the morphology of the molten pool; the two industrial cameras are set to synchronous triggering during the acquisition process, so that the two industrial cameras start acquisition at the same time, and the acquisition results correspond to each other at any time; the acquisition rate of the two industrial cameras in the acquisition process is greater than 100Hz, and the acquisition rate can be set to 200Hz, and then the aluminum alloy plate to be welded is clamped, and after welding starts, acquisition commands are issued to the two industrial cameras at the same time until the welding is completed.

[0044] S2: using two independent three-dimensional convolutional neural networks to process the laser plume and the molten pool morphology respectively to obtain feature vectors; the processing of the laser plume and the molten pool morphology specifically includes the following steps:

[0045] S21: After welding, the data collected by the two industrial cameras are processed by the image acquisition card and then transmitted to the computer. Figure 4 As shown in the figure, the laser plume and molten pool morphology obtained in the pre-experiment are transformed into images of uniform size C*H*W; where C represents the number of image channels, H represents the pixel height of the image, and W represents the pixel width of the image; that is, the size of the transformed image is 8*400*256;

[0046] S22: Stack the images within any time length to form an image sequence of size C*T*H*W, where T represents the number of images; crop the image size to 400*256 pixels, and encapsulate it into an image sequence of size 1*8*400*256, where 1 is the number of image channels, 8 is the number of images, 400 is the pixel height, and 256 is the pixel width;

[0047] S23: Input the laser plume image sequence C1*T1*H1*W1 and the molten pool morphology image sequence C2*T2*H2*W2 into two independent three-dimensional convolutional neural networks respectively, and extract the time information and spatial information of the laser plume and the molten pool morphology at the same time; the three-dimensional convolutional neural network contains two independent convolution paths, and the input size of the convolution path is 8*1*400*256. The laser plume images and molten pool morphology images collected in the experiment need to be uniformly cropped to a size of 400*256 pixels.

[0048] S24: Each of the three-dimensional convolutional neural networks performs convolution at three scales, H, W, and T, and the three-dimensional convolutional neural network includes N layers of convolution, each layer of convolution is followed by a batch normalization layer, a ReLU nonlinear activation layer, and a 4*4 maximum pooling layer. After the maximum pooling layer, the output size of the image becomes one-fourth of the original size. The input image sequence increases in the scale of image channel C, and decreases in the scales of image height H, image width W, and number of images T until the image sequence size becomes C'*1*1*1, thereby obtaining two eigenvectors a and b of size C'*1. After the aforementioned two three-dimensional convolution paths, two eigenvectors of a final size of 256*1*1*1 are output, corresponding to the feature information in the laser plume and molten pool morphology signals respectively; according to experience, the convolution layer can be set to 3-8 layers, and the specific number of layers is determined according to the convolution effect;

[0049] S3: The feature vectors corresponding to the laser plume and the molten pool morphology are fused through weight fusion to obtain a fusion vector for classification, and the fully connected layer is used to distinguish the welding state.

[0050] S31: The fusion of feature vectors output by two 3D convolution paths is achieved through a special weight fusion method. Weight fusion is to combine feature vectors a and b with 5 sets of weight coefficients respectively. and After multiplication and addition, we get 5 fusion vectors c for classification. The results are as follows:

[0051] ;

[0052] The weight coefficient and The value range of is [0,1], and the sum of the two is 1, as follows:

[0053] , ;

[0054] The two weight coefficients are independent for the five welding states, and two weight coefficients are required for each welding state. Multiply vectors a and b by the five sets of weight parameters respectively and then sum them to obtain five fusion vectors, as follows:

[0055] ;

[0056] ;

[0057] The five fused vectors are input into five independent fully connected layers to obtain five output results. The fully connected layers are independent for the five types of welding states, and each welding state requires a fully connected layer. They are concatenated into a 5*1 output vector.

[0058] In this embodiment, the output vector is [-1.5, -0.5, 0.2, 1.1, 0.3].

[0059] S32: If the welding state needs to be distinguished into n categories, n pairs of weight coefficients are used. , we get n vectors , and the vectors Input an independent fully connected layer to obtain n 1*1 output results, and then concatenate the n output results to form a result vector of size n*1.

[0060] S4: The images collected in the pre-experiment are used to train the three-dimensional convolutional neural network, and the output results of each fully connected layer are obtained, and each output result is transformed, and the welding state corresponding to the maximum value after the transformation is used as the final diagnosis result. The output result of each fully connected layer in the neural network only determines the probability of occurrence of the corresponding welding state.

[0061] S41: Using a loss function to perform gradient descent on n pairs of weight coefficients respectively, so that the weight parameters corresponding to different welding states converge to the optimal values ​​independently; the loss function uses a binary cross entropy function. During the training process, the three-dimensional convolutional neural network needs to use a binary cross entropy loss function to train 5 groups of independent fully connected layers and information fusion weights respectively to obtain the optimal weight corresponding to each welding state.

[0062] The gradient descent method is implemented on the torch neural network framework through the Python programming language. First, the molten pool morphology and laser plume images are converted into input tensors through the torch.Dataset() class, and the input tensors are sent to the neural network through the torch.forward() function to implement feedforward operations and obtain the gradient of each layer in the neural network. Then, the gradient backpropagation and gradient descent are performed through the torch.backward() function, and the unknown parameters in the neural network will change along the direction of the maximum gradient; through multiple gradient descents, all unknown parameters in the neural network will gradually converge to a fixed value, so that the prediction results of the neural network for all image inputs are close to the actual welding state, and the value of the binary cross entropy function in S41 is close to 0.

[0063] This embodiment realizes the coordination of multiple information through an innovative weight fusion method. The weight coefficient set in step S3 and It exists directly in the computational path of the neural network, is converted into a trainable parameter through the torch.Parameter() class, and is given an initial value. In this embodiment, the initial value is set to 0.5. During the neural network training process, whenever the gradient descent method is executed through the torch.backward() function, the weight coefficient and It will change in the direction of maximum gradient together with the unknown parameters inside the neural network; through multiple gradient descents, the weight parameters corresponding to the five welding states will gradually converge to a fixed value, so that the prediction results of the neural network for all image inputs are close to the actual welding state, and the value of the binary cross entropy function in S41 is close to 0. After multiple gradient descents of the weight coefficients, the final results are as follows:

[0064] Table 1 Preferred weight fusion parameters in the implementation case of the present invention

[0065]

[0066] The obtained image sequence is divided into training set, validation set and test set in the ratio of 7:2:1. The images in the training set are used for training the three-dimensional convolutional neural network. The training rounds can be set to 500, the learning rate can be set to 0.01, and the learning rate is reduced by half after every 50 rounds of training. The loss function of the training adopts the binary cross entropy function.

[0067] The image sequence in the training set is input into the weighted fusion 3D convolutional neural network. The network structure is as follows Figure 3 The network parameters used are shown in Table 2.

[0068] Table 2 Parameters of weight fusion 3D convolutional neural network

[0069]

[0070] S42: The final output result vector is transformed into the range of [0,1] by the Sigmoid activation function, and the welding state corresponding to the maximum value in the transformation result is taken as the final diagnosis result. The output results of the five neural networks are transformed into the interval of [0,1] by the Sigmoid function to obtain five final results; the five final results represent the probability of occurrence of each welding state; the welding state corresponding to the maximum value among the five final results is the final welding state judgment result.

[0071] The output result vector is transformed into the range of [0,1] by Sigmoid function, and the welding state corresponding to the maximum value is taken as the final diagnosis result. In this embodiment, the transformation result of Sigmoid is [0.03, 0.09, 0.19, 0.47, 0.21], and the maximum value is 0.47, indicating that the welding state at this time is the fourth type of full penetration.

[0072] Through multiple tests, it was found that when only monitoring and distinguishing the molten pool morphology was performed, the distinction accuracy was about 82%, and when only monitoring and distinguishing the laser plume was performed, the distinction accuracy was about 86%. Compared with the traditional single information welding state distinction method, the diagnostic accuracy of this embodiment using the fusion of laser plume and molten pool morphology exceeded 98%, an improvement of more than 15%.

Claims

1. A laser welding state discrimination method based on convolutional neural network, characterized in that: The steps include: S1: Collect the laser plume and molten pool morphology during laser welding through preliminary experiments; S2: The laser plume and melt pool morphology are processed separately by a three-dimensional convolutional neural network to obtain feature vectors; In S2, two independent three-dimensional convolutional neural networks are used to process the laser plume and the molten pool morphology respectively. The processing of the laser plume and the molten pool morphology specifically includes the following steps: S21: transforming the laser plume and molten pool morphology obtained in the pre-experiment into images of uniform size; that is, the size of the transformed image is C*H*W, where C represents the number of image channels, H represents the pixel height of the image, and W represents the pixel width of the image; S22: stacking pictures within any time length to form a picture sequence of size C*T*H*W, where T represents the number of pictures; S23: inputting the image sequence C1*T1*H1*W1 of the laser plume and the image sequence C2*T2*H2*W2 of the molten pool morphology into two independent three-dimensional convolutional neural networks respectively, and extracting the temporal information and spatial information of the laser plume and the molten pool morphology at the same time; wherein C1 is the number of image channels of the laser plume, T1 is the pixel height of the laser plume image, H1 is the pixel height of the laser plume image, and W1 is the pixel width of the laser plume image; C2 is the number of image channels of the molten pool morphology, T2 is the pixel height of the molten pool morphology image, H2 is the pixel height of the molten pool morphology image, and W2 is the pixel width of the molten pool morphology image; S24: Each of the three-dimensional convolutional neural networks simultaneously performs convolution on three scales of H, W, and T, and the input image sequence increases on the scale of the number of image channels C, and decreases on the scales of the pixel height H of the image, the pixel width W of the image, and the number of images T, until the size of the image sequence becomes C'*1*1*1, thereby obtaining two feature vectors a and b with a size of C'*1; In S24, the three-dimensional convolutional neural network includes N layers of convolution, each layer of convolution is followed by a batch normalization layer, a nonlinear activation layer and a maximum pooling layer; S3: The feature vectors corresponding to the laser plume and the molten pool morphology are fused by weight fusion to obtain a fusion vector for classification, and the welding state is distinguished by a fully connected layer; In S3, the specific process of fusing the feature vectors corresponding to the laser plume and the molten pool morphology by weight fusion includes: S31: Eigenvectors a and b are respectively combined with two weight coefficients and After multiplication and addition, we get the fusion vector c for classification. The result is as follows: ； The weight coefficient and The value range of is [0,1], and the sum of the two is 1, as follows: ， ； S32: If the welding state needs to be distinguished into n categories, n pairs of weight coefficients are used. , get n fusion vectors , and the fusion vector Input an independent fully connected layer to obtain n 1*1 output results, and then concatenate the n output results to form a result vector of size n*1; S4: The images collected in the pre-experiment are used to train the three-dimensional convolutional neural network to obtain the output results of each fully connected layer, and each output result is transformed. The welding state corresponding to the maximum value after transformation is used as the final diagnosis result.

2. According to the method for distinguishing laser welding state based on convolutional neural network in claim 1, it is characterized in that: In S3, the welding conditions include insufficient penetration, butt penetration, good penetration, full penetration and excessive penetration.

3. The laser welding state discrimination method based on convolutional neural network according to claim 1 is characterized in that: In S4, the three-dimensional convolutional neural network is trained using the images collected in the pre-experiment, which specifically includes the following steps: S41: using the loss function to perform gradient descent solutions on n pairs of weight coefficients respectively, so that the weight parameters corresponding to different welding states converge to the optimal values ​​independently; S42: The final output result vector is transformed into the range of [0,1] through the Sigmoid activation function, and then the welding state corresponding to the maximum value in the transformation result is taken as the final diagnosis result.

4. The laser welding state identification method based on convolutional neural network according to claim 3 is characterized in that: The loss function adopts a binary cross entropy function.

5. The laser welding state identification method based on convolutional neural network according to claim 1 is characterized in that: In S1, the pre-experimental collection of the laser plume and molten pool morphology during the laser welding process is performed using two industrial cameras; specifically, the first industrial camera (1), the second industrial camera (2), and the laser focusing device (3), wherein the first industrial camera (1) is fixed on one side of the laser focusing device (3) and perpendicular to the horizontal plane, and the first industrial camera (1) is used to collect the laser plume; the second industrial camera (2) is coaxially fixed on the laser focusing device (3), and the optical axes of the two coincide at the welding point of the workpiece (4) to be welded, and the second industrial camera (2) is used to photograph the molten pool morphology.

6. The laser welding state identification method based on convolutional neural network according to claim 5 is characterized in that: When collecting the laser plume and the molten pool morphology, the two industrial cameras are set to be triggered synchronously, and the collection results at any time correspond to each other.

7. The laser welding state identification method based on convolutional neural network according to claim 5 is characterized in that: The acquisition rates of the two cameras in the acquisition process are both greater than 100 Hz.

Images