Multi-joint weld groove contour extraction method based on improved U-Net network
By improving the U-NET network, the introduction of attention mechanism and characteristic fusion module, the problem of poor detection effect of welded sloping mouth outline has been solved, and more efficient welded sewing information extraction and anti-interference capabilities have been achieved. The landing.
Patent Information
- Application Number
- CN202411322425.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2025-01-28
AI Technical Summary
In the prior art, when detecting objects in images, there is a large difference between the detected target and the weld bevel profile, resulting in poor extraction effect and insufficient anti-interference ability.
Improve the U-Net network, enhance the network's characteristic extraction and anti-interference capabilities by introducing channel attention characteristics fusion module, residual pyramid pool module, multi-scale bottleneck attention module, and DICE LOSS loss function.
The effective extraction of the contours of the welded slot of multiple joints is realized, and the extraction effect and anti -interference ability are improved, and it is suitable for welding automation and intelligent fields.
Smart Images
Figure CN119380039A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of intelligent arc welding, and in particular to a method for extracting groove contours of multiple joint welds based on an improved U-Net network. Background Art
[0002] Laser vision sensing is one of the most common sensing methods in applications such as weld tracking and weld formation control. The successful implementation of these applications requires real-time extraction of the contour information of the weld groove under the welding of relevant joints. The weld is a seam formed by melting the metal at the welding rod and the joint using the high temperature of the welding heat source. The groove refers to a groove of a certain geometric shape that is processed and assembled at the welded part of the weldment. Since welding joints include butt joints, fillet joints, overlap joints and other forms, and butt joints and fillet joints have multiple layers and multiple passes of welding, the groove morphology changes during each welding pass. In addition, due to the different sensors designed by users, the morphology and interference background of the collected weld images are different, so the adaptability and anti-interference ability of the algorithm for extracting the weld groove contour of these joints using traditional machine learning technology are difficult to meet actual needs.
[0003] In recent years, with the rapid development of deep learning and artificial intelligence, various neural networks have been widely used in the field of automated and intelligent welding, and have become a research hotspot and focus in this field. Deep learning, especially deep convolutional neural networks, has greatly improved recognition accuracy, anti-interference ability, and robustness compared to traditional image processing methods.
[0004] Although semantic segmentation based on deep convolutional neural networks has achieved good results in target detection in images, these classic networks are all facing natural images, and there are large differences between the detected targets and the weld groove contours, such as: insufficient weld groove contour information in weld images, and the pixel scale of the weld groove contour is too small compared to the background. Therefore, these networks need to be improved to meet this specific demand. Summary of the invention
[0005] 1. Technical issues to be solved
[0006] The technical problem to be solved by the present invention is that when target detection is currently performed in an image, there is a large difference between the detected target and the weld groove profile.
[0007] (II) Technical solution
[0008] To solve the above technical problems, the technical solution provided by the present invention is: a method for extracting the groove contours of various joint welds based on an improved U-Net network, comprising the following steps: Step 1: In view of the information loss problem caused by multiple convolution and pooling operations of the original U-Net network, a channel attention feature fusion module is proposed to effectively learn the global context features of various joints from high- and low-level features; Step 2: In order to solve the similarity problem of laser stripes and welding spatter imaging features, a residual pyramid pooling module is introduced to capture the features of laser stripes and spatter in different spaces and channel dimensions through independent block area pooling, so as to achieve effective distinction between laser stripes and spatter; Step 3: A multi-scale bottleneck attention module is proposed to accumulate multi-scale feature maps and dynamically pay attention to the boundary information of laser stripes, so as to achieve accurate classification of laser stripe boundary pixels; Step 4: The loss function in the original U-Net network is replaced by Dice Loss for model training to solve the imbalance problem of laser stripes and background in pixel scale in the image.
[0009] As an improvement, the channel attention feature fusion module in step 1, which takes the input features Figure X The processing steps of and Y are as follows: Step 1: transform the features Figure X and Y are linearly superimposed; the second step: the results of the linear superposition are processed by global average pooling and global maximum pooling respectively; the third step: the results of the above two pooling processes are linearly superimposed; the fourth step: the result of the linear superposition in the third step is processed by: convolution + batch normalization + rectified linear unit, where the template of the convolution kernel in the convolution process is n×n; the fifth step: the result of the fourth step is processed by: convolution + batch normalization + Sigmoid activation function, where the template of the convolution kernel in the convolution process is n×n, and the obtained feature map is Fa; the sixth step: Fa is respectively compared with the input feature Figure X Multiply X×Fa and Y by each other; Step 7: Concatenate X×Fa, Y×Fa and Fa.
[0010] As an improvement, the residual pyramid pooling module of the step 2 processes the input feature map Z as follows: the first step: performing convolution + batch normalization + corrected linear unit processing on the feature map Z, wherein the convolution template size of the convolution operation is m×m, and the processing result is Z1; the second step: performing maximum pooling processing on Z1 to obtain Z2; the third step: performing maximum pooling processing on Z2 to obtain Z3; the fourth step: performing maximum pooling processing on Z3 to obtain Z4; the fifth step: performing splicing processing on Z1, Z2, Z3 and Z4; the sixth step: performing convolution + batch normalization + corrected linear unit processing on the result of the splicing processing, wherein the convolution template size of the convolution operation is m×m, and the processing result is Z5; the seventh step: performing linear superposition operation on Z1 and Z5, and the result is Z6; the eighth step: performing convolution + batch normalization + corrected linear unit processing on Z6, wherein the convolution template size of the convolution operation is m×m.
[0011] As an improvement, the multi-scale bottleneck attention module in step three processes the x input feature maps as follows: step 1: scale the x input feature maps to obtain a uniform size Q×Q; step 2: use the result of the scale processing as the input of the bottleneck attention sub-module for further processing by the module.
[0012] As an improvement, this submodule consists of a channel attention submodule and a spatial attention submodule.
[0013] As an improvement, the steps for processing the input feature map are as follows: the first step: perform global maximum pooling and global average pooling on the input feature map respectively; the second step: linearly superimpose the two pooling results; the third step: perform convolution (n×n) + modified linear unit activation function on the linear superposition result, where n is the convolution template size of the convolution operation; the fourth step: perform convolution (n×n) + Sigmoid activation function on the result of the third step, where n is the convolution template size of the convolution operation.
[0014] As an improvement, the processing steps for the input feature map are as follows: the first step: perform convolution (n×n) processing on the input feature map, and the processing result is T1, where n is the convolution template size of the convolution operation; the second step: perform convolution (m×m) processing and hole convolution (m×m) processing on T1 respectively, and the results obtained are T2 and T3 respectively, where m is the convolution template size of the convolution operation; the third step: linearly superimpose T1, T2 and T3; the fourth step: perform convolution (n×n) processing on the linear superposition result, where n is the convolution template size of the convolution operation; the fifth step: perform Sigmoid activation function processing on the processing result of the fourth step.
[0015] As an improvement, the processing steps of the above two sub-modules for the input feature map G are: Step 1: input G into the channel attention module to obtain the processing result G1; Step 2: input G into the spatial attention module to obtain the processing result G2; Step 3: multiply and fuse G1 and G2 to obtain G1×G2; Step 4: perform the following linear superposition to obtain the final result: G+G1×G2.
[0016] (III) Beneficial effects
[0017] The advantages of the present invention compared with the prior art are:
[0018] 1. Based on the classic U-Net semantic segmentation network, the present invention introduces an effective attention mechanism to learn different joint morphologies, laser stripe brightness characteristics, spatter characteristics, etc. After obtaining sufficient weld image sample sets and appropriate training methods, a unified, highly robust and anti-interference weld information extraction framework is constructed, achieving better extraction effects, which has a positive effect on promoting the implementation and promotion of laser vision sensing technology in welding automation and intelligence;
[0019] 2. The method for extracting the groove contours of various joint welds based on the improved U-Net network provided by the patent of this invention can effectively extract the groove contours of different welds of butt joints, corner joints, lap joints and multi-layer and multi-pass welding, and has the characteristics of universality. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is a diagram of the improved U-Net network structure provided by the present invention.
[0021] Figure 2 This is the structural diagram of the channel attention feature fusion module in the present invention.
[0022] Figure 3 This is a structural diagram of the residual pyramid pooling module in the present invention.
[0023] Figure 4 This is a structural diagram of the multi-scale bottleneck attention module in the present invention.
[0024] Figure 5 This is the structural diagram of the channel attention submodule in the present invention.
[0025] Figure 6 This is the structural diagram of the spatial attention submodule in the present invention.
[0026] Figure 7 This is the structural diagram of the bottleneck attention submodule in the present invention.
[0027] Figure 8 This is a sample diagram of the training image sample library required by the improved U-Net network in the present invention.
[0028] Fig. 9 This is an example diagram of the butt joint weld groove contour extraction results implemented using the improved U-Net network in the present invention.
[0029] Fig.10 This is an example diagram of the T-joint weld groove contour extraction results implemented using the improved U-Net network in the present invention.
[0030] Fig.11 This is an example diagram of the lap joint weld contour extraction results implemented using the improved U-Net network in the present invention. DETAILED DESCRIPTION
[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0032] The method for extracting groove contours of various joints welds based on an improved U-Net network includes the following steps: Step 1: Aiming at the information loss problem caused by multiple convolution and pooling operations of the original U-Net network, a channel attention feature fusion module is proposed to effectively learn the global context features of various joints from high- and low-level features. The channel attention feature fusion module of step 1 Figure X The processing steps of and Y are as follows: Step 1: transform the features Figure X and Y are linearly superimposed; the second step: the results of the linear superposition are processed by global average pooling and global maximum pooling respectively; the third step: the results of the above two pooling processes are linearly superimposed; the fourth step: the result of the linear superposition in the third step is processed by: convolution + batch normalization + rectified linear unit, where the template of the convolution kernel in the convolution process is n×n; the fifth step: the result of the fourth step is processed by: convolution + batch normalization + Sigmoid activation function, where the template of the convolution kernel in the convolution process is n×n, and the obtained feature map is Fa; the sixth step: Fa is respectively compared with the input feature Figure X Multiply X×Fa, Y×Fa and Fa; Step 7: Concatenate X×Fa, Y×Fa and Fa;
[0033] Step 2: In order to solve the similarity problem of laser stripes and welding spatter imaging features, the residual pyramid pooling module is introduced. Through independent block area pooling, the features of laser stripes and spatter in different spatial and channel dimensions are captured to achieve effective distinction between laser stripes and spatter; Step 3: A multi-scale bottleneck attention module is proposed to accumulate multi-scale feature maps and dynamically pay attention to laser stripe boundary information to achieve accurate classification of laser stripe boundary pixels; Step 4: Replace the loss function in the original U-Net network with Dice Loss for model training to solve the imbalance problem of laser stripes and background in pixel scale in the image. The residual pyramid pooling module of step 2 processes the input feature map Z as follows: Step 1: Perform convolution + batch normalization + rectified linear unit processing on the feature map Z, where the convolution template size of the convolution operation is m×m, and the processing result is Z1; Step 2: Perform maximum pooling on Z1 to obtain Z2; Step 3: Perform maximum pooling on Z2 to obtain Z3; Step 4: Perform maximum pooling on Z2 to obtain Z3; Step 5: Perform maximum pooling on Z1 to obtain Z4; Step 6: Perform maximum pooling on Z2 to obtain Z5; Step 7: Perform maximum pooling on Z2 to obtain Z6; Step 8: Perform maximum pooling on Z1 to obtain Z7; Step 9: Perform maximum pooling on Z2 to obtain Z8; Step 10: Perform maximum pooling on Z1 to obtain Z9; Step 11: Perform maximum pooling on Z1 to obtain Z11; Step 12: Perform maximum pooling on Z1 to obtain Z12; Step 13: Perform maximum pooling on Z2 to obtain Z3; Step 14: Perform maximum pooling on Z1 to obtain Z13; Step 15: Perform maximum pooling on Z1 to obtain Z14; Step 16: Perform maximum pooling on Z1 to obtain Z14; Step 17: Perform maximum pooling on Z1 to obtain Z14; Step 18: Perform maximum pooling on Z1 to obtain Z14; Step 19: Perform Step 1: Perform maximum pooling on Z3 to obtain Z4; Step 5: Concatenate Z1, Z2, Z3 and Z4; Step 6: Perform concatenation on the result of concatenation: convolution + batch normalization + rectified linear unit, where the convolution template size of the convolution operation is m×m, and the result is Z5; Step 7: Perform linear superposition on Z1 and Z5, and the result is Z6; Step 8: Perform convolution + batch normalization + rectified linear unit on Z6, where the convolution template size of the convolution operation is m×m;
[0034] The multi-scale bottleneck attention module in step three processes the x input feature maps as follows: step 1: scale the x input feature maps to obtain a uniform size Q×Q; step 2: use the scale processing result as the input of the bottleneck attention submodule, which is further processed by the module. The submodule consists of a channel attention submodule and a spatial attention submodule.
[0035] The steps for processing the input feature map are as follows: Step 1: Perform global maximum pooling and global average pooling on the input feature map respectively; Step 2: Linearly superimpose the two pooling results; Step 3: Perform convolution (n×n) + modified linear unit activation function on the linear superposition result, where n is the convolution template size of the convolution operation; Step 4: Perform convolution (n×n) + Sigmoid activation function on the result of the third step, where n is the convolution template size of the convolution operation. The steps for processing the input feature map are as follows: Step 1: Perform convolution (n×n) on the input feature map, and the result of the processing is T1, where n is the convolution template size of the convolution operation; Step 2: Perform convolution (m×m) and hole convolution on T1 respectively. The third step is to perform linear superposition (m×m) on T1, T2 and T3, where m is the size of the convolution template of the convolution operation; the fourth step is to perform convolution (n×n) on the linear superposition result, where n is the size of the convolution template of the convolution operation; the fifth step is to perform Sigmoid activation function on the result of the fourth step. The processing steps of the above two sub-modules on the input feature map G are as follows: the first step is to input G into the channel attention module to obtain the processing result G1; the second step is to input G into the spatial attention module to obtain the processing result G2; the third step is to multiply and fuse G1 and G2 to process G1×G2; the fourth step is to perform the following linear superposition to obtain the final result: G+G1×G2.
[0036] In the specific implementation of the present invention, a method for extracting the contours of weld grooves of various joints based on an improved U-Net network has the following main implementation steps: Step 1: assigning an initial value to the template size n of the convolution processing, and n=1; Step 2: assigning an initial value to the template size m of the convolution and hole convolution processing, and m=3; Step 3: initializing the number x of feature maps input by the multi-scale bottleneck attention module, and x=4; Step 4: initializing the size Q of the feature map output by the bottleneck attention module, and Q=256; Step 5: setting the training parameters of the improved U-Net network, and the setting method is: the optimizer is selected as the Adam algorithm, the initialization learning rate is 0.001, the small batch size is 8, and the maximum number of complete traversals is 30; Step 6: training the improved U-Net network proposed in this patent, and the sample ratio of the training set, the verification set and the test set is 3:1:1 Step 7: after the network training is completed, the pixels in the input image are classified into one category as the weld groove contour and the other category as the background.
[0037] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0038] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
[0039] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.
Claims
1. A method for extracting groove contours of various joint welds based on an improved U-Net network, characterized in that: The following steps are involved: Step 1: To address the information loss problem caused by multiple convolution and pooling operations in the original U-Net network, a channel attention feature fusion module is proposed to effectively learn multiple joint global context features from high- and low-level features; Step 2: To solve the similarity problem between the imaging features of laser streaks and welding spatters, the residual pyramid pooling module is introduced. Through independent block region pooling, the features of laser streaks and spatters in different spatial and channel dimensions are captured to achieve effective distinction between laser streaks and spatters. Step 3: Propose a multi-scale bottleneck attention module to accumulate multi-scale feature maps and dynamically focus on the laser stripe boundary information to achieve accurate classification of laser stripe boundary pixels; Step 4: Replace the loss function in the original U-Net network with Dice Loss for model training to solve the imbalance problem between laser stripes and background in the image at the pixel scale.
2. The method for extracting groove contours of various joint welds based on an improved U-Net network according to claim 1, characterized in that: The channel attention feature fusion module of step 1 processes the input feature maps X and Y as follows: Step 1: Linearly superimpose feature maps X and Y; Step 2: Perform global average pooling and global maximum pooling on the results of linear superposition; Step 3: Linearly superimpose the results of the above two pooling processes; Step 4: Perform convolution + batch normalization + rectified linear unit processing on the linear superposition result of step 3, where the template of the convolution kernel in the convolution processing is n×n; Step 5: Perform convolution + batch normalization + Sigmoid activation function processing on the result of step 4, where the template of the convolution kernel in the convolution processing is n×n, and the obtained feature map is Fa; Step 6: Multiply Fa with the input feature maps X and Y respectively; Step 7: Splice X×Fa, Y×Fa and Fa.
3. The method for extracting groove contours of various joint welds based on an improved U-Net network according to claim 1, characterized in that: The residual pyramid pooling module in step 2 processes the input feature map Z as follows: Step 1: Perform convolution + batch normalization + rectified linear unit processing on the feature map Z, where the convolution template size of the convolution operation is m×m, and the processing result is Z1; Step 2: Perform maximum pooling on Z1 to obtain Z2; Step 3: Perform maximum pooling on Z2 to obtain Z3; Step 4: Perform maximum pooling on Z3 to obtain Z4; Step 5: Splice Z1, Z2, Z3 and Z4; Step 6: Perform concatenation processing on the result of the concatenation process: convolution + batch normalization + rectified linear unit processing, where the convolution template size of the convolution operation is m×m, and the processing result is Z5; Step 7: Perform linear superposition operation on Z1 and Z5, and the result is Z6; Step 8: Perform convolution + batch normalization + rectified linear unit processing on Z6, where the convolution template size of the convolution operation is m×m.
4. The method for extracting groove contours of various joint welds based on an improved U-Net network according to claim 1, characterized in that: The multi-scale bottleneck attention module in step 3 processes the x input feature maps as follows: Step 1: Scale the input x feature maps to obtain a uniform size Q×Q; Step 2: The result of the scale processing is used as the input of the bottleneck attention submodule for further processing.
5. The method for extracting groove contours of various joint welds based on the improved U-Net network according to claim 4 is characterized in that: This submodule consists of a channel attention submodule and a spatial attention submodule.
6. The method for extracting groove contours of various joint welds based on an improved U-Net network according to claim 5, characterized in that: The processing steps for the input feature map are: Step 1: Perform global maximum pooling and global average pooling on the input feature map respectively; Step 2: Linearly superimpose the two pooling results; Step 3: Perform convolution (n×n) + rectified linear unit activation function on the linear superposition result, where n is the convolution template size of the convolution operation; Step 4: Perform convolution (n×n) + Sigmoid activation function processing on the result of the third step, where n is the convolution template size of the convolution operation.
7. The method for extracting groove contours of various joint welds based on an improved U-Net network according to claim 5, characterized in that: The processing steps for the input feature map are: Step 1: Perform convolution (n×n) on the input feature map, and the result is T1, where n is the convolution template size of the convolution operation; Step 2: Perform convolution (m×m) and dilated convolution (m×m) on T1, and the results are T2 and T3, respectively, where m is the size of the convolution template of the convolution operation; Step 3: Linearly superimpose T1, T2 and T3; Step 4: Perform convolution (n×n) on the linear superposition result, where n is the convolution template size of the convolution operation; Step 5: Apply Sigmoid activation function to the result of step 4.
8. A channel attention submodule and a spatial attention submodule according to claim 5, characterized in that: The processing steps of the above two sub-modules on the input feature map G are as follows: Step 1: Input G into the channel attention module to obtain the processing result G1; Step 2: Input G into the spatial attention module to obtain the processing result G2; Step 3: Multiply G1 and G2 to form G1×G2; Step 4: Perform the following linear superposition to obtain the final result: G+G1×G2.