Universal method for extracting contour of weld groove
By introducing an attention mechanism into the SegNet network, the problem of insufficient algorithm adaptability and anti-interference ability in the prior art is solved, and better bevel profile extraction effect and spatial structure capture are achieved.
Patent Information
- Application Number
- CN202411631937.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-15
- Publication Date
- 2025-03-18
AI Technical Summary
In the prior art, when extracting the weld bevel profile, the adaptability and anti-interference ability of the algorithm are difficult to meet the actual requirements, especially when there is insufficient information in the bevel image and large background interference.
Based on the classic SegNet semantic segmentation network, an attention mechanism is introduced, and through the efficient convolution attention module and attention feature fusion module, different joint morphology, laser stripe brightness characteristics, splash characteristics, etc. are learned to improve the extraction effect.
The adaptability and anti-interference ability of weld bevel profile extraction are achieved, which can better distinguish the characteristics of weld contour and splash, effectively capture the spatial structure of weld contour in the image, and improve the problem of category imbalance.
Smart Images
Figure CN119648727A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of automated and intelligent welding, and in particular to a general method for extracting weld groove contours. Background Art
[0002] In recent years, with the rapid development of deep learning and artificial intelligence, various neural networks have been widely used in the field of automated and intelligent welding, and have become a hot topic and focus of research in this field. Compared with traditional image processing methods, deep learning methods have been greatly improved in recognition accuracy, anti-interference ability, and robustness. Among them, semantic segmentation based on deep convolutional neural networks for target detection in images is a typical representative of successful practical applications.
[0003] At the same time, laser vision sensing is one of the common sensing methods in applications such as weld tracking and weld formation detection, which require real-time extraction of weld (groove) contour information. Although semantic segmentation based on deep convolutional neural networks has achieved good results in target detection in images. However, these networks are all facing natural images, and there are large differences between the detected targets and the groove contours, such as: insufficient groove contour information in the groove image, the groove contour pixel scale is too small compared to the background, the groove has multiple forms such as butt joints, corner joints, and overlaps, butt joints and corner joints have multiple layers and multiple passes of welding, the groove morphology will change during welding, and the interference background of the groove image is different, resulting in the algorithm's adaptability and ability to resist various interferences when extracting the groove contour. It is difficult to meet the actual requirements.
[0004] Therefore, these networks need to be improved to meet this specific demand. Based on the classic SegNet semantic segmentation network, the present invention introduces the attention mechanism to learn different joint morphologies, laser stripe brightness characteristics, splash characteristics, etc. After obtaining sufficient groove image sample sets and appropriate training methods, a better extraction effect is achieved, which improves the problems existing in the prior art. Summary of the invention
[0005] The purpose of the present invention is to introduce an attention mechanism to learn different joint morphologies, laser stripe brightness characteristics, splash characteristics, etc., and to improve the problem that the adaptability and anti-interference ability of the algorithm when extracting the groove contour in the existing technology are difficult to meet the actual requirements after obtaining a sufficient set of groove image samples and a suitable training method.
[0006] The present invention provides a general method for extracting weld groove contour, comprising the following steps:
[0007] The encoder and decoder in the original SegNet network are used to implement the standardized processing flow of feature extraction: convolution, batch normalization and rectified linear unit, which are uniformly set to two batches;
[0008] An efficient convolution attention module is embedded between the encoder and the decoder, and an attention feature fusion module is embedded in the decoder to obtain a groove contour extraction model; wherein the efficient convolution attention module is composed of an efficient channel attention submodule and a spatial attention submodule connected in series, and is used to perform efficient convolution processing on the output features of the encoder and then input them into the decoder; the attention feature fusion module is used to perform feature fusion on the features before downsampling in the encoder and the features after the first batch of processing in the decoder;
[0009] A marked welding groove sample set is obtained, and the groove contour extraction model is trained based on the sample set, and the groove contour extraction is performed on the groove image to be extracted based on the trained groove contour extraction model.
[0010] The general method for extracting weld groove contour provided by the present invention has the technical effect of strong adaptability and strong anti-interference ability. It can better distinguish the characteristics of different channels of weld (groove) contour and welding spatter, and effectively capture the spatial structure of weld (groove) contour in the image, and can improve the category imbalance problem of large pixel size difference between background and weld (groove) contour;
[0011] Optionally, the efficient convolution processing step includes:
[0012] Input the encoder output features into the efficient channel attention submodule;
[0013] The efficient channel attention submodule performs pooling, convolution, and activation function processing on the input features and then outputs them.
[0014] The input features and output features of the efficient channel attention submodule are fused, and the fused features are used as the input of the spatial attention submodule;
[0015] The spatial attention submodule performs global average pooling and global maximum pooling on the input features respectively, and linearly superimposes, convolves, and performs activation function processing on the results of the two pooling processes;
[0016] The input features and output features of the spatial attention submodule are fused and fed into the decoder.
[0017] Optionally, the step of performing feature fusion by the attention feature fusion module includes:
[0018] Linearly superimpose the features before downsampling in the encoder and the features after the first batch of processing in the decoder to obtain linear superposition features, and pool the linear superposition features to obtain pooling features;
[0019] The linear superposition feature and the pooling feature are convolved, batch normalized, rectified linear unit, re-convolved, and re-batch normalized respectively, and the two results are linearly superimposed, activated and processed to fuse with the linear superposition feature to obtain the preliminary fusion feature;
[0020] The preliminary fusion feature is weighted for the first time and fused with the feature before encoder downsampling to obtain the first feature. The preliminary fusion feature is weighted for the second time and fused with the feature after the first batch of processing of the decoder to obtain the second feature. The first feature and the second feature are linearly superimposed to obtain the fusion feature.
[0021] Optionally, the convolution in the efficient convolution processing step is a dilated convolution operation.
[0022] Optionally, the fusion in the efficient convolution processing step is multiplication fusion.
[0023] Optionally, the pooling in the efficient convolution processing step is global average pooling.
[0024] Optionally, the convolution and reconvolution in the step of feature fusion performed by the attention feature fusion module are both point-by-point convolution operations.
[0025] Optionally, the fusion in the step of feature fusion performed by the attention feature fusion module is multiplication fusion.
[0026] Optionally, the pooling in the step of feature fusion performed by the attention feature fusion module is global maximum pooling.
[0027] Optionally, in the step of feature fusion performed by the attention feature fusion module, the sum of the first weighted weight and the second weighted weight of the preliminary fusion feature is 1. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 The improved SegNet network structure diagram provided by the present invention;
[0029] Figure 2 This is a structural diagram of the efficient convolutional attention module in the present invention;
[0030] Figure 3 This is a structural diagram of the efficient channel attention submodule in the present invention;
[0031] Figure 4 This is a structural diagram of the spatial attention submodule in the present invention;
[0032] Figure 5 This is a structural diagram of the multi-scale channel attention submodule in the present invention;
[0033] Figure 6 This is a structural diagram of the attention feature fusion module in the present invention;
[0034] Figure 7 A sample image of a training image sample library required by the improved SegNet network of the present invention;
[0035] Figure 8 This is an example diagram of the butt joint weld groove contour extraction result implemented by using the improved SegNet network in the present invention;
[0036] Fig. 9 This is an example diagram of the corner joint weld groove contour extraction result implemented by the improved SegNet network in the present invention;
[0037] Fig.10 This is an example diagram of the lap joint weld contour extraction results implemented using the improved SegNet network in the present invention. DETAILED DESCRIPTION
[0038] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0039] The embodiment of the present invention provides a general method for extracting weld groove contour, comprising the following steps:
[0040] In the first step, the encoder and decoder in the original SegNet network are used to implement the standardized processing flow of feature extraction: convolution, batch normalization, and rectified linear units, which are uniformly set to two batches;
[0041] The second step is to embed an efficient convolution attention module between the encoder and the decoder, and embed an attention feature fusion module in the decoder to obtain a groove contour extraction model; wherein the efficient convolution attention module is composed of an efficient channel attention submodule and a spatial attention submodule connected in series, and is used to perform efficient convolution processing on the output features of the encoder and then input them into the decoder; the attention feature fusion module is used to perform feature fusion on the features before downsampling in the encoder and the features after the first batch of processing in the decoder;
[0042] The third step is to obtain a marked welding groove sample set, and train the groove contour extraction model based on the sample set, and perform groove contour extraction on the groove image to be extracted based on the trained groove contour extraction model.
[0043] When executing the second step, the improved SegNet network structure is as follows Figure 1 shown.
[0044] The structure of the efficient convolutional attention module is as follows Figure 2 As shown in Figure 2, the efficient channel attention submodule is Figure 3 As shown in Figure 2, the spatial attention submodule is Figure 4 shown.
[0045] The specific steps of the efficient convolution attention module for efficient convolution processing are as follows:
[0046] The input feature map F is average pooled and the result is subjected to a dilated convolution operation with a template of n×n.
[0047] The result of the n×n dilated convolution operation is processed by the Sigmoid activation function, and the result is multiplied and fused with the feature map F to obtain the feature map M. e ;
[0048] Combine the feature map F with the feature map M e Multiply and fuse to get the feature map E;
[0049] Perform global average pooling and global maximum pooling on the feature map E respectively;
[0050] The results of the two pooling processes are linearly superimposed, and a dilated convolution operation with a template of m×m is performed on the superimposed result;
[0051] The result of the m×m dilated convolution operation is processed by the Sigmoid activation function, and the result is multiplied and fused with the feature map E to obtain the feature map M s ;
[0052] Combine feature map E with feature map M s Multiply and fuse to get the feature map M R .
[0053] Among them, the template size n of the dilated convolution in the efficient channel attention submodule is initialized to 5, and the template size m of the dilated convolution in the spatial attention submodule is initialized to 7.
[0054] When executing the second step, the attention feature fusion module structure is as follows Figure 6 As shown in the figure, the core multi-scale channel attention submodule structure is as follows Figure 5 shown.
[0055] The specific steps of feature fusion in the attention feature fusion module are as follows:
[0056] Linearly superimpose the input feature map Y and feature map Z to obtain the feature Figure X ;
[0057] Features Figure X Perform global maximum pooling and perform point-by-point convolution on the result;
[0058] The point-by-point convolution results are batch normalized and the results are processed by the rectified linear unit;
[0059] The results of the rectified linear unit processing are then subjected to point-by-point convolution processing, and the results are batch normalized to obtain the features Figure X 1;
[0060] Features Figure X Perform point-by-point convolution and perform batch normalization on the results;
[0061] The result of batch normalization is processed by the rectified linear unit, and the result is convolved point by point; the result of point-by-point convolution is processed by batch normalization to obtain the feature Figure X 2;
[0062] Features Figure X 1 and features Figure X 2 Perform linear superposition and activate the result with Sigmoid function;
[0063] The result after Sigmoid function activation and feature Figure X Perform multiplication and fusion processing to obtain feature map N;
[0064] Multiply the feature map Y with the first weighted feature map N to obtain the feature map N1;
[0065] Combine the feature map Z with the second weighted feature Figure 1 -N multiply and fuse to obtain feature map N2;
[0066] N1 and N2 are linearly superimposed to obtain a fused feature map.
[0067] Among them, the convolution kernel size of point-by-point convolution is The convolution kernel size for point-by-point convolution is C is the number of feature channels, r is the channel reduction ratio, the number of channels C and the channel reduction ratio r in the multi-scale channel attention submodule are initialized, and C=3, r=4, and the sum of the weights of the first weighting and the second weighting is 1.
[0068] When executing the third step, the sample images of the training image sample library are as follows Figure 7 shown.
[0069] The training parameters of the improved SegNet network are set as follows: the optimizer is selected as the Adam algorithm, the initial learning rate is 0.001, the mini-batch size is 8, and the maximum number of complete traversals is 30;
[0070] The sample ratio of the training set, validation set, and test set is 3:1:1
[0071] The pixels in the input image are classified into two categories: one is the weld (groove) contour and the other is the background.
[0072] The example of the extraction result of the butt joint weld groove contour implemented by the improved SegNet network in the present invention is shown in the figure Figure 8 shown.
[0073] The example of the corner joint weld groove contour extraction result implemented by the improved SegNet network in the present invention is shown in the figure Fig. 9 shown.
[0074] The example of the weld contour extraction result of the lap joint implemented by the improved SegNet network in the present invention is shown in the figure Fig.10 shown.
[0075] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.
Claims
1. A general method for extracting weld groove contour, characterized in that: The following steps are involved: The encoder and decoder in the original SegNet network are used to implement the standardized processing flow of feature extraction: convolution, batch normalization and rectified linear unit, which are uniformly set to two batches; An efficient convolution attention module is embedded between the encoder and the decoder, and an attention feature fusion module is embedded in the decoder to obtain a groove contour extraction model; wherein the efficient convolution attention module is composed of an efficient channel attention submodule and a spatial attention submodule connected in series, and is used to perform efficient convolution processing on the output features of the encoder and then input them into the decoder; the attention feature fusion module is used to perform feature fusion on the features before downsampling in the encoder and the features after the first batch of processing in the decoder; A marked welding groove sample set is obtained, and the groove contour extraction model is trained based on the sample set, and the groove contour extraction is performed on the groove image to be extracted based on the trained groove contour extraction model.
2. The method for extracting groove profile according to claim 1, characterized in that: The steps for efficient convolution processing include: Input the encoder output features into the efficient channel attention submodule; The efficient channel attention submodule performs pooling, convolution, and activation function processing on the input features and then outputs them. The input features and output features of the efficient channel attention submodule are fused, and the fused features are used as the input of the spatial attention submodule; The spatial attention submodule performs global average pooling and global maximum pooling on the input features respectively, and linearly superimposes, convolves, and performs activation function processing on the results of the two pooling processes; The input features and output features of the spatial attention submodule are fused and fed into the decoder.
3. The method for extracting groove profile according to claim 1, characterized in that: The steps of feature fusion in the attention feature fusion module include: Linearly superimpose the features before downsampling in the encoder and the features after the first batch of processing in the decoder to obtain linear superposition features, and pool the linear superposition features to obtain pooling features; The linear superposition feature and the pooling feature are convolved, batch normalized, rectified linear unit, re-convolved, and re-batch normalized respectively, and the two results are linearly superimposed, activated and processed to fuse with the linear superposition feature to obtain the preliminary fusion feature; The preliminary fusion feature is weighted for the first time and fused with the feature before encoder downsampling to obtain the first feature. The preliminary fusion feature is weighted for the second time and fused with the feature after the first batch of processing of the decoder to obtain the second feature. The first feature and the second feature are linearly superimposed to obtain the fusion feature.
4. The method for extracting groove profile according to claim 2, characterized in that: The convolution is a dilated convolution operation.
5. The method for extracting groove profile according to claim 2, characterized in that: The fusion is multiplicative fusion.
6. The method for extracting groove profile according to claim 2, characterized in that: The pooling is global average pooling.
7. The method for extracting groove profile according to claim 3, characterized in that: The convolution and reconvolution are both point-by-point convolution operations.
8. The method for extracting groove profile according to claim 3, characterized in that: The fusion is multiplicative fusion.
9. The method for extracting groove profile according to claim 3, characterized in that: The pooling is global maximum pooling.
10. The method for extracting groove profile according to claim 3, characterized in that: The sum of the weight of the first weighting of the preliminary fusion feature and the weight of the second weighting is 1.