Multi-layer and multi-pass welding seam identification method and system for multi-source noise environment

Abstract

The invention discloses a multi-source noise environment-oriented multi-layer and multi-pass weld joint identification method and system, and the method comprises the steps: constructing a multi-layer and multi-pass weld joint data set, marking weld joint image data in the data set, and dividing the data according to a proportion; based on the LSEGN model, fusing a plurality of dual-channel attention modules; based on an LSEGN model with a dual-channel attention module, introducing a lightweight feature fusion module to obtain an improved LSEGN model; introducing a combined loss function; the improved LSEGN model is trained; optimizing the training result of the model to obtain a welding seam recognition model; testing the weld joint identification model; performing performance evaluation on the weld joint identification model; and performing weld joint identification by using the weld joint identification model. According to the method, under the interference of multi-source noise such as splashing, electric arc, impurities and fuzziness, accurate recognition of multiple layers of welding seams can be achieved, and more accurate welding seam feature information is provided for process optimization and welding seam tracking in the welding process.

Description

Technical Field

[0001] The invention belongs to the field of machine vision, and in particular relates to a multi-layer and multi-pass weld recognition method and system for a multi-source noise environment. Background Art

[0002] Since thick plate structures need to bear large loads and serve in complex environments, multi-layer and multi-pass welding processes are usually used to improve the connection strength and durability of the weld. However, the complexity of this welding method brings new technical challenges. The heat-affected zone of each weld and the remelting effect of the material may cause changes in the weld microstructure, making the weld more sensitive to external interference. In addition, in the actual welding process, due to the blurring effect caused by arc light, spatter, impurities and welding movement, the weld image is often severely interfered by noise, which leads to false detection, missed detection and edge blurring in weld recognition.

[0003] Weld seam recognition technology based on laser vision sensors provides an effective technical means to solve the above problems. This technology can extract the key feature points of the weld in real time and dynamically adjust the welding parameters to ensure the accuracy and stability of the welding process. This technology can not only effectively reduce the impact of traditional noise such as spatter and arc light, but also improve the recognition accuracy of welds in complex environments, especially in the recognition of multi-layer and multi-pass welds. Through precise feature point extraction and automated adjustment strategies, welding robots can cope with complex welding tasks, ensure the quality of weld formation, and improve welding efficiency and stability.

[0004] In existing research, a Chinese invention patent application with application number CN202410578927.7 discloses a robot weld recognition and tracking method based on deep learning. An end-to-end image denoising deep model is designed in this scheme. However, the end-to-end image denoising deep model can only be used to process noise sources such as arc light and metal vapor. Other existing technologies can only process a small number of noise sources, and the superposition effect of multi-source noise (such as impurities and fuzzy noise) is insufficient. There are still technical bottlenecks in welding tasks of complex welds and special workpieces. The interaction of multiple noise sources often leads to blurred weld edges, discontinuous contours, or even obscuration, which seriously affects the accuracy of weld recognition. Therefore, developing a weld recognition method that can effectively handle multi-source noise is a technical problem that needs to be solved urgently in the current welding field. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a multi-layer and multi-pass weld recognition method and system for a multi-source noise environment, which can realize accurate recognition of multi-layer and multi-pass welds under the interference of multi-source noise such as spatter, arc, impurities and blur.

[0006] Technical solution: A multi-layer and multi-pass weld recognition method for a multi-source noise environment of the present invention comprises:

[0007] The weld image data on the workpiece is acquired through an image acquisition system to form a multi-layer and multi-pass weld data set, and the weld image data in the multi-layer and multi-pass weld data set is marked and divided into a training set and a test set in proportion;

[0008] Based on the LSEGN model, multiple dual-channel attention modules are integrated to obtain the LSEGN model with dual-channel attention modules. The dual-channel attention module is used for adaptive weight update of the backbone network under the interference of splash and arc noise.

[0009] Based on the LSEGN model with dual-channel attention module, a lightweight feature fusion module is introduced to obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to enhance the deep mining of multi-level features;

[0010] The combined loss function of Dice Loss and Boundary Loss is introduced to deal with the problems of sample category imbalance and weld overfitting in weld classification;

[0011] The improved LSEGN model is trained using the training set; the Dice Loss and Boundary Loss combined loss function is used to optimize the model training process to obtain the weld recognition model; the weld recognition model is tested using the test set; and the performance of the weld recognition model is evaluated;

[0012] The tested weld seam recognition model is used to identify the weld seam during the welding process of the workpiece.

[0013] Furthermore, based on the LSEGN model, multiple dual-channel attention modules are integrated to obtain the LSEGN model with dual-channel attention modules, including:

[0014] A dual-channel attention module is respectively connected to the output ends of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 in the main part of the LSEGN model, and an updated weight model is obtained through GCP and LFP. The updated weight model is used as the LSEGN model with a dual-channel attention module.

[0015] Furthermore, a dual-channel attention module is connected to the output ends of Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 of the main part of the LSEGN model, respectively, and an updated weight model is obtained through GCP and LFP. The updated weight model has an LSEGN model with a dual-channel attention module, including:

[0016] The outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the backbone of the LSEGN model are used as the inputs x2, x3, x4, x5, and x6 of the dual-channel attention module, respectively, and GCP is used to obtain their global context information f 21 、f 31 、f 41 、f 51 、f 61 ;

[0017] The outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the backbone of the LSEGN model are used as the inputs x2, x3, x4, x5, and x6 of the dual-channel attention module, respectively, and LFP is used to obtain their local feature information f 22 、f 32 、f 42 、f 52 、f 62 ;

[0018] The global context information f 21 、f 31 、f 41 、f 51 、f 61 and local feature information f 22 、f 32 、f 42 、f 52 、f 62 Multiply them with the backbone network output x2, x3, x4, x5, x6 respectively and then add them together to obtain the updated weight models y2, y3, y4, y5, y6. The processing process is as follows:

[0019] y i =f i1 *x i +f i2 *x i ,i∈{2,3,4,5,6}.

[0020] Furthermore, GCP is used to obtain its global context information f 21 、f 31 、f 41 、f51 、f 61 , the processing process is as follows:

[0021] f i1 =Convup(Sigmoid(Relu(Conv(Avp(x i ))))),i∈{2,3,4,5,6}

[0022] In the formula, Avp represents the global average pooling operation; Relu and Sigmoid represent activation functions; Conv and Convup represent 3×3 convolution operations.

[0023] Furthermore, LFP is used to obtain its local feature information f 22 、f 32 、f 42 、f 52 、f 62 , the processing process is as follows:

[0024] f i2 =Convup(Softmax(Relu(Conv(Amp(x i ))))),i∈{2,3,4,5,6}

[0025] In the formula, Amp represents the global maximum pooling operation; Softmax is the activation function; Conv and Convup represent 3×3 convolution operations.

[0026] Furthermore, based on the LSEGN model with a dual-channel attention module, a lightweight feature fusion module is introduced to obtain an improved LSEGN model, including:

[0027] After the dual-channel attention modules at the output of Conv3-3, Conv4-3, and Conv5-3, a lightweight feature fusion module is connected respectively; the output y3, y4, and y5 of the dual-channel attention modules are used as the input x3, x4, and x5 of the lightweight feature modules, and are named S (3) , S (4) and S (5) The improved LSEGN model is finally obtained through the processing of downsampling, feature enhancement, upsampling and feature fusion, which can effectively extract clear weld detail features.

[0028] Furthermore, the outputs y3, y4, and y5 of the dual-channel attention module are used as the inputs x3, x4, and x5 of the lightweight feature module, and are named S (3) , S (4) and S (5)The improved LSEGN model is finally obtained through the process of downsampling, feature enhancement, upsampling and feature fusion, so as to effectively extract clear weld detail features, including:

[0029] The outputs of the dual-channel attention module, y3, y4, and y5, are used as the inputs of the lightweight feature module, denoted as x3, x4, and x5, and processed through two convolutional layers and normalization operations to extract preliminary features. The processing process is as follows:

[0030]

[0031] In the formula, They represent the first convolution output of the first layer x3, x4, and x5 before downsampling, respectively. Represents the second convolution output of the first layer x3, x4, x5 before downsampling; Conv1 and Conv2 represent 3×3 convolution operations; BN represents normalization; ReLU represents the activation function;

[0032] In order to further extract deep features, downsampling operation is performed. The processing process is as follows:

[0033]

[0034]

[0035] In the formula, S (3) , S (4) and S (5) The n corresponding to the side roads are 4, 3, and 2 respectively; Indicates downsampling S (i) The first convolution output of the jth layer on the side; Indicates downsampling S (i) The second convolution output of the jth layer on the side; MaxPool represents maximum pooling downsampling;

[0036] The downsampled feature map is further processed by global average pooling and gated fusion. The processing process is as follows:

[0037]

[0038] In the formula, f i 1 Indicates downsampling S (i) The gated convolution output of the side channel; f i 2 Indicates S (i) Global average pooling convolution output of the side channel; Conv represents 1×1 convolution operation;

[0039] In order to restore the spatial resolution of the feature map, an upsampling operation is used to process the output of the global feature and gate fusion. The processing process is as follows:

[0040]

[0041] In the formula, Indicates the upsampling S (i) The first convolution output of the jth layer on the side; Indicates the upsampling S (i) The second convolution output of the jth layer of the side path; Γ is the skip connection function;

[0042] The upsampled features are fused with the gated convolution output and the global average pooling convolution output to obtain the final feature map y i , the processing process is as follows:

[0043]

[0044] Where OutConv is the convolution operation.

[0045] Furthermore, the Dice Loss and Boundary Loss combined loss functions are used to optimize the training results of the improved LSEGN model to obtain the weld recognition model, including:

[0046] The combined loss function of Dice Loss and Boundary Loss is introduced. By calculating the binary classification loss, similarity loss and edge loss of the feature map generated by the improved LSEGN model and the label, the training results of the improved LSEGN model are optimized to obtain the weld recognition model.

[0047] Furthermore, the Dice Loss and Boundary Loss combined loss function is introduced. By calculating the binary classification loss, similarity loss and edge loss of the feature map generated by the improved LSEGN model and the label, the training results of the improved LSEGN model are optimized to obtain the weld recognition model, including:

[0048] Calculate the binary classification loss, similarity loss, and edge loss of the feature map generated by the improved LSEGN model and the label. The calculation formula is as follows:

[0049] L CE =-p i logy i -(1-p i )log(1-y i )

[0050]

[0051]

[0052] Loss = αL CE +βL DL +θL BD

[0053] Where, L CE represents the BCE loss function; L DL represents the Dice loss function; L BD represents the Boundary loss function; p i represents the predicted probability of the i-th pixel; y i represents the true label of the i-th pixel; Sigmoid represents the activation function; K x represents the Sobel X convolution kernel; K y represents the Sobel Y convolution kernel; conv2d represents the convolution operation; ρ represents 1e-6; MSE represents the mean square error; α, β, and θ represent weight coefficients.

[0054] Based on the same inventive concept, the present invention provides a multi-layer and multi-pass weld recognition system for a multi-source noise environment, comprising:

[0055] A data acquisition module is used to acquire weld image data on a workpiece through an image acquisition system to form a multi-layer multi-pass weld data set, and mark the weld image data in the multi-layer multi-pass weld data set and divide it into a training set and a test set in proportion;

[0056] A model building module is used to integrate multiple dual-channel attention modules based on the LSEGN model to obtain an LSEGN model with a dual-channel attention module; wherein the dual-channel attention module is used for adaptive weight update of the backbone network under the interference of splash and arc noise;

[0057] The model building module is used to introduce a lightweight feature fusion module based on the LSEGN model with a dual-channel attention module to obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to enhance the deep mining of multi-level features;

[0058] The loss function establishment module is used to introduce the combined loss function of Dice Loss and Boundary Loss. The combined loss function of Dice Loss and Boundary Loss is used to deal with the sample category imbalance and weld overfitting problems in weld classification;

[0059] The model training module is used to train the improved LSEGN model using the training set; optimize the model training process using the combined loss function of Dice Loss and Boundary Loss to obtain the weld recognition model; test the weld recognition model using the test set; and evaluate the performance of the weld recognition model;

[0060] The weld seam recognition module is used to perform weld seam recognition during the workpiece welding process using a tested weld seam recognition model.

[0061] Beneficial effects: Compared with the prior art, the significant technical effects of the present invention are as follows:

[0062] The dual-channel attention module proposed in this invention introduces an adaptive weight update mechanism to suppress the coverage effect caused by noise such as spatter and arc during welding from both the spatial and channel levels, thereby preventing the loss of weld information. This module can effectively enhance the ability to express weld features, ensure accurate capture of weld edges and details in complex noise environments, and reduce the probability of false detection caused by spatter and arc noise interference.

[0063] The lightweight feature fusion module proposed in this invention enhances the network's ability to capture weld details, global position and morphology by deeply mining and fusing multi-level feature information, while effectively suppressing the interference of impurities and fuzzy noise, avoiding problems such as feature loss, false detection and edge blur. While maintaining a low computational complexity, this module significantly improves the model's feature extraction effect, thereby ensuring the overall recognition accuracy of multi-layer and multi-pass welds.

[0064] The combined loss function proposed in the present invention, by introducing the combined loss function of Dice Loss and Boundary Loss, the model can simultaneously handle weld classification, sample data imbalance and edge overfitting problems under complex noise interference. DiceLoss effectively handles the problem of unbalanced ratio of positive and negative samples between welds and backgrounds, while Boundary Loss helps to strengthen the learning of edge information and accurately identify the true shape of welds, including the irregular subtle features of the weld surface, effectively reducing the risk of edge blur and inaccuracy. The loss function design significantly improves the classification ability and edge detail capture ability of the weld recognition model, making weld recognition more robust in complex noise environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flow chart of a multi-layer and multi-pass weld recognition method for a multi-source noise environment disclosed in an embodiment of the present invention;

[0066] Figure 2 It is a schematic diagram of a welding image acquisition system disclosed in an embodiment of the present invention;

[0067] Figure 3 is a principle structure diagram of a dual-channel attention module disclosed in an embodiment of the present invention;

[0068] Figure 4 is a test comparison diagram before and after adding the dual-channel attention module disclosed in the embodiment of the present invention;

[0069] Figure 5 It is a S disclosed in the embodiment of the present invention. (3) The principle structure diagram of the side lightweight feature fusion module;

[0070] Figure 6 It is a S disclosed in the embodiment of the present invention. (4) The principle structure diagram of the side lightweight feature fusion module;

[0071] Figure 7 It is a S disclosed in the embodiment of the present invention. (5) The principle structure diagram of the side lightweight feature fusion module;

[0072] Figure 8 This is a test comparison diagram before and after adding a lightweight feature fusion module disclosed in an embodiment of the present invention;

[0073] Fig. 9 It is a before-and-after comparison diagram after the combined loss function is introduced according to the embodiment of the present invention;

[0074] Fig.10 It is a schematic diagram of the overall network principle structure disclosed in the embodiment of the present invention;

[0075] Fig.11 is a prediction error graph of image feature points by the improved LSEGN model disclosed in an embodiment of the present invention;

[0076] Fig.12 It is a structural schematic diagram of a multi-layer and multi-pass weld recognition system for a multi-source noise environment disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It will be understood by those skilled in the art that the purposes and advantages that can be achieved with the present invention are not limited to the specific description of the above beneficial effects, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description.

[0078] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed in the present invention can be implemented in hardware, software or a combination of the two. Whether it is implemented in hardware or software depends on the specific application and design of the technical solution. Professional and technical personnel can use different methods to implement the functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0079] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0080] Example 1

[0081] See also Figure 1 , Figure 1 It is a flow chart of a multi-layer and multi-pass weld recognition method for a multi-source noise environment disclosed in an embodiment of the present invention. Figure 1 The multi-layer multi-pass weld seam identification method described is applied to the field of welding, such as weld seam identification when a thick plate structure is welded using a multi-layer multi-pass welding process. The embodiments of the present invention are not limited. Figure 1 As shown, the multi-layer and multi-pass weld recognition method for a multi-source noise environment may include the following operations:

[0082] S1. The weld image data on the workpiece is acquired through the image acquisition system to form a multi-layer multi-pass weld data set. The weld image data in the multi-layer multi-pass weld data set is marked and divided into a training set and a test set in proportion. The specific steps are as follows:

[0083] S1.1. The image acquisition system is used to acquire weld images of V-shaped welds of different thicknesses, different layers, and different noises on the workpiece. All acquired weld image data form a multi-layer and multi-pass weld data set.

[0084] In this embodiment, Figure 2 As shown, the image acquisition system includes a laser vision sensor 1 for scanning the weld. The laser vision sensor 1 is mounted on a robotic arm 2, and then the data set is collected and processed through an industrial computer 3.

[0085] S1.2. Label the weld image data in the multi-layer and multi-pass weld dataset to obtain the true label, and then divide the image data into a training set and a test set according to a 6:4 ratio.

[0086] S2. Based on the LSEGN (laser edge-guided neural network) model, multiple dual-channel attention modules are integrated to obtain the LSEGN model with a dual-channel attention module; among them, the dual-channel attention module is used for adaptive weight update of the backbone network under the interference of splash and arc noise.

[0087] The dual-channel attention module is used to adaptively update the weights of the backbone network under the interference of spatter and arc noise, and to suppress the coverage effect of spatter and arc noise on the weld.

[0088] Under the interference of spatter and arc noise, although the LSEGN model can capture the main contour of the weld more accurately, there is still a problem of false detection. When spatter is attached to the weld, it is difficult for the model to effectively distinguish the weld from spatter, resulting in the inability to accurately extract the true contour of the weld; and the strong light radiation brought by the arc noise makes it impossible for the model to clearly distinguish the weld from the background, and there is a situation where the noise is misjudged as part of the weld. The root of this problem is that the backbone network of the LSEGN model is based on the VGGNET architecture. Although VGGNET has a certain feature extraction capability, it lacks an adaptive mechanism for weight updating and fails to effectively constrain the weights of the interference features, that is, it also gives improper weights to the false weld features, so it cannot well suppress the influence of spatter and arc features, and then misjudges the false weld as a real weld, which ultimately leads to the model's weak robustness under these noise interferences, and poor ability to effectively separate noise and welds. To this end, the present invention designs a dual-channel attention module.

[0089] In this embodiment, based on the LSEGN model, multiple dual-channel attention modules are integrated to obtain an LSEGN model with a dual-channel attention module, which specifically includes the following:

[0090] like Fig.10 As shown in Figure 2, a dual-channel attention module is connected to the output ends of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 in the main part of the LSEGN model, respectively. Figure 3 As shown in the figure, the outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 in the main part of the LSEGN model are used as the input of the dual-channel attention module, and the updated weight model is obtained through GCP (Global Context Path) and LFP (Local Feature Path). The updated weight model is used as the LSEGN model with a dual-channel attention module. The details are as follows:

[0091] S2.1. For the outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the backbone of the LSEGN model, they are used as the inputs x2, x3, x4, x5, and x6 of the dual-channel attention module, respectively, and their global context information f is obtained using GCP. 21 、f 31 、f 41 、f 51 、f 61 , the processing process is as follows:

[0092] f i1 =Convup(Sigmoid(Relu(Conv(Avp(x i ))))),i∈{2,3,4,5,6}

[0093] In the formula, Avp represents the global average pooling operation; Relu and Sigmoid represent activation functions; Conv and Convup represent 3×3 convolution operations;

[0094] S2.2. For the outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the backbone of the LSEGN model, they are used as the inputs x2, x3, x4, x5, and x6 of the dual-channel attention module, respectively, and LFP is used to obtain their local feature information f 22 、f 32 、f 42 、f 52 、f 62 , the processing process is as follows:

[0095] f i2 =Convup(Softmax(Relu(Conv(Amp(x i ))))),i∈{2,3,4,5,6}

[0096] Where Amp represents the global maximum pooling operation; Softmax is the activation function.

[0097] S2.3, the global context information f 21 、f 31 、f 41 、f 51 、f 61 and local feature information f 22 、f 32 、f 42 、f 52 、f 62Multiply them with the backbone network output x2, x3, x4, x5, x6 respectively and then add them together to obtain the updated weight models y2, y3, y4, y5, y6. The processing process is as follows:

[0098] y i =f i1 *x i +f i2 *x i ,i∈{2,3,4,5,6}.

[0099] By adding a dual-channel attention module after the output of different layers of VGGNET, fine-grained attention and weighting can be achieved on multiple feature levels, thereby comprehensively utilizing the feature information of each layer, enabling the model to more effectively capture details and contextual information when processing complex weld images, thereby improving weld detection capabilities. Figure 4 As shown, Figure 4 Figure (a) is the original image. Figure 4 Figure (b) shows the test results before adding. Figure 4 Figure (c) shows the test results after adding the LSEGN model. The LSEGN model with dual-channel attention module reduces the probability of false detection caused by spatter and arc interference, thereby obtaining accurate weld laser stripes.

[0100] S3. Based on the LSEGN model with a dual-channel attention module, a lightweight feature fusion module is introduced to obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to enhance the deep mining of multi-level features.

[0101] The lightweight feature fusion module is used to strengthen the deep mining of multi-level features and enhance the network's ability to capture subtle weld features under the interference of impurities and fuzzy noise.

[0102] When dealing with the interference of impurities and fuzzy noise, the LSEGN model exposes problems such as feature loss, false detection, and unclear edges. Due to the occlusion of the weld contour by impurities, it is difficult for the model to effectively detect the occluded part, resulting in missed detection in edge extraction; and the blurring effect of the weld edge greatly reduces the image clarity, making it difficult for the model to accurately capture the fine structure of the weld, and the boundary between the weld and the background becomes blurred, ultimately causing misjudgment and unclear edges. These shortcomings stem from the limitations of the model's multi-scale feature extraction module, whose design mainly relies on three layers of convolution for feature extraction, and the depth of the convolution layer is limited, resulting in insufficient expression of the model at the feature level. In the face of impurities and fuzzy interference, the image clarity is poor and the features are not obvious, making it difficult for the model to extract deeper feature information. In addition, in terms of feature fusion, the model only upsamples the deep global information in the backbone network and then simply fuses it with the shallow information, failing to fully explore the depth and adaptive characteristics of each layer of information, ignoring the local information in each layer of features, making it impossible to effectively capture subtle feature differences, thereby affecting the recognition effect of the model. To this end, the present invention designs a lightweight feature fusion module.

[0103] In this embodiment, based on the LSEGN model with a dual-channel attention module, a lightweight feature fusion module is introduced to obtain an improved LSEGN model, which is as follows:

[0104] like Fig.10 As shown in Figure 1, a lightweight feature fusion module is connected after the dual-channel attention module at the output of Conv3-3, Conv4-3, and Conv5-3. The outputs y3, y4, and y5 of the dual-channel attention module are used as the inputs x3, x4, and x5 of the lightweight feature module, and are named S (3) , S (4) and S (5) Side road, corresponding Figure 5 , Figure 6 , Figure 7 , and through the process of downsampling, feature enhancement, upsampling and feature fusion, the improved LSEGN model is finally obtained, so as to effectively extract clear weld detail features. The specific steps are as follows:

[0105] S3.1. The outputs of the dual-channel attention module, y3, y4, and y5, are used as the inputs of the lightweight feature module, denoted as x3, x4, and x5. They are processed through two convolutional layers and normalization operations to extract preliminary features. The processing process is as follows:

[0106]

[0107] In the formula, They represent the first convolution outputs of the first layer x3, x4, and x5 before downsampling respectively; Represents the second convolution output of the first layer x3, x4, x5 before downsampling; Conv1 and Conv2 represent 3×3 convolution operations; BN represents normalization; ReLU represents the activation function.

[0108] S3.2. In order to further extract deep features, downsampling operation is performed. The processing process is as follows:

[0109]

[0110] In the formula, S (3) , S (4) and S (5) The n corresponding to the side roads are 4, 3, and 2 respectively; Indicates downsampling S (i) The first convolution output of the jth layer on the side; Indicates downsampling S (i) The second convolution output of the jth layer on the side; MaxPool represents maximum pooling downsampling.

[0111] S3.3, the downsampled feature map is further processed by global average pooling and gated fusion. The processing process is as follows:

[0112]

[0113] In the formula, f i 1 Indicates downsampling S (i) The gated convolution output of the side channel; f i 2 Indicates S (i) Global average pooling convolution output of the side path; Conv represents a 1×1 convolution operation.

[0114] S3.4. In order to restore the spatial resolution of the feature map, an upsampling operation is used to process the output of the global feature and gate fusion. The processing process is as follows:

[0115]

[0116] Indicates the upsampling S (i) The first convolution output of the jth layer on the side; Indicates the upsampling S (i) The second convolution output of the jth layer of the side path; Γ is the skip connection function;

[0117] The upsampled features are fused with the gated convolution output and the global average pooling convolution output to obtain the final feature map y i , the processing process is as follows:

[0118]

[0119] Where OutConv is the convolution operation.

[0120] Through S (3) , S (4) and S (5) Lightweight feature fusion modules of different depths are applied to downsample the feature map to a higher number of channels, thereby improving the detailed expression of features in high-dimensional space; the multiple jump connection mechanism introduced in the upsampling process ensures the full integration of features of different scales, thereby effectively restoring the problem of weld detail loss caused by blur; to further improve the robustness to impurity noise, global features and local features are fused at the bottom layer of the network, which improves the robustness of the model under the interference of impurity noise. The lightweight feature fusion module not only maintains the integrity of the global structure, but also captures the local features in detail, effectively reducing the interference of impurities and fuzzy noise on weld recognition, such as Figure 8 As shown, Figure 8 Figure (a) is the original image. Figure 8 Figure (b) shows the test results before adding. Figure 8 Figure (c) shows the test results after adding the impurities. Due to the occlusion of the weld contour by impurities and the blurring effect of the weld and background boundary, the weld feature information becomes weaker and the image clarity decreases, which eventually leads to feature loss, false detection, and unclear edges in weld detection. The LSEGN module after adding the lightweight feature fusion module significantly alleviates such problems and improves the accuracy of weld detection.

[0121] S4. The combined loss function of Dice Loss and Boundary Loss is introduced. The combined loss function of Dice Loss and Boundary Loss is used to deal with the problems of sample category imbalance and weld overfitting in weld classification.

[0122] The combined loss function of Dice Loss and Boundary Loss is designed to address the problem of sample category imbalance in weld classification, enhance the model's ability to accurately capture weld edges under complex noise conditions, and effectively prevent edge overfitting.

[0123] The LSEGN model uses the Binary Cross-Entropy (BCE) loss function. The main advantage of BCE is that it is simple to calculate and performs well in binary classification problems. It can effectively measure the difference between the predicted results and the true labels, thereby driving the model to achieve higher classification accuracy in most cases. However, due to the small proportion of the weld in the image, the number of positive samples is far less than the negative samples, resulting in a significant class imbalance problem in the data set. This imbalance makes the model more inclined to learn negative samples during training, thereby ignoring the details of the weld area. In addition, due to the fact that the weld edge is usually blurred and the interference of complex background, the model shows certain limitations in edge detection. The generated weld edge image is often too smooth and it is difficult to accurately reflect the true complexity of the weld surface. In view of this, the present invention introduces Dice Loss and Boundary Loss on the basis of BCE Loss. Dice Loss is a loss function that measures the similarity between the predicted results and the actual results. By directly optimizing the performance of the model on small targets, it can effectively reduce the negative impact caused by sample imbalance. Especially when processing small targets such as welds, it shows excellent results. BoundaryLoss focuses on optimizing edge details. Through refined edge detection, the model can more accurately identify the true shape of the weld, including the irregular subtle features of the weld surface, significantly improving the detection accuracy of the weld edge and effectively reducing the risk of edge blur and inaccuracy.

[0124] S5. Use the training set to train the improved LSEGN model; use the Dice Loss and Boundary Loss combined loss function to optimize the model training process to obtain the weld recognition model; use the test set to test the weld recognition model; and evaluate the performance of the weld recognition model. The details are as follows:

[0125] S5.1. Use the training set to train the improved LSEGN model.

[0126] like Fig.10 As shown, according to the improved LSEGN model, the experimental environment and parameter configuration are carried out, as shown in Table 1;

[0127] Table 1:

[0128]

[0129] S5.2. Use the Dice Loss and Boundary Loss combined loss function to optimize the training process and obtain the weld recognition model; use the test set to test the weld recognition model.

[0130] In this embodiment, the Dice Loss and Boundary Loss combined loss functions are used to optimize the model training process to obtain a weld recognition model, including:

[0131] The Dice Loss and Boundary Loss combined loss function is introduced. By calculating the binary classification loss, similarity loss, and edge loss of the feature map generated by the improved LSEGN model and the label, the training results of the improved LSEGN model are optimized to obtain the weld recognition model. The calculation formula of the loss function is as follows:

[0132] L CE =-p i logy i -(1-p i )log(1-y i )

[0133]

[0134] Loss = αL CE +βL DL +θL BD

[0135] Where, L CE represents the BCE loss function; L DL represents the Dice loss function; L BD represents the Boundary loss function; p i represents the predicted probability of the i-th pixel; y i represents the true label of the i-th pixel (0 or 1); Sigmoid represents the activation function; K x represents the Sobel X convolution kernel; K y represents the Sobel Y convolution kernel; conv2d represents the convolution operation; ρ represents 1e-6; MSE represents the mean square error; α, β, and θ represent weight coefficients.

[0136] By introducing a combination of Boundary Loss and Dice Loss into the loss function, a more accurate fit can be achieved on edge details and overall structure, thereby comprehensively improving the edge detection capability of the model.

[0137] Boundary Loss focuses on gradient information, allowing the model to better capture the details of the weld edge; while Dice Loss ensures accurate recognition of the overall area. This combined loss function can effectively improve the clarity and accuracy of edge details in weld images. Fig. 9 As shown, Fig. 9 Figure (a) is the original image. Fig. 9 Figure (b) shows the test results before adding. Fig. 9 Figure (c) shows the test results after adding the combined loss function. It can be seen that the introduction of the combined loss function significantly improves the detail performance of the weld edge and avoids the edge blur problem caused by noise.

[0138] S5.3. Evaluate the performance of the weld identification model.

[0139] In this embodiment, when evaluating the performance of the weld recognition model, the root mean square error (RMSE), peak signal-to-noise ratio (PSNR), structural similarity (SSIM), category mean pixel accuracy (mPA), category mean intersection over union ratio (mIoU), edge overlap rate (EOR) and feature point prediction error are used as evaluation indicators to verify the accuracy of the weld detection model.

[0140] In order to verify the multi-layer and multi-pass weld recognition method for multi-source noise environment in the present invention, the constructed weld dataset was used for testing, and RMSE, PSNR, SSIM, mPA, mIoU and EOR were used as evaluation indicators. The test results were compared with those of the U-Net, R3Net, ICON and LSEGN models. The comparative test results are shown in Table 2.

[0141] Table 2:

[0142]

[0143]

[0144] As can be seen from Table 2, in terms of RMSE index, the score of the model of the present invention is 0.048, which is the lowest among all the comparison models, indicating that the error between its prediction result and the annotation image is the smallest and the accuracy is the highest. In terms of PSNR index, the score of the model of the present invention is 26.427, which is better than U-Net, R3Net, ICON and LSEGN models, showing its strong expressiveness in image quality. The mPA of the model of the present invention is 0.966, which is significantly higher than U-Net (0.948) and R3Net (0.942), and only slightly lower than LSEGN by 1.1%. This is because the black background in the weld image collected in the actual scene occupies most of the image area, causing the problem of sample imbalance, and the Dice Loss loss function introduced by the model of the present invention reduces the impact of this problem, so the value of mPA is slightly lower than LSEGN. In terms of mIoU index, the model of the present invention reaches 0.925, which is about 0.8% higher than the second highest ICON model (0.918), which shows that the model of the present invention has higher accuracy in pixel classification and prediction in the weld area. The SSIM value of the model of the present invention is 0.984, which is the highest among all models, indicating that it can better retain the structural information of the image, and the quality of the generated weld image is closer to the real annotation image. In terms of the EOR evaluation index, the model of the present invention reached 0.723, which is significantly higher than all other models, especially R3Net, with an improvement of 64.3%. This shows that the model of the present invention has significant advantages in weld edge detection and can capture edge information more accurately. Compared with other models, the model of the present invention can better eliminate background noise and identify the subtle structural details of the weld, so that the generated weld edge is closer to the actual situation, which proves the effectiveness and robustness of the model of the present invention in the weld identification task. As can be seen from the above, the method of the present invention is more suitable for weld identification under multi-source noise interference.

[0145] S6. Use the tested weld recognition model to perform weld recognition during the workpiece welding process.

[0146] In order to verify the effectiveness of the structure proposed in this paper, a series of ablation experiments were carried out on the constructed weld dataset. The experiment used the LSEGN model as the basic model, and added the dual-channel attention (DCA) module, the lightweight feature fusion (LFF) module and the combined loss function (Combined Loss) in turn for comparison. The experimental results are shown in Table III.

[0147] Table 3:

[0148]

[0149] It can be concluded from Table 3 that the LSEGN1 model after the introduction of the DCA module has a 3.17% reduction in RMSE, a 0.99% increase in PSNR, a 3.09% increase in mIoU, and a 7.76% increase in EOR compared to the LSEGN model. The results show that the DCA module has a significant improvement in improving the interference of the model to weld noise. On the basis of LSEGN1, the present invention further introduces LFF to improve the feature fusion capability of the model. After adding the LFF module, the LSEGN2 model has achieved more significant improvements in various indicators: RMSE is reduced by 18.03%, PSNR is increased by 6.85%, mIoU is increased by 5.19%, and EOR is increased by 27.43%. This shows that the LFF module shows beneficial effects in the feature fusion and multi-scale information processing of the model. Although the LSEGN 2 The performance of the model has been significantly improved, but there is still room for improvement in dealing with sample imbalance and edge complexity. To this end, the present invention introduces a combined loss function to further optimize the classification ability and edge detection effect of the model. After adding the combined loss function, the model of the present invention performs best: RMSE is reduced by 4.00%, PSNR is improved by 1.23%, mIoU is improved by 1.43%, and EOR is increased by 10.38%. Finally, compared with the LSEGN model, the model of the present invention improves RMSE, PSNR, mIoU, SSIM and EOR by 23.8%, 9.2%, 9.9%, 1.0%, 51.5% respectively, and decreases by 1.1% in mPA. In summary, by gradually introducing the DCA module, LFF module and combined loss function, the performance of the LSEGN model in feature extraction, multi-scale fusion, and loss calculation has been significantly improved. Each improved module shows significant effects in solving different problems of the model, so that the final improved model performs better than LSEGN in the weld recognition task. It can be seen from the above that the model of the present invention effectively solves the problem of feature loss, false detection and missed detection in the LSEGN model when identifying welds under multi-source noise interference.

[0150] In order to more intuitively reflect the accuracy of the model of the present invention in predicting weld feature points, the present invention extracts weld feature points, and the error is as follows Fig.11 The results show that the prediction deviation of the model of the present invention for most images is within 4 pixels, corresponding to an actual deviation of less than 0.2 mm, which meets the requirements of weld tracking.

[0151] Example 2

[0152] See also Fig.12 , Fig.12The structure diagram of a multi-layer multi-pass weld recognition system for a multi-source noise environment disclosed in an embodiment of the present invention can realize weld recognition when a thick plate structure is welded using a multi-layer multi-pass welding process, and specifically includes:

[0153] A data acquisition module is used to acquire weld image data on a workpiece through an image acquisition system to form a multi-layer multi-pass weld data set, and mark the weld image data in the multi-layer multi-pass weld data set and divide it into a training set and a test set in proportion;

[0154] A model building module is used to integrate multiple dual-channel attention modules based on the LSEGN model to obtain an LSEGN model with a dual-channel attention module; wherein the dual-channel attention module is used for adaptive weight update of the backbone network under the interference of splash and arc noise;

[0155] The model building module is used to introduce a lightweight feature fusion module based on the LSEGN model with a dual-channel attention module to obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to enhance the deep mining of multi-level features;

[0156] The loss function establishment module is used to introduce the combined loss function of Dice Loss and Boundary Loss. The combined loss function of Dice Loss and Boundary Loss is used to deal with the sample category imbalance and weld overfitting problems in weld classification;

[0157] The model training module is used to train the improved LSEGN model using the training set; optimize the model training process using the combined loss function of Dice Loss and Boundary Loss to obtain the weld recognition model; test the weld recognition model using the test set; and evaluate the performance of the weld recognition model;

[0158] The weld seam identification module is used to perform weld seam identification using a tested weld seam identification model.

[0159] In an optional embodiment, a multi-layer and multi-pass weld recognition method for a multi-source noise environment includes: a) acquiring weld image data on a workpiece to form a multi-layer and multi-pass weld data set, annotating the weld image data in the multi-layer and multi-pass weld data set and dividing it into a training set and a test set in proportion; b) based on the LSEGN model, fusing multiple dual-channel attention modules to obtain an LSEGN model with a dual-channel attention module; c) based on the LSEGN model with a dual-channel attention module, introducing a lightweight feature fusion module to obtain an improved LSEGN model; d) introducing a combined loss function of Dice Loss and Boundary Loss; d) using the training set to train the improved LSEGN model; using the combined loss function of Dice Loss and Boundary Loss to optimize the training process of the model to obtain a weld recognition model; using the test set to test the weld recognition model; and performing performance evaluation on the weld recognition model; e) using the tested weld recognition model to perform weld recognition.

Claims

1. A multi-layer and multi-pass weld recognition method for a multi-source noise environment, characterized in that: include: The weld image data on the workpiece is acquired through an image acquisition system to form a multi-layer and multi-pass weld data set, and the weld image data in the multi-layer and multi-pass weld data set is annotated and then divided into a training set and a test set in proportion; Based on the LSEGN model, multiple dual-channel attention modules are integrated to obtain the LSEGN model with dual-channel attention modules. The dual-channel attention module is used for adaptive weight update of the backbone network under the interference of splash and arc noise. Based on the LSEGN model with dual-channel attention module, a lightweight feature fusion module is introduced to obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to enhance the deep mining of multi-level features; The combined loss function of Dice Loss and Boundary Loss is introduced to deal with the problems of sample category imbalance and weld overfitting in weld classification; The improved LSEGN model is trained using the training set; the Dice Loss and Boundary Loss combined loss function is used to optimize the model training process to obtain the weld recognition model; the weld recognition model is tested using the test set; and the performance of the weld recognition model is evaluated; The tested weld seam recognition model is used to identify the weld seam during the welding process of the workpiece.

2. The multi-layer and multi-pass weld identification method for a multi-source noise environment according to claim 1 is characterized in that: Based on the LSEGN model, multiple dual-channel attention modules are integrated to obtain the LSEGN model with dual-channel attention modules, including: A dual-channel attention module is respectively connected to the output ends of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 in the main part of the LSEGN model, and an updated weight model is obtained through GCP and LFP. The updated weight model is used as the LSEGN model with a dual-channel attention module.

3. The multi-layer and multi-pass weld identification method for a multi-source noise environment according to claim 2 is characterized in that: A dual-channel attention module is connected to the output ends of Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 of the main part of the LSEGN model, and an updated weight model is obtained through GCP and LFP. The updated weight model has an LSEGN model with a dual-channel attention module, including: The outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the backbone of the LSEGN model are used as the inputs x2, x3, x4, x5, and x6 of the dual-channel attention module, respectively, and GCP is used to obtain their global context information f 21 、f 31 、f 41 、f 51 、f 61 ; The outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the backbone of the LSEGN model are used as the inputs x2, x3, x4, x5, and x6 of the dual-channel attention module, respectively, and LFP is used to obtain their local feature information f 22 、f 32 、f 42 、f 52 、f 62 ; The global context information f 21 、f 31 、f 41 、f 51 、f 61 and local feature information f 22 、f 32 、f 42 、f 52 、f 62 Multiply them with the backbone network output x2, x3, x4, x5, x6 respectively and then add them together to obtain the updated weight models y2, y3, y4, y5, y6. The processing process is as follows: y i =f i1 *x i +f i2 *x i ,i∈{2,3,4,5,6}。 4. The multi-layer and multi-pass weld identification method for a multi-source noise environment according to claim 3 is characterized in that: Use GCP to get its global context information f 21 、f 31 、f 41 、f 51 、f 61 , the processing process is as follows: f i1 =Convup(Sigmoid(Relu(Conv(Avp(x i ))))),i∈{2,3,4,5,6} In the formula, Avp represents the global average pooling operation; Relu and Sigmoid represent activation functions; Conv and Convup represent 3×3 convolution operations.

5. The multi-layer and multi-pass weld identification method for a multi-source noise environment according to claim 3 is characterized in that: Use LFP to obtain its local feature information f 22 、f 32 、f 42 、f 52 、f 62 , the processing process is as follows: f i2 =Convup(Softmax(Relu(Conv(Amp(x i ))))),i∈{2,3,4,5,6} In the formula, Amp represents the global maximum pooling operation; Softmax is the activation function; Conv and Convup represent 3×3 convolution operations.

6. The multi-layer and multi-pass weld identification method for a multi-source noise environment according to claim 2 is characterized in that: Based on the LSEGN model with a dual-channel attention module, a lightweight feature fusion module is introduced to obtain an improved LSEGN model, including: After the dual-channel attention modules at the output of Conv3-3, Conv4-3, and Conv5-3, a lightweight feature fusion module is connected respectively; the output y3, y4, and y5 of the dual-channel attention modules are used as the input x3, x4, and x5 of the lightweight feature modules, and are named S (3) , S (4) and S (5) The improved LSEGN model is finally obtained through the processing of downsampling, feature enhancement, upsampling and feature fusion, which can effectively extract clear weld detail features.

7. The multi-layer and multi-pass weld identification method for a multi-source noise environment according to claim 6 is characterized in that: The outputs y3, y4, and y5 of the dual-channel attention module are used as the inputs x3, x4, and x5 of the lightweight feature module, and are named S (3) , S (4) and S (5) The improved LSEGN model is finally obtained through the process of downsampling, feature enhancement, upsampling and feature fusion, so as to effectively extract clear weld detail features, including: The outputs of the dual-channel attention module, y3, y4, and y5, are used as the inputs of the lightweight feature module, denoted as x3, x4, and x5, and processed through two convolutional layers and normalization operations to extract preliminary features. The processing process is as follows: In the formula, They represent the first convolution outputs of the first layer x3, x4, and x5 before downsampling respectively; Represents the second convolution output of the first layer x3, x4, x5 before downsampling; Conv1 and Conv2 represent 3×3 convolution operations; BN represents normalization; ReLU represents the activation function; In order to further extract deep features, downsampling operation is performed. The processing process is as follows: In the formula, S (3) , S (4) and S (5) The n corresponding to the side roads are 4, 3, and 2 respectively; Indicates downsampling S (i) The first convolution output of the jth layer on the side; Indicates downsampling S (i) The second convolution output of the jth layer on the side; MaxPool represents maximum pooling downsampling; The downsampled feature map is further processed by global average pooling and gated fusion. The processing process is as follows: In the formula, f i 1 Indicates downsampling S (i) The gated convolution output of the side channel; f i 2 Indicates S (i) The global average pooling convolution output of the side channel; Conv represents a 1×1 convolution operation; In order to restore the spatial resolution of the feature map, an upsampling operation is used to process the output of the global feature and gate fusion. The processing process is as follows: In the formula, Indicates the upsampling S (i) The first convolution output of the jth layer on the side; Indicates the upsampling S (i) The second convolution output of the jth layer of the side path; Γ is the skip connection function; The upsampled features are fused with the gated convolution output and the global average pooling convolution output to obtain the final feature map y i , the processing process is as follows: Where OutConv is the convolution operation.

8. The multi-layer and multi-pass weld identification method for a multi-source noise environment according to claim 1 is characterized in that: The Dice Loss and Boundary Loss combined loss functions are used to optimize the training results of the improved LSEGN model to obtain the weld recognition model, including: The combined loss function of Dice Loss and Boundary Loss is introduced. By calculating the binary classification loss, similarity loss and edge loss of the feature map generated by the improved LSEGN model and the label, the training results of the improved LSEGN model are optimized to obtain the weld recognition model.

9. The multi-layer and multi-pass weld identification method for a multi-source noise environment according to claim 8, characterized in that: The Dice Loss and Boundary Loss combined loss function is introduced. By calculating the binary classification loss, similarity loss and edge loss of the feature map generated by the improved LSEGN model and the label, the training results of the improved LSEGN model are optimized to obtain the weld recognition model, including: Calculate the binary classification loss, similarity loss, and edge loss of the feature map generated by the improved LSEGN model and the label. The calculation formula is as follows: L CE =-p i logy i -(1-p i )log(1-y i ) Loss=αL CE +βL DL +θL BD Where, L CE represents the BCE loss function; L DL represents the Dice loss function; L BD represents the Boundary loss function; p i represents the predicted probability of the i-th pixel; y i represents the true label of the i-th pixel; Sigmoid represents the activation function; K x represents the Sobel X convolution kernel; K y represents the Sobel Y convolution kernel; conv2d represents the convolution operation; ρ represents 1e-6; MSE represents the mean square error; α, β, and θ represent weight coefficients.

10. A multi-layer and multi-pass weld recognition system for a multi-source noise environment, characterized in that: include: A data acquisition module is used to acquire weld image data on a workpiece through an image acquisition system to form a multi-layer multi-pass weld data set, and mark the weld image data in the multi-layer multi-pass weld data set and divide it into a training set and a test set in proportion; A model building module is used to integrate multiple dual-channel attention modules based on the LSEGN model to obtain an LSEGN model with a dual-channel attention module; wherein the dual-channel attention module is used for adaptive weight update of the backbone network under the interference of splash and arc noise; The model building module is used to introduce a lightweight feature fusion module based on the LSEGN model with a dual-channel attention module to obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to enhance the deep mining of multi-level features; The loss function establishment module is used to introduce the combined loss function of Dice Loss and Boundary Loss. The combined loss function of Dice Loss and Boundary Loss is used to deal with the sample category imbalance and weld overfitting problems in weld classification; The model training module is used to train the improved LSEGN model using the training set; the Dice Loss and Boundary Loss combined loss function is used to optimize the model training process to obtain the weld recognition model; Use the test set to test the weld recognition model; evaluate the performance of the weld recognition model; The weld seam recognition module is used to perform weld seam recognition during the workpiece welding process using a tested weld seam recognition model.

Images