A multi-layer multi-pass weld seam recognition method and system for a multi-source noise environment

Abstract

The application discloses a kind of multi-layer multi-pass weld seam identification method and system for multi-source noise environment, the method includes: constructing multi-layer multi-pass weld seam dataset, after marking weld seam image data in dataset, proportion is divided;With LSEGN model as foundation, fusion multiple double-channel attention module;With the LSEGN model with double-channel attention module as foundation, introduce light feature fusion module, obtain improved LSEGN model;Introduce combination loss function;Improved LSEGN model is trained;Optimize the training result of model, obtain weld seam identification model;Weld seam identification model is tested;Weld seam identification model is evaluated;Weld seam identification is carried out using weld seam identification model.The method of the application can realize accurate identification of multi-layer multi-pass weld seam under the interference of multiple sources of noise such as spatter, arc, impurities and blur, provide more accurate weld seam feature information for process optimization and weld seam tracking in the welding process.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of machine vision, and particularly relates to a multi-layer multi-pass weld joint recognition method and system for a multi-source noise environment. BACKGROUND

[0002] Due to the need to bear large loads and serve in complex environments, thick plate structures usually adopt a multi-layer multi-pass welding process to improve the connection strength and durability of the weld joint. However, the complexity of this welding method brings new technical challenges. The heat-affected zone of each layer of weld pass and the remelting effect of the material can cause changes in the microstructure of the weld joint, making the weld joint more sensitive to external interference. In addition, in the actual welding process, due to the arc light, spatter, impurities and blur effects caused by welding movement, the weld joint image is often severely disturbed by noise, which in turn leads to problems such as false detection, missed detection and edge blurring in weld joint recognition.

[0003] Weld joint recognition technology based on laser vision sensors provides an effective technical means to solve the above problems. This technology can extract key feature points of the weld joint in real time and dynamically adjust welding parameters to ensure the accuracy and stability of the welding process. This technology not only effectively reduces the impact of traditional noise such as spatter and arc light, but also improves the recognition accuracy of weld joints in complex environments, especially in multi-layer multi-pass weld joint recognition. Through precise feature point extraction and automated adjustment strategies, welding robots can handle complex welding tasks, ensure weld joint forming quality, and improve welding efficiency and stability.

[0004] In existing research, a Chinese invention patent application with application number CN202410578927.7 discloses a robot weld joint recognition and tracking method based on deep learning. In this scheme, an end-to-end image denoising deep model is designed. However, the end-to-end image denoising deep model can only be used to process noise sources such as arc light and metal vapor. Existing other technologies can only handle a small amount of noise sources, and are insufficient in handling the superposition effect of multiple noise sources (such as impurities and blur noise), especially in complex weld joints and special workpiece welding tasks. The interaction of multiple noise sources often leads to blurred weld joint edges, discontinuous contours, and even being covered, which seriously affects the weld joint recognition accuracy. Therefore, developing a weld joint recognition method that can effectively handle multiple noise sources is a technical problem that needs to be solved in the current welding field. SUMMARY

[0005] The purpose of the present application is to provide a multi-layer multi-pass weld joint recognition method and system for a multi-source noise environment, which can accurately recognize multi-layer multi-pass weld joints under the interference of multiple noise sources such as spatter, arc, impurities and blur.

[0006] Technical solution: A multi-layer multi-pass weld seam identification method for a multi-source noise environment, comprising:

[0007] Obtain the weld seam image data on the workpiece through the image acquisition system, form a multi-layer multi-pass weld seam data set, and divide the weld seam image data in the multi-layer multi-pass weld seam data set into a training set and a test set after marking.

[0008] Based on the LSEGN model, a plurality of double-channel attention modules are fused to obtain an LSEGN model with a double-channel attention module, wherein the double-channel attention module is used for adaptive updating of the weight of the backbone network under the interference of spatter and arc noise.

[0009] Based on the LSEGN model with a double-channel attention module, a lightweight feature fusion module is introduced to obtain an improved LSEGN model, wherein the lightweight feature fusion module is used to strengthen the deep mining of multi-level features.

[0010] A Dice Loss and Boundary Loss combined loss function is introduced, which is used to handle the sample class imbalance and weld seam overfitting problem in weld seam classification.

[0011] The improved LSEGN model is trained using the training set, the Dice Loss and Boundary Loss combined loss function is used to optimize the training process of the model, and a weld seam identification model is obtained.

[0012] The tested weld seam identification model is used for weld seam identification in the workpiece welding process.

[0013] Further, based on the LSEGN model, a plurality of double-channel attention modules are fused to obtain an LSEGN model with a double-channel attention module, comprising:

[0014] A double-channel attention module is connected to the output end of Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 of the LSEGN model backbone part, and an updated weight model is obtained through GCP and LFP, which is used as the LSEGN model with a double-channel attention module.

[0015] Furthermore, a dual-channel attention module is connected to the outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 in the backbone of the LSEGN model, respectively. An updated weight model is obtained through GCP and LFP. This updated weight model is an LSEGN model with dual-channel attention modules, including:

[0016] The outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the backbone of the LSEGN model are used as inputs x2, x3, x4, x5, and x6 to the dual-channel attention module, respectively, and their global context information f is obtained using GCP. 21 f 31 f 41 f 51 f 61 ;

[0017] The outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the backbone of the LSEGN model are used as inputs x2, x3, x4, x5, and x6 to the dual-channel attention module, respectively, and LFP is used to obtain their local feature information f. 22 f 32 f 42 f 52 f 62 ;

[0018] global context information f 21 f 31 f 41 f 51 f 61 and local feature information f 22 f 32 f 42 f 52 f 62 The weights are multiplied by the backbone network outputs x2, x3, x4, x5, and x6 respectively, and then summed to obtain the updated weight model y2, y3, y4, y5, and y6. The processing procedure is as follows:

[0019] y i =f i1 *x i +f i2 *x i ,i∈{2,3,4,5,6}.

[0020] Furthermore, GCP is used to obtain its global context information f. 21 f 31 f 41 f51 , f 61 , the processing process is as follows:

[0021] f i1 = Convup (Softmax (Relu (Conv (Amp (x i ))))), i ∈ {2, 3, 4, 5, 6}

[0022] In the formula, Amp represents the global maximum pooling operation; Softmax is an activation function; Conv and Convup represent 3×3 convolution operation.

[0023] Further, the LFP is used to obtain its local feature information f 22 , f 32 , f 42 , f 52 , f 62 , the processing process is as follows:

[0024] f i2 = Convup (Softmax (Relu (Conv (Amp (x i ))))), i ∈ {2, 3, 4, 5, 6}

[0025] In the formula, Amp represents the global maximum pooling operation; Softmax is an activation function; Conv and Convup represent 3×3 convolution operation.

[0026] Further, based on the LSEGN model with a dual-channel attention module, a lightweight feature fusion module is introduced to obtain an improved LSEGN model, including:

[0027] After the dual-channel attention module at the output end of Conv3-3, Conv4-3 and Conv5-3, a lightweight feature fusion module is connected respectively; y3, y4, y5 output by the dual-channel attention module are taken as the input x3, x4, x5 of the lightweight feature module, and are respectively named S (3) , S (4) and S (5) side roads, and through the processing process of down-sampling, feature enhancement, up-sampling and feature fusion, the improved LSEGN model is finally obtained, so as to effectively extract clear weld detail features.

[0028] Further, y3, y4, y5 output by the dual-channel attention module are taken as the input x3, x4, x5 of the lightweight feature module, and are respectively named S (3) , S (4) and S (5)The model is then processed through downsampling, feature enhancement, upsampling, and feature fusion to obtain an improved LSEGN model, which effectively extracts clear weld detail features, including:

[0029] The outputs y3, y4, and y5 from the dual-channel attention module are used as inputs to the lightweight feature module, denoted as x3, x4, and x5. These features are then processed through two convolutional layers and a normalization operation to extract preliminary features. The processing steps are as follows:

[0030]

[0031]

[0032] In the formula, These represent the first convolutional block outputs of layers x3, x4, and x5 before downsampling. This represents the second convolutional block output of the first layer x3, x4, and x5 before downsampling; Conv1 and Conv2 represent 3×3 convolution operations; BN represents normalization; ReLU represents the activation function.

[0033] To further extract deeper features, a downsampling operation is performed. The processing steps are as follows:

[0034]

[0035]

[0036] In the formula, S (3) S (4) and S (5) The n values ​​corresponding to the side paths are 4, 3, and 2 respectively; Indicates S during downsampling (i) The first convolutional output of the j-th layer of the edge path; Indicates S during downsampling (i) The second convolutional output of the j-th layer on the side path; MaxPool represents max pooling downsampling;

[0037] The downsampled feature maps are further processed using global average pooling and gated fusion, as follows:

[0038]

[0039]

[0040] In the formula, f i 1 Indicates S during downsampling (i) Gated convolution output of the side path; f i 2 S represents (i)The global average pooling convolution output of the side branch; Conv represents a 1*1 convolution operation;

[0041] In order to restore the spatial resolution of the feature map, the output of the global feature and the gated fusion is processed using an upsampling operation, and the processing process is as follows:

[0042]

[0043]

[0044] In the formula, S (i) The first block convolution output of the jth layer of the side branch; S (i) The second block convolution output of the jth layer of the side branch; Γ is a jump connection function;

[0045] The features after upsampling are fused with the gated convolution output and the global average pooling convolution output to obtain the final feature map y i , and the processing process is as follows:

[0046]

[0047] In the formula, OutConv is a convolution operation.

[0048] Further, the training results of the improved LSEGN model are optimized by using a Dice Loss and a Boundary Loss combined loss function to obtain a weld seam recognition model, including:

[0049] The Dice Loss and the Boundary Loss combined loss function are introduced, the binary classification loss, the similarity loss and the edge loss of the feature map generated by the improved LSEGN model and the label are calculated, the training results of the improved LSEGN model are optimized, and a weld seam recognition model is obtained.

[0050] Further, the training results of the improved LSEGN model are optimized by introducing a Dice Loss and a Boundary Loss combined loss function, calculating the binary classification loss, the similarity loss and the edge loss of the feature map generated by the improved LSEGN model and the label, and obtaining a weld seam recognition model, including:

[0051] The binary classification loss, the similarity loss and the edge loss of the feature map generated by the improved LSEGN model and the label are calculated, and the calculation formula is as follows:

[0052] L CE =-p i logy i -(1-p i) log(1-y i )

[0053]

[0054]

[0055]

[0056]

[0057] Loss=αL CE +βL DL +θL BD

[0058] In the formula, L CE represents the BCE loss function; L DL represents the Dice loss function; L BD represents the Boundary loss function; p i represents the predicted probability of the i-th pixel; y i represents the true label of the i-th pixel; Sigmoid represents the activation function; K x represents the Sobel X convolution kernel; K y represents the Sobel Y convolution kernel; conv2d represents the convolution operation; p represents 1e-6; MSE represents the mean square error; a, b, and theta represent weight coefficients.

[0059] Based on the same inventive concept, a multi-layer multi-pass weld seam recognition system for a multi-source noise environment of the application comprises:

[0060] A data acquisition module is configured to acquire weld seam image data on a workpiece through an image acquisition system, form a multi-layer multi-pass weld seam data set, and divide the weld seam image data in the multi-layer multi-pass weld seam data set into a training set and a test set after marking.

[0061] A model construction module is configured to take the LSEGN model as the basis, fuse multiple double-channel attention modules, and obtain the LSEGN model with the double-channel attention modules; wherein the double-channel attention module is used for adaptive updating of the weights of the backbone network under the interference of spatter and arc noise.

[0062] The model construction module is configured to take the LSEGN model with the double-channel attention module as the basis, introduce a lightweight feature fusion module, and obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to strengthen the deep mining of multi-level features.

[0063] The loss function establishment module is used for introducing a Dice Loss and Boundary Loss combined loss function, and the Dice Loss and Boundary Loss combined loss function is used for processing sample category imbalance and weld overfitting in weld classification.

[0064] The model training module is used for training the improved LSEGN model by using a training set, optimizing the training process of the model by using the Dice Loss and Boundary Loss combined loss function, obtaining a weld identification model, testing the weld identification model by using a test set, and evaluating the performance of the weld identification model.

[0065] The weld identification module is used for identifying a weld in a workpiece welding process by using the tested weld identification model.

[0066] Advantages: Compared with the prior art, the significant technical effects of the present application are as follows:

[0067] The double-channel attention module introduced in the present application can effectively enhance the expression ability of the weld feature, ensure accurate capture of the weld edge and details in a complex noise environment, and reduce the false detection probability caused by splash and arc noise interference.

[0068] The lightweight feature fusion module introduced in the present application can effectively suppress the interference of impurities and fuzzy noise, avoid feature loss, false detection and edge blur, and the like. The module can significantly improve the feature extraction effect of the model under the premise of maintaining a low calculation complexity, thereby ensuring the overall recognition accuracy of the multi-layer multi-pass weld.

[0069] The combined loss function introduced in the present application can simultaneously process the weld classification, sample data imbalance and edge overfitting problem under complex noise interference by introducing the Dice Loss and Boundary Loss combined loss function. The Dice Loss can effectively process the imbalance problem of positive and negative samples of the weld and the background, and the Boundary Loss can help to strengthen the learning of edge information and accurately identify the real shape of the weld, including the irregular fine features on the surface of the weld, thereby effectively reducing the risk of edge blur and inaccuracy. The loss function design can significantly improve the classification ability and edge detail capture ability of the weld identification model, so that the weld identification can perform more stably in a complex noise environment. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 is a flowchart of a multi-layer multi-pass weld joint recognition method for a multi-source noise environment disclosed by an embodiment of the present application.

[0071] Figure 2 is a schematic diagram of a welding image acquisition system disclosed by an embodiment of the present application.

[0072] Figure 3 is a principle structure diagram of a dual-channel attention module disclosed by an embodiment of the present application.

[0073] Figure 4 is a test comparison diagram before and after adding a dual-channel attention module disclosed by an embodiment of the present application.

[0074] Figure 5 is a principle structure diagram of a S (3) side road lightweight feature fusion module disclosed by an embodiment of the present application.

[0075] Figure 6 is a principle structure diagram of a S (4) side road lightweight feature fusion module disclosed by an embodiment of the present application.

[0076] Figure 7 is a principle structure diagram of a S (5) side road lightweight feature fusion module disclosed by an embodiment of the present application.

[0077] Figure 8 is a test comparison diagram before and after adding a lightweight feature fusion module disclosed by an embodiment of the present application.

[0078] Figure 9 is a comparison diagram before and after introducing a combined loss function disclosed by an embodiment of the present application.

[0079] Figure 10 is a principle structure diagram of an overall network disclosed by an embodiment of the present application.

[0080] Figure 11 is a prediction error diagram of an improved LSEGN model for image feature points disclosed by an embodiment of the present application.

[0081] Figure 12 is a structure schematic diagram of a multi-layer multi-pass weld joint recognition system for a multi-source noise environment disclosed by an embodiment of the present application. DETAILED DESCRIPTION

[0082] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description of beneficial effects, and the above and other purposes that can be achieved by the present application will be more clearly understood according to the following detailed description.

[0083] Those of ordinary skill in the art will appreciate that the various illustrative components, systems and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or both. The particular implementation is dependent on the specific application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation does not cause a departure from the scope of the application.

[0084] Reference to an "embodiment" in this disclosure means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of other embodiments. It is expressly understood that the embodiments described herein can be combined with each other in their various permutations and combinations.

[0085] Embodiment 1

[0086] See Figure 1 , Figure 1 is a flowchart of a multi-layer multi-pass weld joint recognition method for a multi-source noise environment according to an embodiment of the application. Figure 1 The multi-layer multi-pass weld joint recognition method described is applied in the field of welding, such as weld joint recognition when thick plate structures are welded using a multi-layer multi-pass welding process. The embodiments of the application are not limited. As shown in Figure 1 The multi-layer multi-pass weld joint recognition method for a multi-source noise environment can include the following operations:

[0087] S1, acquire weld joint image data on a workpiece through an image acquisition system to form a multi-layer multi-pass weld joint data set, and after marking the weld joint image data in the multi-layer multi-pass weld joint data set, divide it into a training set and a test set in proportion. The specific steps are as follows:

[0088] S1.1, acquire V-shaped weld joint images of different thicknesses, different layers, and different noises on the workpiece through the image acquisition system, and all the acquired weld joint image data form a multi-layer multi-pass weld joint data set.

[0089] In this embodiment, as shown in Figure 2 The image acquisition system includes a laser vision sensor 1 for scanning the weld joint, which is loaded on a mechanical arm 2, and then the data set is acquired and processed through an industrial computer 3.

[0090] S1.2, mark the weld joint image data in the multi-layer multi-pass weld joint data set to obtain real labels, and then divide the image data into a training set and a test set according to a ratio of 6:4.

[0091] S2, based on the LSEGN (laser edge guided neural network) model, a plurality of double-channel attention modules are fused to obtain an LSEGN model with a double-channel attention module; wherein the double-channel attention module is used for adaptive updating of the weight of the main network under the interference of spatter and arc noise.

[0092] The double-channel attention module is used for adaptive updating of the weight of the main network under the interference of spatter and arc noise, and inhibits the covering effect of spatter and arc noise on the weld.

[0093] Under the interference of spatter and arc noise, although the LSEGN model can accurately capture the main contour of the weld, there is still a false detection problem. When the spatter adheres around the weld, the model cannot effectively distinguish the weld from the spatter, resulting in the inability to accurately extract the real contour of the weld; and the strong light radiation caused by the arc noise makes the model unable to clearly distinguish the weld from the background, and there is a situation of misjudging the noise as part of the weld. The root cause of this problem lies in that the main network of the LSEGN model is based on the VGGNET architecture. Although VGGNET has certain feature extraction capability, it lacks an adaptive mechanism for weight updating, and cannot effectively constrain the weight of the interference features, that is, it also gives undue weight to false weld features, so it cannot well inhibit the influence of spatter and arc features, and further misjudges false welds as real welds, ultimately resulting in that the model has weak robustness under the interference of these noises, and poor effective separation ability of the noise and the weld. Therefore, the present application designs a double-channel attention module.

[0094] In this embodiment, based on the LSEGN model, a plurality of double-channel attention modules are fused to obtain an LSEGN model with a double-channel attention module, which specifically includes the following:

[0095] As shown in Figure 10 , a double-channel attention module is connected to the output end of Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 of the main part of the LSEGN model, as shown in Figure 3 , the outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 of the main part of the LSEGN model are used as the input of the double-channel attention module, and the updated weight model is obtained through GCP (Global Context Path) and LFP (Local Feature Path), and the updated weight model is used as the LSEGN model with a double-channel attention module. Specifically as follows:

[0096] S2.1, for the outputs of the Conv2-2, Conv3-3, Conv4-3, Conv5-3, Conv6-3 layers of the backbone part in the LSEGN model, respectively as the input x2, x3, x4, x5, x6 of the double-channel attention module, the global context information f is obtained by using GCP 21 31 41 51 61 The processing process is as follows:

[0097] i1 = Convup (Sigmoid (Relu (Conv (Avp (x i ))), i ∈ {2, 3, 4, 5, 6}

[0098] In the formula, Avp represents the global average pooling operation; Relu and Sigmoid represent the activation function; Conv and Convup represent 3x3 convolution operation;

[0099] S2.2, for the outputs of the Conv2-2, Conv3-3, Conv4-3, Conv5-3, Conv6-3 layers of the backbone part in the LSEGN model, respectively as the input x2, x3, x4, x5, x6 of the double-channel attention module, the local feature information f is obtained by using LFP 22 32 42 52 62 The processing process is as follows:

[0100] i2 = Convup (Softmax (Relu (Conv (Amp (x i ))), i ∈ {2, 3, 4, 5, 6}

[0101] In the formula, Amp represents the global maximum pooling operation; Softmax is the activation function.

[0102] S2.3, the global context information f 21 31 41 51 61 and the local feature information f 22 32 42 52 62 ​​​​​​​​​​​​​​​​​​Respectively, after multiplication with the backbone network output x2, x3, x4, x5, x6 and then addition, the updated weight model y2, y3, y4, y5, y6 is obtained, and the processing process is as follows:

[0103] y i =f i1 *x i +f i2 *x i ,i∈{2,3,4,5,6}。

[0104] By adding a double-channel attention module after the output of different layers of VGGNET, fine-grained attention and weighting can be achieved on multiple feature levels, so as to comprehensively utilize the feature information of each layer, so that the model can more effectively capture details and context information when processing complex weld images, and improve the weld detection capability. As shown in Figure 4 , the (a) graph in Figure 4 is the original image, Figure 4 (b) is the test result before joining, Figure 4 (c) is the test result after joining. The LSEGN model fused with the double-channel attention module reduces the false detection probability caused by spatter and arc interference, so as to obtain the accurate weld laser stripe.

[0105] S3, based on the LSEGN model with a double-channel attention module, a lightweight feature fusion module is introduced to obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to strengthen the deep mining of multi-level features.

[0106] The lightweight feature fusion module is used to strengthen the deep mining of multi-level features, and enhance the network's ability to capture subtle weld features under the interference of impurities and fuzzy noise.

[0107] When dealing with interference from impurities and fuzzy noise, the LSEGN model exhibits problems such as feature loss, false detections, and unclear edges. Due to the occlusion of the weld contour by impurities, the model struggles to effectively detect the occluded portion, leading to missed detections in edge extraction. Furthermore, the blurring effect of weld edges significantly reduces image clarity, making it difficult for the model to accurately capture the fine structure of the weld, blurring the boundary between the weld and the background, ultimately resulting in misjudgments and unclear edges. These shortcomings stem from the limitations of the model's multi-scale feature extraction module, which primarily relies on three convolutional layers for feature extraction. The limited depth of these convolutional layers results in insufficient expressive power at the feature level. Faced with impurities and fuzzy interference, poor image clarity and indistinct features make it difficult for the model to extract deeper feature information. In addition, the model's feature fusion approach simply fuses deep global information from the backbone network with shallower information, failing to fully exploit the depth and adaptive characteristics of each layer and neglecting local information within each layer. This prevents the effective capture of subtle feature differences, thus affecting the model's recognition performance. Therefore, this invention designs a lightweight feature fusion module.

[0108] In this embodiment, an improved LSEGN model is obtained by introducing a lightweight feature fusion module based on the LSEGN model with a dual-channel attention module, as detailed below:

[0109] like Figure 10 As shown, a lightweight feature fusion module is connected after the dual-channel attention modules at the outputs of Conv3-3, Conv4-3, and Conv5-3. The outputs y3, y4, and y5 of the dual-channel attention modules are used as the inputs x3, x4, and x5 of the lightweight feature modules, and are named S respectively. (3) S (4) and S (5) Side lanes, corresponding Figure 5 , Figure 6 , Figure 7 Through downsampling, feature enhancement, upsampling, and feature fusion, an improved LSEGN model is obtained, effectively extracting clear weld detail features. The specific steps are as follows:

[0110] S3.1. The outputs y3, y4, and y5 from the dual-channel attention module are used as inputs to the lightweight feature module, denoted as x3, x4, and x5. These are processed through two convolutional layers and a normalization operation to extract preliminary features. The processing steps are as follows:

[0111]

[0112]

[0113] In the formula, These represent the first convolutional block outputs of the first layers x3, x4, and x5 before downsampling; This represents the second convolutional output of the first layer x3, x4, and x5 before downsampling; Conv1 and Conv2 represent 3×3 convolution operations; BN represents normalization; and ReLU represents the activation function.

[0114] S3.2 To further extract deeper features, a downsampling operation is performed. The processing procedure is as follows:

[0115]

[0116]

[0117] In the formula, S (3) S (4) and S (5) The n values ​​corresponding to the side paths are 4, 3, and 2 respectively; Indicates S during downsampling (i) The first convolutional block output of the j-th layer of the edge path; Indicates S during downsampling (i) The second convolutional output of the j-th layer on the side path; MaxPool represents max pooling downsampling.

[0118] S3.3. The downsampled feature map is further processed by global average pooling and gated fusion. The processing procedure is as follows:

[0119]

[0120]

[0121] In the formula, f i 1 Indicates S during downsampling (i) Gated convolution output of the side path; f i 2 S represents (i) The global average pooling convolution output of the side path; Conv represents a 1×1 convolution operation.

[0122] S3.4. To restore the spatial resolution of the feature map, an upsampling operation is used to process the global features and the output after gated fusion. The processing procedure is as follows:

[0123]

[0124]

[0125] Indicates S in upsampling (i) The first convolutional block output of the j-th layer of the edge path; Indicates S in upsampling (i)The output of the second convolutional block in the j-th layer of the edge path; Γ is the skip connection function;

[0126] The upsampled features are fused with the outputs of the gated convolution and the global average pooling convolution to obtain the final feature map y. i The processing procedure is as follows:

[0127]

[0128] In the formula, OutConv represents the convolution operation.

[0129] By analyzing S (3) S (4) and S (5) Lightweight feature fusion modules of varying depths are applied to the side paths to downsample the feature maps to a higher channel count, thereby enhancing the detailed representation of features in high-dimensional space. A multi-skip connection mechanism introduced during upsampling ensures the full integration of features at different scales, effectively recovering the weld detail loss caused by blurring. To further improve robustness against noise, global and local features are fused at the lowest layer of the network, improving the model's robustness to noise interference. The lightweight feature fusion module maintains the integrity of the global structure while meticulously capturing local features, effectively reducing the interference of noise and blurring on weld recognition. Figure 8 As shown, Figure 8 Figure (a) in the image is the original image. Figure 8 Figure (b) shows the test results before addition. Figure 8 The test results in Figure (c) after the addition of the impurities are shown. Due to the occlusion of the weld contour by impurities and the blurring effect of the weld-background boundary, the weld feature information is weakened and the image clarity is reduced. This ultimately leads to problems such as feature loss, false detection, and unclear edges in weld detection. The LSEGN module with the addition of the lightweight feature fusion module significantly alleviates these problems and improves the accuracy of weld detection.

[0130] S4. Introduce the combined loss function of Dice Loss and Boundary Loss. The combined loss function of Dice Loss and Boundary Loss is used to deal with the sample class imbalance and weld overfitting problems in weld classification.

[0131] The combined loss function of Dice Loss and Boundary Loss is designed to address the problem of sample class imbalance in weld classification, enhance the model's ability to accurately capture weld edges under complex noise conditions, and effectively prevent edge overfitting.

[0132] The LSEGN model adopts a binary cross-entropy (BCE) loss function. The main advantage of BCE is that it is simple to calculate and performs well in binary classification problems, and can effectively measure the difference between the predicted results and the true labels, thereby driving the model to achieve high classification accuracy in most cases. However, due to the small proportion of the weld in the image, the number of positive samples is much less than that of negative samples, resulting in a significant class imbalance problem in the data set. This imbalance makes the model more inclined to learn negative samples during training, thereby ignoring the details of the weld area. In addition, due to the usually fuzzy weld edge and the interference of the complex background, the model has certain limitations in edge detection. The generated weld edge image is often too smooth, making it difficult to accurately reflect the true complexity of the weld surface. In view of this, the present application introduces Dice Loss and Boundary Loss on the basis of BCE Loss. Dice Loss is a loss function that measures the similarity between the predicted results and the actual results. By directly optimizing the model's performance on small targets, it can effectively reduce the negative impact of sample imbalance. Especially when dealing with small targets such as welds, it performs excellently. Boundary Loss, on the other hand, focuses on optimizing edge details. Through refined edge detection, the model can more accurately identify the true shape of the weld, including irregular fine features on the weld surface, significantly improving the detection accuracy of the weld edge and effectively reducing the risk of edge blur and inaccuracy.

[0133] S5. Train the improved LSEGN model using the training set; use the Dice Loss and Boundary Loss combined loss function to optimize the training process of the model to obtain a weld recognition model; test the weld recognition model using the test set; and evaluate the performance of the weld recognition model. Specifically as follows:

[0134] S5.1, train the improved LSEGN model using the training set.

[0135] As shown in Figure 10 , according to the improved LSEGN model, the experimental environment and parameter configuration are as shown in Table 1;

[0136] Table 1:

[0137]

[0138] S5.2, use the Dice Loss and Boundary Loss combined loss function to optimize the training process to obtain a weld recognition model; test the weld recognition model using the test set.

[0139] In this embodiment, the training process of the model is optimized by using the combination loss function of Dice Loss and Boundary Loss to obtain a weld joint recognition model, which includes:

[0140] The combination loss function of Dice Loss and Boundary Loss is introduced, the binary classification loss, similarity loss and edge loss of the feature map generated by the improved LSEGN model and the label are calculated, the training result of the improved LSEGN model is optimized, and the weld joint recognition model is obtained. The calculation formula of the loss function is as follows:

[0141] L CE =-p i logy i -(1-p i )log(1-y i )

[0142]

[0143]

[0144]

[0145]

[0146] Loss=αL CE +βL DL +θL BD

[0147] In the formula, L CE represents the BCE loss function; L DL represents the Dice loss function; L BD represents the Boundary loss function; p i represents the prediction probability of the i-th pixel; y i represents the true label (0 or 1) of the i-th pixel; Sigmoid represents the activation function; K x represents the Sobel X convolution kernel; K y represents the Sobel Y convolution kernel; conv2d represents the convolution operation; ρ represents 1e-6; MSE represents the mean square error; α, β, θ represent the weight coefficients.

[0148] By introducing the combination of Boundary Loss and Dice Loss in the loss function, more accurate fitting can be achieved on the edge details and overall structure, thereby comprehensively improving the edge detection capability of the model.

[0149] Boundary Loss focuses on gradient information, enabling the model to better capture the details of the weld edge; while Dice Loss ensures the accurate identification of the overall area. This combined loss function can effectively improve the clarity and accuracy of the edge details in the weld image. As shown in FIGS. 1(a), 1(b) and 1(c), Figure 9 Figure 9 FIG. 1(a) is the original image, Figure 9 FIG. 1(b) is the test result before adding, Figure 9 FIG. 1(c) is the test result after adding, it can be seen that the introduction of the combined loss function significantly improves the performance of the weld edge details, avoiding the problem of edge blur caused by noise.

[0150] S5.3, performance evaluation of the weld recognition model.

[0151] In this embodiment, when evaluating the performance of the weld recognition model, the root mean error (RMSE), peak signal-to-noise ratio (PSNR), structural similarity (SSIM), class average pixel accuracy (mPA), class average intersection over union (mIoU), edge overlap rate (EOR), and feature point prediction error are used as evaluation indicators to verify the accuracy of the weld detection model.

[0152] In order to verify the multi-layer multi-pass weld recognition method in the present application facing multi-source noise environment, the weld data set constructed is tested, and the RMSE, PSNR, SSIM, mPA, mIoU and EOR are used as evaluation indicators, the test results are compared with the U-Net, R3Net, ICON and LSEGN four kinds of models, and the comparison test results are shown in Table 2.

[0153] Table 2:

[0154]

[0155]

[0156] ​As can be seen from Table II, in terms of the RMSE index, the score of the model of the present application is 0.048, which is the lowest among all the comparative models, indicating that the error between the prediction result and the labeled image is the smallest and the accuracy is the highest. In terms of the PSNR index, the score of the model of the present application is 26.427, which is superior to the U-Net, R3Net, ICON and LSEGN models, showing that it has strong performance in image quality. The mPA of the model of the present application is 0.966, which is significantly higher than the U-Net (0.948) and R3Net (0.942), and is only 1.1% lower than the LSEGN. This is because the black background occupies most of the area of the image collected in the actual scene, causing the problem of sample imbalance, and the Dice Loss loss function introduced in the model of the present application reduces the influence of this problem, so the value of mPA is slightly lower than that of LSEGN. In terms of the mIoU index, the model of the present application reaches 0.925, which is about 0.8% higher than the second highest ICON model (0.918), which indicates that the model of the present application has higher accuracy in pixel classification and prediction of the weld area. The SSIM value of the model of the present application is 0.984, which is the highest among all the models, indicating that it can better preserve the structural information of the image and generate a weld image with higher quality closer to the real labeled image. In terms of the EOR evaluation index, the model of the present application reaches 0.723, which is significantly higher than all other models, especially the R3Net, with an improvement of 64.3%. This indicates that the model of the present application has a significant advantage in weld edge detection and can more accurately capture edge information. Compared with other models, the model of the present application can better eliminate background noise and identify the subtle structural details of the weld, making the generated weld edge closer to the real situation, proving the effectiveness and robustness of the model of the present application in the weld recognition task. As can be seen from the above, the method of the present application is more suitable for weld recognition under multi-source noise interference.

[0157] S6, using the tested weld recognition model to recognize the weld in the workpiece welding process.

[0158] In order to verify the effectiveness of the structure of the present application, a series of ablation experiments were conducted on the constructed weld dataset. The LSEGN model was used as the basic model, and the double-channel attention (DCA) module, the lightweight feature fusion (LFF) module and the combined loss function (Combined Loss) were added in turn for comparison, and the experimental results are shown in Table III.

[0159] Table III:

[0160]

[0161] As can be seen from Table III, after introducing the DCA module, the LSEGN 1Compared with the LSEGN model, the model reduces the RMSE by 3.17%, increases the PSNR by 0.99%, increases the mIoU by 3.09%, and increases the EOR by 7.76%. The results show that the DCA module significantly improves the model's interference with weld seam noise. On the basis of the LSEGN 1 model, the present application further introduces LFF to improve the feature fusion capability of the model. After adding the LFF module, the LSEGN 2 model achieves more significant improvements in various indicators: the RMSE is reduced by 18.03%, the PSNR is increased by 6.85%, the mIoU is increased by 5.19%, and the EOR is increased by 27.43%. This shows that the LFF module has beneficial effects in feature fusion and multi-scale information processing of the model. Although the performance of the LSEGN 2 model has been significantly improved, there is still room for improvement in dealing with sample imbalance and edge complexity. Therefore, the present application introduces a combined loss function to further optimize the classification ability and edge detection effect of the model. After adding the combined loss function, the model of the present application performs best: the RMSE is reduced by 4.00%, the PSNR is increased by 1.23%, the mIoU is increased by 1.43%, and the EOR is increased by 10.38%. Finally, compared with the LSEGN model, the RMSE of the model of the present application is finally reduced by 23.8%, the PSNR, mIoU, SSIM and EOR are increased by 9.2%, 9.9%, 1.0%, 51.5% respectively, and the mPA is decreased by 1.1%. In summary, by gradually introducing the DCA module, the LFF module and the combined loss function, the performance of the LSEGN model in feature extraction, multi-scale fusion and loss calculation has been significantly improved. The improved modules have shown significant effects in solving different problems of the model, making the final improved model perform better than the LSEGN in the weld seam recognition task. As can be seen from the above, the model of the present application effectively solves the problems of feature loss, false detection and missed detection of the LSEGN model in weld seam recognition under multi-source noise interference.

[0162] To more intuitively reflect the accuracy of the model of the present application in weld seam feature point prediction, the present application extracts the weld seam feature points, and the error is shown as follows. Figure 11 The results show that the prediction deviation of the model of the present application for most images is within 4 pixels, corresponding to an actual deviation of less than 0.2 mm, which meets the requirements of weld seam tracking.

[0163] Example 2

[0164] Please refer to Figure 12 , Figure 12A structure schematic diagram of a multi-layer multi-pass weld joint recognition system for a multi-source noise environment is disclosed in the embodiments of the present application. The system can realize weld joint recognition when a multi-layer multi-pass welding process is used to weld thick plate structures. Specifically, the system comprises:

[0165] A data acquisition module is configured to acquire weld joint image data on a workpiece through an image acquisition system, form a multi-layer multi-pass weld joint data set, and divide the weld joint image data in the multi-layer multi-pass weld joint data set into a training set and a test set after marking.

[0166] A model construction module is configured to obtain an LSEGN model with a double-channel attention module based on the LSEGN model and the double-channel attention module.

[0167] The model construction module is configured to obtain an improved LSEGN model based on the LSEGN model with the double-channel attention module and introduce a lightweight feature fusion module.

[0168] A loss function establishment module is configured to introduce a Dice Loss and Boundary Loss combined loss function, which is used to handle the sample class imbalance and weld joint overfitting problems in weld joint classification.

[0169] A model training module is configured to train the improved LSEGN model using the training set, optimize the training process of the model using the Dice Loss and Boundary Loss combined loss function, obtain a weld joint recognition model, test the weld joint recognition model using the test set, and evaluate the performance of the weld joint recognition model.

[0170] A weld joint recognition module is configured to recognize weld joints using the tested weld joint recognition model.

[0171] In an optional embodiment, a multi-layer multi-pass weld seam recognition method for a multi-source noise environment comprises: a) acquiring weld seam image data on a workpiece to form a multi-layer multi-pass weld seam data set, and after labeling the weld seam image data in the multi-layer multi-pass weld seam data set, dividing the weld seam image data into a training set and a test set in proportion; b) taking an LSEGN model as a basis, fusing multiple double-channel attention modules to obtain an LSEGN model with double-channel attention modules; c) taking the LSEGN model with double-channel attention modules as a basis, introducing a lightweight feature fusion module to obtain an improved LSEGN model; d) introducing a Dice Loss and Boundary Loss combined loss function; d) training the improved LSEGN model using the training set; using the Dice Loss and Boundary Loss combined loss function to optimize the training process of the model to obtain a weld seam recognition model; testing the weld seam recognition model using the test set; evaluating the performance of the weld seam recognition model; e) using the tested weld seam recognition model to recognize a weld seam.

Claims

1. A multi-layer multi-pass weld seam recognition method for a multi-source noise environment, characterized by, The method comprises the following steps: Obtaining weld image data on a workpiece through an image acquisition system to form a multi-layer multi-pass weld data set, and dividing the weld image data in the multi-layer multi-pass weld data set into a training set and a test set after labeling and proportioning; Based on the LSEGN model, a plurality of double-channel attention modules are fused to obtain an LSEGN model with double-channel attention modules, which comprises: A double-channel attention module is connected to the output end of Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 in the main part of the LSEGN model, and an updated weight model is obtained through GCP and LFP, which is the LSEGN model with double-channel attention modules, comprising: The outputs of the Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 layers in the backbone part of the LSEGN model are respectively taken as inputs of a double-channel attention module 、 、 、 、 The global context information of the GCP is obtained 、 、 、 、 ; The outputs of the Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 layers in the backbone part of the LSEGN model are respectively taken as inputs of a double-channel attention module 、 、 、 、 The local feature information of the LFP is obtained 、 、 、 、 ; global context information , , , , and local feature information , , , , The outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the main body are respectively connected. , , , , Multiply and then add to obtain the updated weight model. , , , , The processing procedure is as follows: ; wherein the dual-channel attention module is used for adaptive updating of weights of the backbone network under spatter and arc noise interference; Based on the LSEGN model with double-channel attention modules, a lightweight feature fusion module is introduced to obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to strengthen the deep mining of multi-level features; A Dice Loss and Boundary Loss combined loss function is introduced, which is used to handle the sample class imbalance and weld overfitting problem in weld classification; The improved LSEGN model is trained using the training set; the Dice Loss and Boundary Loss combined loss function is used to optimize the training process of the model to obtain a weld recognition model; the weld recognition model is tested using the test set; and the performance of the weld recognition model is evaluated. The tested weld recognition model is used for weld recognition in the workpiece welding process.

2. The multi-layer multi-pass weld seam recognition method for a multi-source noise environment according to claim 1, characterized by, The global context information f is obtained by using GCP 21 , f 31 , f 41 , f 51 , f 61 The processing procedure is as follows: ； In the formula, Avp represents a global average pooling operation; Relu and Sigmoid represent activation functions; and Conv and Convup represent 3*3 convolution operations.

3. The multi-layer multi-pass weld seam recognition method for a multi-source noise environment according to claim 1, characterized by, LFP is used to obtain local feature information , , , , , the processing procedure is as follows: ； In the formula, Amp represents a global maximum pooling operation; Softmax is an activation function; Conv and Convup represent 3*3 convolution operations; and Relu represents an activation function.

4. The multi-layer multi-pass weld seam recognition method for a multi-source noise environment according to claim 1, characterized by, Based on the LSEGN model with double-channel attention modules, a lightweight feature fusion module is introduced to obtain an improved LSEGN model, comprising: After the double-channel attention module at the output end of Conv3-3, Conv4-3 and Conv5-3, a light feature fusion module is connected respectively; the output of the double-channel attention module is input into the light feature fusion module 、 、 as the input of the light feature module 、 、 , and is named as 、 and side road respectively, and through the processing process of downsampling, feature enhancement, upsampling and feature fusion, the improved LSEGN model is finally obtained, so as to effectively extract clear weld detail features.

5. The multi-layer multi-pass weld seam recognition method for a multi-source noise environment according to claim 4, characterized in that, The outputs of the dual-channel attention module are taken as the inputs of the light feature module 、 、 , 、 、 , and are respectively named as 、 and side roads, and finally an improved LSEGN model is obtained through a processing process of down-sampling, feature enhancement, up-sampling and feature fusion, so as to effectively extract clear weld detail features, including: The outputs of the dual-channel attention module are taken as the inputs of the lightweight feature module, denoted as , , , , The lightweight feature module is processed through two convolutional layers and normalization operations to extract preliminary features, and the processing process is as follows:​ ； ； wherein, , , respectively represent the first block convolution output of the first layer , , before down-sampling; , , represent the second block convolution output of the first layer , , before down-sampling; Conv1 and Conv2 represent 3x3 convolution operation; BN represents normalization; ReLU represents activation function; In order to further extract deep features, a down-sampling operation is performed, and the processing process is as follows: ； ； wherein , and corresponding to the are 4, 3, 2, respectively; denotes the first block convolution output of the layer of the side path in the down-sampling; denotes the second block convolution output of the layer of the side path in the down-sampling; MaxPool denotes a max-pooling down-sampling; The down-sampled feature map is further processed through global average pooling and gate fusion, and the processing process is as follows: ； ； wherein represents the down-sampling in the gated convolution output of the side branch; represents the the global average pooling convolution output of the side branch; Conv represents a 1 x 1 convolution operation; In order to restore the spatial resolution of the feature map, an up-sampling operation is used to process the global features and the output of the gate fusion, and the processing process is as follows: ； ； wherein denotes upsampling in the first block convolution output of the j-th layer of the side path; denotes upsampling in the first block convolution output of the j-th layer of the side path; the first block convolution output of the j-th layer of the side path; is a skip connection function; The up-sampled features are fused with the gated convolution output and the global average pooling convolution output to obtain a final feature map y i The processing procedure is as follows: ； In the formula, OutConv is a convolution operation.

6. The multi-layer multi-pass weld seam recognition method for a multi-source noise environment according to claim 1, characterized by, The training results of the improved LSEGN model are optimized using the Dice Loss and Boundary Loss combined loss function to obtain a weld recognition model, comprising: The Dice Loss and Boundary Loss combined loss function is introduced, the binary classification loss, similarity loss and edge loss of the feature map generated by the improved LSEGN model and the label are calculated, the training results of the improved LSEGN model are optimized, and a weld recognition model is obtained.

7. The multi-layer multi-pass weld seam recognition method for multi-source noise environment according to claim 6, characterized in that, The Dice Loss and the Boundary Loss combination loss function are introduced, the binary classification loss, the similarity loss and the edge loss of the feature map generated by the improved LSEGN model and the label are calculated, the training result of the improved LSEGN model is optimized, and a weld seam recognition model is obtained, including: The binary classification loss, the similarity loss and the edge loss of the feature map generated by the improved LSEGN model and the label are calculated, and the calculation formula is as follows: ； ； ； ； ； ； In the formula, represents the BCE loss function; represents the Dice loss function; represents the Boundary loss function; represents the prediction probability of the th pixel; represents the true label of the th pixel; Sigmoid represents the activation function; represents the Sobel X convolution kernel; represents the Sobel Y convolution kernel; conv2d represents the convolution operation; The value of ; MSE represents the mean square error; , , represents the weight coefficient.

8. A multi-layer multi-pass weld seam recognition system oriented towards multi-source noise environments, characterized in that, Including: The data acquisition module is used for acquiring the weld seam image data on the workpiece through the image acquisition system, forming a multi-layer multi-pass weld seam data set, marking the weld seam image data in the multi-layer multi-pass weld seam data set, and dividing the training set and the test set according to the proportion; The model construction module is used to take the LSEGN model as the basis, fuse multiple double-channel attention modules, and obtain the LSEGN model with double-channel attention modules, including: A double-channel attention module is connected to the output end of Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 in the LSEGN model main part, and the updated weight model is obtained through GCP and LFP, which is used as the LSEGN model with double-channel attention modules, including: The outputs of the Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 layers in the backbone part of the LSEGN model are respectively taken as inputs of a double-channel attention module 、 、 、 、 The global context information of the GCP is obtained 、 、 、 、 ; The outputs of the Conv2-2, Conv3-3, Conv4-3, Conv5-3 and Conv6-3 layers in the backbone part of the LSEGN model are respectively taken as inputs of a double-channel attention module 、 、 、 、 The local feature information of the LFP is obtained 、 、 、 、 ; global context information , , , , and local feature information , , , , The outputs of Conv2-2, Conv3-3, Conv4-3, Conv5-3, and Conv6-3 layers in the main body are respectively connected. , , , , Multiply and then add to obtain the updated weight model. , , , , The processing procedure is as follows: ; wherein the dual-channel attention module is used for adaptive updating of weights of the backbone network under spatter and arc noise interference; The model construction module is used to take the LSEGN model with double-channel attention modules as the basis, introduce a lightweight feature fusion module, and obtain an improved LSEGN model; wherein the lightweight feature fusion module is used to strengthen the deep mining of multi-level features; The loss function establishment module is used to introduce the Dice Loss and the Boundary Loss combination loss function, and the Dice Loss and the Boundary Loss combination loss function are used to process the sample class imbalance and weld seam overfitting problem in weld seam classification; The model training module is used to train the improved LSEGN model by using the training set; the training process of the model is optimized by using the Dice Loss and the Boundary Loss combination loss function, and a weld seam recognition model is obtained; the weld seam recognition model is tested by using the test set; and the performance of the weld seam recognition model is evaluated; The weld seam recognition module is used to recognize the weld seam in the workpiece welding process by using the tested weld seam recognition model.

Images