Laser welding seam key point automatic detection method based on deep neural network

Abstract

The invention discloses a laser welding seam key point automatic detection method based on a deep neural network, and the method comprises the steps: collecting a welding seam image in real time through a binocular industrial camera, carrying out the regression of the three-dimensional coordinates of a welding seam key point through an end-to-end multi-task deep neural network, and enabling the network to comprise a residual backbone, multi-scale feature fusion, a confidence coefficient branch and a coordinate regression branch. And an attention mechanism is introduced to inhibit arc light and splashing interference. The MLLA-YOLO novel multi-layer linear attention network structure of the method integrates a multi-scale information guiding mechanism and a dynamic receptive field control capability, has an excellent cross-scale key point sensing capability, and can maintain high-precision and high-robustness key point detection performance and output a final key point track even for V-shaped welding seam structures with different geometrical shapes. According to the method, manual participation or external targets are not needed, full-automatic, real-time and high-precision detection of the weld joint key points can be achieved in a complex industrial environment, the single-frame reasoning time is smaller than or equal to 10 ms, the average absolute error is smaller than or equal to 0.05 mm, the MLLA-YOLO can achieve the average positioning error of smaller than or equal to 0.05 mm in various weld joint styles, good universality and engineering adaptability are achieved, and the method is suitable for large-scale popularization and application. And the laser welding quality control precision can be obviously improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a laser weld key point automatic detection method based on a deep neural network, which is suitable for weld quality monitoring and automatic welding guidance, especially for weld structures with significant shape changes, such as V-shaped welds, curved surface welds, etc. BACKGROUND

[0002] Traditional weld detection relies on manual visual inspection or edge extraction based on simple image processing, which has problems such as insufficient accuracy, low efficiency, and poor anti-interference ability. In recent years, deep learning has made significant progress in target detection and key point positioning, which can effectively improve the detection accuracy of weld key structural features. However, existing technologies lack targeted design for automatic detection of weld key points for special-shaped welds, especially for welds with large structural changes, such as V-shaped welds, stepped welds, and wave-shaped welds, which cannot meet the real-time and accuracy requirements of industrial sites. SUMMARY

[0003] The present application provides a laser weld key point automatic detection method based on a deep neural network, which realizes efficient positioning of multiple structural key points in weld images through multi-layer convolution feature extraction combined with a spatial information fusion module, significantly improving the accuracy and robustness of key point detection. To adapt to the diversity of weld types, the present application uses a newly proposed MLLA-YOLO network structure as the key point detection backbone. This network combines the Mamba linear attention module with an improved feature fusion strategy, introduces cross-scale context enhancement channels in the backbone, and uses a multi-layer fusion attention (MLLA) mechanism for guided enhancement in the decoding stage. Through this design, the network can effectively adapt to weld structures with different widths, angles, and depths, especially in the detection of the top angle and end key points of V-shaped welds.

[0004] The technical solution adopted by the present application to achieve the above-mentioned purposes is as follows: a laser weld key point automatic detection method based on a deep neural network, comprising the following steps:

[0005] A binocular industrial camera installed at the welding station is used to collect weld images in real time, obtain visual information covering the entire weld area, and combine a calibration plate or an industrial target to construct an image annotation dataset containing multiple weld cross-section key points; the key points include but are not limited to the arc starting point, the arc ending point, the left fusion width point, the right fusion width point, the excess height point, and the fusion depth point, each key point corresponding to a unique annotation label and spatial position;

[0006] A multi-task deep neural network for key point detection is constructed, which takes a feature extraction module composed of a residual structure as the backbone, combines a feature pyramid structure to realize multi-scale spatial information fusion, embeds a spatial and channel fusion attention mechanism in the high-level feature map of the network to enhance the representation ability of the weld area and suppress the noise of the non-weld area including but not limited to arc light, spatter and metal reflection; a parallel coordinate regression branch and a confidence prediction branch are set to realize the synchronous output of key point position information and reliability information; an adaptive weighted robust loss function-based training strategy is used to train the deep neural network in multiple stages, including global preheating training in the initial stage to stabilize the parameter distribution, multi-task joint optimization in the middle stage to balance the weights of different tasks, and fine-tuning in the later stage to improve the positioning accuracy and generalization performance;

[0007] The trained ideal key point detection model is deployed to an edge computing device supporting AI inference, which performs real-time forward inference on the collected images during the actual welding process, and outputs the confidence heat map and three-dimensional physical coordinate estimation of each key point; a non-maximum suppression (NMS) operation is performed on the key point prediction results to remove redundant responses, and the point with the highest confidence is selected as the final positioning result, and the key point sequence is fitted and smoothed with a cubic spline curve to improve the continuity and stability of the spatial trajectory; finally, it is used for automatic detection of various irregular laser welds.

[0008] The data set construction in the step adopts a combination of semi-automatic labeling and manual review, and uses a line structured light sensor to obtain a high-precision three-dimensional profile of the weld section as the true value.

[0009] The multi-task deep neural network structure for key point detection is as follows:

[0010] Input input layer: used for receiving two-angle irregular welds I 左 、I 右 , performing concat splicing fusion to eliminate lens distortion, and outputting the fused image I;

[0011] Backbone layer: including conv convolution layer, CC_EIFM feature extraction layer, MLLA_Backone multi-layer linear attention mechanism layer;

[0012] conv convolution layer, extracting image I features;

[0013] CC_EIFM feature extraction layer, using CC_EIFM module to extract multi-scale refined features F1, F2 and F3 in parallel;

[0014] MLLA_Backone multi-layer linear attention mechanism layer, the network outputs further enhanced features F1', F2', F3' through MLLA Block and LinearAttention modules respectively;

[0015] Head head output layer, using BiFPN network to fuse F1', F2', F3' from the upper layer input and output F1'', F2'', F3'', that is, three different size feature maps P3 / 8-small, P4 / 16-medium, P5 / 32-large; then through BiFPN network to finally fuse into a single feature map F; finally, using multi-head attention mechanism, calculating image confidence Confidence, using non-maximum suppression (NMS) to remove redundant responses, through Coord Head, further Smoothing processing.

[0016] The CC_EIFM module uses Bottleneck and EIFM modules to gradually enhance the expression ability of features, so that the feature map flows between different modules, and finally generates a feature representation with rich information after multiple convolution, normalization and activation operations; wherein:

[0017] The Bottleneck module: the input feature map x is processed and the output feature map y=x+SiLU(Norm(Conv(x))) is outputted;

[0018] The EIFM module: through feature expansion, deep convolution, coordinate attention mechanism and feature projection operation, the expression ability of the feature map is enhanced; residual connection alleviates gradient disappearance; the specific steps are as follows:

[0019] Feature expansion: using a convolution layer expand to expand the channel number of the input feature map to hidden=in_channels*expansion; output x expanded =act1(bn1(expand(x))) and then through batch normalization layer bn1 and activation function act1 for feature transformation;

[0020] Deep convolution: using a deep convolution layer dwconv to extract spatial features of the expanded feature map, output x depthwis =act2(bn2(dwconv(x expanded ))) and then through batch normalization bn2 and activation function act2 for feature transformation;

[0021] Coordinate attention mechanism: through focusing on the height and width dimensions of the feature map to enhance the model's perception of spatial information, through learning attention weights, dynamically emphasizing important spatial positions in the feature map to make the model focus on key areas, output x attention= attn(x depthwis );

[0022] Feature Projection: Project the channel number of the feature map back to the original channel number using the convolution layer project, and then perform feature transformation through the batch normalization layer bn3, output x projected = bn3(project(x attention ))。

[0023] The MLLA_Backone multi-layer linear attention mechanism layer includes:

[0024] Define the input feature map x∈R B×C×H×W ,B: batch size, C: channel number, H: height of feature map, W: width of feature map;

[0025] First normalization: y norm1 = Norm(x);

[0026] Feature extraction: feature extraction y mlla = MLLA_Block(y norm1 );

[0027] Residual connection: add the input feature map x and the output y mlla of the MLLA Block to form a residual connection y residual =y mlla +x, which helps gradient backpropagation;

[0028] Second normalization: y norm2 = Norm(y residual );

[0029] Feature fusion: feature fusion and nonlinear transformation through the multi-layer perceptron (MLP), further enhancing the feature y mlp = MLP(y norm2 ).

[0030] The MLLA Block module includes:

[0031] Define the input feature map x∈B B×C×H×W ,B: batch size, C: channel number, H: height of feature map, W: width of feature map;

[0032] Linear transformation: linear layer transforms the input feature map into query (Q), key (K), and value (V), q = Linear(x), k = Linear(x), v = Linear(x);

[0033] Attention mechanism: d kis the characteristic dimension of (key) used to scale the dot product to stabilize training;

[0034] Feature fusion: fuse attention weights and values through linear layers and convolution layers, y = Linear (Attention (q, k, v)) + Conv (Linear (x)) to obtain the final output y.

[0035] The BiFPN network effectively integrates feature information of different scales through bidirectional paths and weighted feature fusion.

[0036] In the smoothing process, the predicted key point sequence is smoothed by using a cubic spline curve to eliminate jitter and jump noise; the non-maximum suppression threshold in the step is 0.5, and the cubic spline smoothing uses a cubic polynomial interpolation to eliminate high-frequency noise.

[0037] The adaptive weighted robust loss function in the step is composed of a coordinate regression loss and a confidence loss, wherein the coordinate regression loss adopts an improved robust norm, and the confidence loss adopts a binary cross entropy.

[0038] The edge computing device in the step is an NVIDIA RTX 4060, the inference frame rate is greater than or equal to 30Hz, and the single-frame inference time is less than or equal to 10ms.

[0039] The present application has the following beneficial effects and advantages:

[0040] 1. The welding key point automatic detection method based on the deep neural network has automatic and high-precision detection, does not need manual participation or external targets, can realize full-automatic, real-time and high-precision detection of welding key points in a complex industrial environment, the single-frame inference time is less than or equal to 10ms, and the average absolute error is less than or equal to 0.05mm, thereby significantly improving the laser welding quality control precision.

[0041] 2. The welding key point automatic detection method based on the deep neural network has strong robustness and anti-interference ability, since the attention mechanism (channel-space mixed attention) is introduced, noise interference such as arc light, spatter and metal reflection is effectively inhibited, the feature expression of the welding area is enhanced, and stable detection performance is maintained in a complex working condition.

[0042] 3. The welding key point automatic detection method based on the deep neural network has the advantages of a multi-task deep neural network, adopts an end-to-end multi-task deep neural network, fuses residual backbone, multi-scale feature fusion, confidence branch and coordinate regression branch, synchronously outputs key point position information and reliability information, and ensures the accuracy and reliability of the detection result.

[0043] 4. The invention discloses a kind of based on deep neural network welding key point automatic detection method with multi-stage training strategy, based on the training strategy of adaptive weighted robust loss function, including the global preheating of initial stage, the joint optimization of multi-task in intermediate stage and the fine tuning of later stage, balance different task weight, improve the positioning accuracy and generalization performance of model.

[0044] 5. The invention discloses a kind of based on deep neural network welding key point automatic detection method with real-time and high efficiency, inference frame rate ≥30Hz, single frame inference time ≤10ms, meet the real-time detection demand of industrial welding, effectively guarantee production line efficiency.

[0045] 6. The invention discloses a kind of based on deep neural network welding key point automatic detection method with universality and engineering adaptability, MLLA-YOLO network structure uses multi-layer linear attention module, enhances the modeling ability of cross-scale space features, adapts to the V-shaped weld structure and key point distribution change of different geometric shapes, realizes ≤0.05mm average positioning error under a variety of weld styles.

[0046] 7. The invention discloses a kind of based on deep neural network welding key point automatic detection method with data set construction and true value acquisition data set construction, uses the way that semi-automatic labeling and manual review are combined, and utilizes line structured light sensor to obtain high-precision three-dimensional profile of weld section as true value, ensure data set quality and labeling accuracy.

[0047] 8. The invention discloses a kind of based on deep neural network welding key point automatic detection method with post-processing optimization. Through non-maximum suppression to remove redundant response, combined with cubic spline curve fitting and smoothing processing, improve the continuity and stability of key point trajectory, further optimize detection result.

[0048] 9. The invention discloses a kind of based on deep neural network welding key point automatic detection method with cost effectiveness and safety, reduce the dependence on artificial monitoring, reduce labor cost and safety risk, while improving the stability of welding quality, help to reduce the rate of defective products and rework cost.

[0049] In summary, the invention shows significant advantages in automation degree, detection accuracy, robustness, real-time, universality and engineering adaptability, etc., provides an efficient and reliable solution for laser welding quality monitoring, has broad application prospect and industrial value. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 It is the method flow chart of the invention.

[0051] Figure 2 It is the network structure schematic diagram used in the method of the invention.

[0052] Figure 3 The schematic diagram of the network structure of the CC-EIFM module used in the method of the application.

[0053] Figure 4 The schematic diagram of the network structure of the EIFM module used in the method of the application.

[0054] Figure 5 The schematic diagram of the network structure of the CoordAttention module used in the method of the application.

[0055] Figure 6 The schematic diagram of the network structure of the MLLA_Backone module used in the method of the application.

[0056] Figure 7 The schematic diagram of the network structure of the BiFPN module used in the method of the application.

[0057] Figure 8 The various special-shaped welds used in the application.

[0058] Figure 9 The effect diagram of the weld key points processed by the network architecture, compared with the original. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the specific implementation method of the application will be described in detail below with reference to the drawings. In the following description, a large number of specific details are set forth in order to facilitate a full understanding of the application. However, the application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the spirit of the application, so the application is not limited by the specific implementation disclosed below.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application belongs. The terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the application.

[0061] The application proposes a deep learning-based automatic weld key point detection method, which adopts a lightweight network architecture MLLA-YOLO with a multi-layer linear attention mechanism as the detection backbone network. The network combines multi-scale information fusion and dynamic spatial perception ability, has good cross-scale robustness and geometric adaptability, and is especially suitable for scenes with large changes in weld structure, such as V-shaped welds, stepped welds, and waveform welds.

[0062] The specific implementation steps of the method are as follows:

[0063] Step 1: Image Acquisition and Camera Calibration. A binocular industrial camera is deployed at the welding station. The Zhang Zhengyou calibration method combined with a multi-view reprojection optimization algorithm is used to obtain the camera's intrinsic and extrinsic parameter matrices, achieving the mapping from image pixel coordinates to physical 3D coordinates. Simultaneously, the left and right cameras are triggered to acquire two images. 左 I 右 .

[0064] Step 2: Dataset Construction and Annotation. A dataset containing various weld structures such as V-shaped, U-shaped, and beveled welds was constructed. The labelme tool was used to annotate two images. 左 I 右 Extract key point information such as arc initiation point, arc termination point, left / right melt width point, residual height point, and melt depth point, and perform three-dimensional verification using a line structured light profile measurement system to correct the coordinates of key points.

[0065] Step 3: MLLA-YOLO network structure design. The Backbone part is a lightweight residual network structure to extract basic spatial features. The CC_EIFM structure integrates residual connections and coordinate attention to form a multi-layer linear attention path to enhance the response of key point regions. The MLLA_Backone structure uses a multi-layer linear attention mechanism and has the ability to model long-distance dependencies.

[0066] The Bidirectional Feature Pyramid Network employs efficient bidirectional cross-scale connections and weighted feature fusion, and introduces cross-layer residual connections to achieve multi-scale keypoint information aggregation. The output includes a keypoint heatmap head and a 3D coordinate regression head, enabling simultaneous output of spatial location estimation and confidence prediction.

[0067] like Figure 2 The diagram shown is a schematic of the network structure used in the method of this invention.

[0068] Input layer: Used to receive the irregular weld seam I from the left and right cameras. 左 I 右 Perform concat stitching and fusion to eliminate lens distortion, and output the fused image I.

[0069] Backbone layer:

[0070] 1) Construct a multi-layer linear attention mechanism layer consisting of a conv convolutional layer, a CC_EIFM feature extraction layer, and an MLLA_Backone layer;

[0071] The conv convolutional layer extracts image I features.

[0072] CC_EIFM feature extraction layer:

[0073] First, the fine features F1, F2, F3 of different scales are extracted in parallel, as shown in Figure 3 The whole network uses Bottleneck and EIFM modules to gradually enhance the expressive ability of features through hierarchical feature extraction and fusion. The feature maps flow between different modules, and after multiple convolution, normalization and activation operations, the final feature representation with rich information is generated.

[0074] Bottleneck module: a standard residual bottleneck structure for efficient feature extraction. It consists of the following parts, convolution layer: for feature extraction and channel adjustment; normalization layer: for accelerating training and improving model stability; activation function: introduce nonlinear activation function to enhance the expressive ability of the model. y = x + SiLU(Norm(Conv(x))), x: input feature map, y: output feature map.

[0075] EIFM module: through feature expansion, deep convolution, coordinate attention mechanism, as shown in Figure 5 and feature projection, etc. operations, enhance the expressive ability of feature maps. The use of residual connection helps to alleviate the problem of gradient disappearance, making the network easier to train. Feature expansion: x expanded = act1(bn1(expand(x))), use convolution layer expand to expand the channel number of input feature map to hidden = in_channels * expansion. Then through batch normalization layer bn1 and activation function act1 for feature transformation; Deep convolution: x depthwise = act2(bn2(dwconv(x expanded ))), use deep convolution layer dwconv to extract spatial features of expanded feature map, and then through batch normalization bn2 and activation function act2 for feature transformation; Coordinate attention mechanism: x attention = attn(x depthwise ), through independently focusing on the height and width dimensions of the feature map to enhance the model's perception of spatial information. The main idea is to dynamically emphasize important spatial positions in the feature map by learning attention weights, thereby improving the model's focusing ability on key regions; Feature projection: x projected = bn3(project(x attention )), use convolution layer project to project the channel number of feature map back to the original channel number, and then through batch normalization layer bn3 for feature transformation.

[0076] MLLA_Backone multi-layer linear attention mechanism layer:

[0077] The network realizes efficient feature extraction and attention mechanism through MLLA Block and Linear Attention module. MLLA Block stabilizes the training process through residual connection and normalization layer, while Linear Attention module enhances the expression ability of features through linear transformation and attention mechanism. The combination of the two can effectively process the refined features F1, F2, F3 of different scales extracted from the CC_EIFM feature extraction layer, and output features F1', F2', F3', respectively. As shown in Figure 6 , the feature expression is further enhanced, and high-quality input is provided for the subsequent feature fusion module, effectively improving the detection ability of the model.

[0078] MLLA Block module: enhance the expression ability of feature maps, through multi-layer structure design, which may include residual connection, normalization layer, activation function, etc., to improve the quality and diversity of features. First input feature map x ∈ R B×C×H×W ,B: batch size, C: channel number, H: height of feature map, W: width of feature map, first normalization: y norm = Norm(x), the input feature map is normalized through the normalization layer to stabilize the training process; feature extraction: y mlla = MLLA_Block(y norm1 ), feature extraction is realized through MLLA Block, which contains several layers (such as convolution layer, batch normalization layer and activation function) for feature transformation; residual connection: y residual = y mlla + x, the input feature map x is added to the output y mlla of MLLA Block to form a residual connection, which helps gradient backpropagation; second normalization: y norm2 = Norm(y residual ), the output of the residual connection is normalized again through the normalization layer; feature fusion: y mlp = MLP(y norm2 ), feature fusion and nonlinear transformation are realized through multi-layer perceptron (MLP) to further enhance the expression ability of features.

[0079] Linear Attention module: enhance the focusing ability of the model on important information in the input feature map through attention mechanism. It is usually used to capture the relationship between different positions in the feature map, and through the learned attention weight to strengthen the key features, so as to improve the recognition ability of the model to the target. First input feature map x ∈ R B×C×H×WB: batch size, C: channel number, H: height of feature map, W: width of feature map; linear transformation: linear layer transforms input feature map into query (Q), key (K) and value (V), q = Linear(x), k = Linear(x), v = Linear(x), attention mechanism: d k is the feature dimension of (key), which is used to scale the dot product to stabilize training; feature fusion: the attention weight and value are fused by linear layer and convolution layer, y = Linear(Attention(q, k, v)) + Conv(Linear(x)) to get the final output y.

[0080] Head output layer, adopts BiFPN network, as shown in Figure 7 , F1', F2', F3' from the upper layer input are fused and output F1'', F2'', F3'' three different size feature maps P3 / 8-small, P4 / 16-medium, P5 / 32-large, then fused into a single feature map F through BiFPN network for subsequent operation, using multi-head attention mechanism, calculating image confidence, using non-maximum suppression (NMS) to remove redundant response, through Coord Head, further smoothing processing using cubic spline curve to smooth the predicted key point sequence, eliminating jitter and jump noise. The non-maximum suppression threshold is 0.5, and the cubic spline smoothing uses cubic polynomial interpolation to eliminate high frequency noise.

[0081] BiFPN network: through bidirectional path and weighted feature fusion, it effectively integrates feature information of different scales and improves the performance of target detection. Input feature map: BiFPN receives multi-scale feature maps from backbone, usually including P3, P4, P5, P6 and P7, the resolution of these feature maps decreases from high to low; through the top-down path: starting from high-resolution feature map, information is transmitted layer by layer downward, each layer of feature map is fused with the feature map of the next layer through up-sampling to generate new feature map and the top-up path: starting from low-resolution feature map, information is transmitted layer by layer upward, each layer of feature map is fused with the feature map of the next layer through down-sampling. represents the input feature map, represents the output feature map, w1 and w2 represent the learned weights, and ∈ represents a constant for numerical stability. Then, the output result is obtained through fast normalization fusion. I i represents the input feature map, w i represents the learned weights.

[0082] Step 4: Iterative training with a three-stage training mechanism, backpropagation to adjust parameters to optimize network structure;

[0083] Preheating stage: freeze backbone parameters to stabilize convergence;

[0084] Multi-task stage: coordinate the weights of coordinate error and confidence loss;

[0085] Fine-tuning stage: unfreeze the entire network structure to refine the positioning accuracy. The loss function uses a combination of adaptive weighted robust regression term and binary cross-entropy loss.

[0086] The loss function is as follows:

[0087]

[0088] where α and β are used to balance the influence of the two loss terms, represents the adaptive weighted robust regression term, represents the binary cross-entropy loss.

[0089]

[0090] where w i is the adaptive weight of the i-th sample, is the regression loss.

[0091]

[0092] Step 5: After training, the model is deployed on the RTX4060 platform and integrated into the welding robot control system. The system performs real-time key point prediction on consecutive frame images at a speed of ≥30FPS, outputting key point estimation heat map Ir, which is used for stable path trajectory of welding guidance.

[0093] As shown in Figure 8 , various irregular welds used in the present application.

[0094] As shown in Figure 9 , the key points of the weld after processing by the network architecture, and the effect diagram compared with the original. (a) is the original picture, (b) is the picture processed by the network structure of the present application. And the original network structure parameters are shown in Table 1, and the network structure parameters of the present method are shown in Table 2. After adding the network structure, the processing speed of a single picture, model parameter amount, model size and comprehensive accuracy are all effectively improved.

[0095] Table 1

[0096]

[0097] Table 2

[0098]

[0099] Experiments show that MLLA-YOLO can maintain high accuracy in welds of various shapes, especially in V-shaped weld top corners and end regions, and can still stably detect key points, with an average positioning error controlled within 0.045 mm, meeting the industrial welding path planning precision requirements.

[0100] The above description is merely that of a specific implementation and modifications various from this implementation will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other implementations without departing from the spirit or scope of the application. Accordingly, the application will not be limited to the implementations illustrated herein but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A deep neural network-based automatic detection method for laser weld key points, characterized in that, The method comprises the following steps: Real-time acquisition of weld seam images by a binocular industrial camera installed at a welding station, obtaining visual information covering the entire weld seam area, and combining a calibration plate or an industrial target to construct an image annotation dataset containing multiple weld seam cross-section key points; the key points include but are not limited to an arc starting point, an arc ending point, a left fusion width point, a right fusion width point, a crown point, and a penetration point, each key point corresponding to a unique annotation label and spatial position; A multi-task deep neural network for key point detection is constructed, the network taking a feature extraction module in a residual structure as the backbone, combining a feature pyramid structure to realize multi-scale spatial information fusion, embedding a spatial and channel fusion attention mechanism in the network's high-level feature map to enhance the weld seam area representation ability and suppress non-weld seam area noise including but not limited to arc light, spatter, and metal reflection; a parallel coordinate regression branch and a confidence prediction branch are set to realize the synchronous output of key point position information and reliability information; an adaptive weighted robust loss function-based training strategy is used to train the deep neural network in multiple stages, including global preheating training in the initial stage to stabilize the parameter distribution, multi-task joint optimization in the middle stage to balance different task weights, and fine-tuning in the later stage to improve positioning accuracy and generalization performance; The ideal key point detection model after training is deployed to an edge computing device supporting AI inference, and real-time forward inference is performed on the collected images during actual welding to output the confidence heat map and three-dimensional physical coordinate estimation of each key point; a non-maximum suppression (NMS) operation is performed on the key point prediction results to remove redundant responses, and the point with the highest confidence is selected as the final positioning result; a cubic spline curve fitting and smoothing process is performed on the key point sequence to improve the continuity and stability of the spatial trajectory; and finally, automatic detection of various irregular laser welds is realized.

2. The method according to claim 1, wherein, The dataset construction in the steps adopts a semi-automatic annotation combined with manual review, and a line structured light sensor is used to obtain a high-precision three-dimensional profile of the weld seam section as the true value.

3. The method of claim 1, wherein, The multi-task deep neural network structure for key point detection is as follows: Input input layer: for receiving two angles of left and right cameras special-shaped weld I 左 、 右 , Perform concat splicing fusion to eliminate lens distortion, output fused image I; Backbone backbone layer: including conv convolution layer, CC_EIFM feature extraction layer, and MLLA_Backone multi-layer linear attention mechanism layer; conv convolution layer, extracting image I features; CC_EIFM feature extraction layer, using CC_EIFM module to extract refined features F1, F2, and F3 of different scales in multiple parallel paths; MLLA_Backone multi-layer linear attention mechanism layer, the network outputs further enhanced features F1', F2', and F3' through MLLA Block and Linear Attention modules respectively; Head head output layer, using BiFPN network to fuse F1', F2', F3' from the upper layer input and output F1'', F2'', F3'', that is, three different size feature maps P3 / 8-small, P4 / 16-medium, P5 / 32-large; then through the BiFPN network is finally fused into a single feature map F; finally, using multi-head attention mechanism, calculating the image confidence Confidence, using non-maximum suppression (NMS) to remove redundant responses, through Coord Head, further Smoothing processing.

4. The method according to claim 3, wherein, The CC_EIFM module gradually enhances the expression ability of the features by using the Bottleneck and EIFM modules, so that the feature maps flow between different modules, undergo multiple convolution, normalization and activation operations, and finally generate feature representations with rich information; wherein: The Bottleneck module: the input feature map x is processed and the output feature map y is x+SiLU(Norm(Conv(x))); The EIFM module: through feature expansion, deep convolution, coordinate attention mechanism and feature projection operation, the expression ability of the feature map is enhanced; residual connection alleviates gradient vanishing; the specific steps are as follows: Feature expansion: use the convolution layer expand to expand the channel number of the input feature map to hidden = in_channels * expansion; output x expanded = act1(bn1(expand(x))), which is transformed by the batch normalization layer bn1 and the activation function act1; Deep convolution: use deep convolution layer dwconv to extract spatial features of the expanded feature map, output x depthwis = act2(bn2(dwconv(x expanded ))) to change the features through batch normalization bn2 and activation function act2; Coordinate attention mechanism: enhance the model's perception of spatial information by focusing on the height and width dimensions of the feature map, dynamically emphasize important spatial positions in the feature map by learning attention weights, so that the model focuses on key areas, output x attention = attn(x depthwise ); Feature projection: use the convolution layer project to project the channel number of the feature map back to the original channel number, and then pass through the batch normalization layer bn3 for feature transformation, output x projected = bn3(project(x attention )).

5. The method of claim 3, wherein, The MLLA_Backone multi-layer linear attention mechanism layer comprises: Define the input feature map x e R B×C×H×W B: batch size, C: number of channels, H: height of the feature map, W: width of the feature map; First normalization: y norm1 = Norm(x); Feature extraction: Feature extraction is performed by the MLLA Block y mkla = MLLA_Block(y norm1 ); Residual connection: add input feature map x and output y of MLLA Block mlla to form residual connection y residual = y mlla + x, help gradient backpropagation; Second normalization: y norm2 = Norm(y residual ); Feature fusion: feature fusion and nonlinear transformation are performed by a multilayer perceptron (MLP), further enhancing the features y mlp = MLP(y norm2 ).

6. The method according to claim 5, wherein, The MLLA Block module comprises: Define input feature map x e B B×V×H×W B: batch size, C: number of channels, H: height of feature map, W: width of feature map Linear transformation: the linear layer transforms the input feature map into query (Query), key (Key) and value (Value), q=Linear(x), k=Linear(x), v=Linear(x); Attention mechanism: d k is a feature dimension of the Key that is used to scale the dot product to stabilize training; Feature fusion: fuse the attention weight and value through the linear layer and the convolution layer, y=Linear(Attention(q,k,v))+Conv(Linear(x)) to get the final output y.

7. The method of claim 1, wherein, The BiFPN network effectively integrates feature information of different scales through bidirectional path and weighted feature fusion.

8. The method of claim 1, wherein, In the smoothing process, the predicted key point sequence is smoothed by using a cubic spline curve to eliminate jitter and jump noise; the non-maximum suppression threshold in the step is 0.5, and the cubic spline smoothing uses a cubic polynomial interpolation to eliminate high-frequency noise. 9.The method of claim 1, wherein, The adaptive weighted robust loss function in the step is composed of coordinate regression loss and confidence loss, wherein the coordinate regression loss uses an improved robust norm, and the confidence loss uses binary cross entropy.

10. The method of claim 1, wherein, The edge computing device in the step is NVIDIA RTX 4060, the inference frame rate is greater than or equal to 30Hz, and the single frame inference time is less than or equal to 10ms.

Images