An automatic detection system for welding seams of electric welding machines based on CV

Abstract

The present invention relates to the field of machine vision technology, and specifically discloses a CV-based electric welding machine weld automatic detection system, including an image acquisition module, a model building module, a model training module, a path fitting module, a defect detection module, and an integrated application module. This solution adds a NAM attention mechanism, uses channel attention and spatial attention to focus the model's attention on the features of the weld area, dynamically adapts to changes in the shape and size of the weld through convolution replacement operations, and greatly reduces the number of parameters and computational complexity of the model through lightweight improvements; uses edge detection to find the outer edge contour of the weld area, uses a path fitting algorithm to ensure a smooth transition of the path, and generates a continuous and smooth welding path; simplifies the defect detection model, focuses on higher-resolution feature extraction, more effectively captures subtle defect information, and greatly improves detection accuracy, and adds a coordinate attention mechanism to improve the model's sensitivity to defects and positioning accuracy.

Description

Technical Field

[0001] The invention relates to the technical field of machine vision, and in particular to an automatic weld seam detection system for electric welders based on CV. Background Art

[0002] In welding inspection of electric welders, machine vision can be used to detect the shape, position and quality of welds and help identify defects in welds. General detection models have poor adaptability. For situations where the shape and size of welds change greatly, the accuracy is low, and the complexity and computational cost are high, making it difficult to meet the requirements of real-time detection. General weld detection methods have insufficient edge detection accuracy, discontinuous contour extraction, and fixed path fitting parameter selection. They cannot adapt to slight changes in the edge of the weld and have a risk of overfitting. General defect detection methods perform poorly when detecting small defects and areas with irregular boundaries, and the defect area is not accurately located. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an automatic weld detection system for electric welding machines based on CV. In view of the poor adaptability of general detection models, low accuracy for situations where the weld shape and size change greatly, and high complexity and computational cost, it is difficult to meet the requirements of real-time detection. This solution adds a NAM attention mechanism, uses channel attention to highlight weld features, uses spatial attention to focus the model's attention on the weld area, and dynamically adapts to changes in weld shape and size through convolution replacement operations to improve detection accuracy. While maintaining a high feature extraction capability, the model's parameter volume and computational complexity are greatly reduced through lightweight improvements, and it is adapted to the real-time weld detection requirements of embedded devices. In view of the insufficient edge detection accuracy, discontinuous contour extraction, and path fitting parameter selection of general weld detection methods, this solution also provides a new method for automatic weld detection of electric welding machines based on CV. The solution uses edge detection to find the outer edge contour of the weld area, uses a path fitting algorithm to ensure a smooth transition of the path, balances fitting accuracy and path smoothness, makes the welding path fit the weld edge as closely as possible, and avoids overfitting. It generates a continuous and smooth welding path to ensure that the welder moves along the fitting path points to complete accurate welding operations. In view of the problems that general defect detection methods are insufficient in detecting small defects and areas with irregular boundaries, and the defect area is inaccurately located, this solution simplifies the defect detection model, focuses on higher-resolution feature extraction, more effectively captures subtle defect information, greatly improves detection accuracy, adds a coordinate attention mechanism, and more accurately extracts row and column position information of the defect area, thereby improving the model's sensitivity to defects and positioning accuracy.

[0004] The present invention provides a CV-based automatic weld detection system for electric welders, comprising an image acquisition module, a model building module, a model training module, a path fitting module, a defect detection module, and an integrated application module;

[0005] The image acquisition module uses an industrial camera to acquire weld images, annotates weld edges and defective areas in the weld images, generates image labels, processes the weld images using an image enhancement method, integrates the processed weld images into a data set, and sends the data set to a model training module;

[0006] The model building module introduces a pre-trained basic segmentation model, uses the NAM attention mechanism, dynamic convolution replacement operation, and lightweight processing to improve the basic segmentation model, and sends the improved basic segmentation model to the model training module;

[0007] The model training module divides the data set into a training set and a test set, uses the improved basic segmentation model for training on the training set, adjusts the learning rate and the number of iterations of the basic segmentation model during the training process, tests the accuracy of the basic segmentation model on the test set and further optimizes it, generates a weld detection model, and sends the weld detection model to the path fitting module and the integrated application module;

[0008] The path fitting module uses the weld detection model to output a weld segmentation map, performs edge detection on the weld segmentation map, extracts the weld edge contour, obtains the best fitting path of the weld through a fitting algorithm, and sends the best fitting path to the integrated application module;

[0009] The defect detection module introduces a pre-trained target detection model, simplifies the target detection model, and adds a coordinate attention mechanism to obtain a defect detection model, and sends the defect detection model to the integrated application module;

[0010] The integrated application module integrates the weld detection model and the defect detection model into the industrial computer, connects the industrial camera and the electric welder, sets the user interface to display the weld segmentation map, the best fit path and other welding-related parameters, and tracks the welding path in real time when the electric welder performs the welding operation. If an offset and a defective area are detected in the welding path, an alarm is automatically issued and abnormal data is recorded.

[0011] Furthermore, the model construction module includes: a model import unit, a NAM attention mechanism unit, a dynamic convolution replacement unit, and a lightweight unit;

[0012] The model importing unit imports a pre-trained basic segmentation model, obtains pre-trained weights, and initializes a trunk structure, a neck structure, and a head structure of the basic segmentation model. The basic segmentation model uses a convolution layer to perform a convolution operation on the weld image, and each output channel generates a feature map.

[0013] The NAM attention mechanism unit adds a NAM attention mechanism to the basic segmentation model, including channel attention and spatial attention: the channel attention is weighted on the feature map generated by each output channel of the basic segmentation model, and the spatial attention is weighted on the spatial dimension, and finally outputs the spatial weighted feature map;

[0014] The dynamic convolution replacement unit replaces the standard convolution in the neck structure of the basic segmentation model with a dynamic convolution, assigns dynamic weights to each input channel to process the spatial weighted feature map, and outputs a dynamic convolution feature map. The formula used is as follows:

[0015] ;

[0016] In the formula, represents the dynamic convolution feature map, represents the channel index, Indicates the number of channels, represents the spatial weighted feature map Channel values, Represents the weight of the dynamic convolution kernel;

[0017] The lightweight unit replaces the backbone structure of the basic segmentation model with a lightweight feature extraction structure, decomposes the convolution into depth-separable convolution and channel shuffle operations, inputs a dynamic convolution feature map, and outputs a lightweight feature map. The formula used is as follows:

[0018] ;

[0019] In the formula, Represents the lightweight feature map of the final output, represents a depth-wise separable convolution operation, represents the point convolution operation, Represents a channel shuffle operation.

[0020] Further, the path fitting module includes: an edge detection unit, a contour extraction unit, a path fitting unit, and a path generation unit;

[0021] The edge detection unit generates a weld segmentation map using a weld detection model, and extracts the outer edge of the weld in the weld segmentation map using an edge detection operator to generate an edge map;

[0022] The contour extraction unit extracts continuous weld edge contour points in the edge map to obtain a contour point set;

[0023] The path fitting unit generates a fitting curve using a fitting algorithm based on the contour point set;

[0024] The path generation unit samples the fitting curve into a set of ordered path points, performs path smoothing operations and continuous operations on the path points, and generates and outputs the best fitting path.

[0025] Further, the defect detection module includes a layer replacement unit, a coordinate attention mechanism unit, and a defect location unit;

[0026] The hierarchical replacement unit simplifies the pooling layer of the target detection model, deletes the low-resolution feature layer, and adds a ReLU activation function;

[0027] The coordinate attention mechanism unit adds a coordinate attention mechanism to the neck structure of the model and obtains channel and defect position information through average pooling operation;

[0028] The defect location unit adds a loss function, trains and tests the target detection model, adjusts the model parameters, and obtains the optimal defect detection model to extract the defect area coordinate points.

[0029] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0030] (1) In view of the poor adaptability of general detection models, low accuracy, high complexity and computational cost, and difficulty in meeting the requirements of real-time detection when the shape and size of welds change greatly, this solution adds a NAM attention mechanism, uses channel attention to highlight weld features, uses spatial attention to focus the model's attention on the weld area, and dynamically adapts to changes in weld shape and size through convolution replacement operations to improve detection accuracy. Through model lightweight improvements, while maintaining a high feature extraction capability, the model's parameter volume and computational complexity are greatly reduced, and it can meet the real-time weld detection requirements of embedded devices.

[0031] (2) In view of the problem that the general weld detection method has insufficient edge detection accuracy, discontinuous contour extraction, fixed path fitting parameters, and cannot adapt to slight changes in the weld edge, there is a risk of overfitting. This solution uses edge detection to find the outer edge contour of the weld area and uses a path fitting algorithm to ensure a smooth transition of the path, balance fitting accuracy and path smoothness, make the welding path fit the weld edge as closely as possible, avoid overfitting, generate a continuous and smooth welding path, and ensure that the electric welder moves along the fitting path points to complete accurate welding operations.

[0032] (3) In view of the fact that general defect detection methods perform poorly in detecting small defects and areas with irregular boundaries, and have the problem of inaccurate positioning of defect areas, this solution simplifies the defect detection model and focuses on higher-resolution feature extraction to more effectively capture subtle defect information, greatly improve detection accuracy, and add a coordinate attention mechanism to more accurately extract the row and column position information of the defect area, thereby improving the model's sensitivity to defects and positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of an automatic weld seam detection system for electric welders based on CV proposed by the present invention;

[0034] Figure 2 Schematic diagram of building blocks for the model;

[0035] Figure 3 is a schematic diagram of the path fitting module;

[0036] Figure 4 Schematic diagram of the defect detection module.

[0037] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0039] Example 1, see Figure 1 The present invention provides a CV-based automatic weld detection system for electric welding machines, comprising an image acquisition module, a model building module, a model training module, a path fitting module, a defect detection module, and an integrated application module;

[0040] The image acquisition module uses an industrial camera to acquire weld images, selects different lighting conditions and background environments, covers common welding scenes, annotates weld edges and defective areas in weld images, generates image labels, processes weld images using rotation, scaling, sharpening and splicing methods, integrates the processed weld images into a data set, and sends the data set to a model training module;

[0041] The model building module introduces a pre-trained YOLOv8n-seg model, uses the NAM attention mechanism, dynamic convolution replacement operation, and lightweight processing to improve the YOLOv8n-seg model, and sends the improved YOLOv8-seg model to the model training module;

[0042] The model training module divides the data set into a training set and a test set, uses the improved YOLOv8-seg model for training on the training set, adjusts the learning rate and the number of iterations of the YOLOv8-seg model during the training process, tests the accuracy of the YOLOv8-seg model on the test set and further optimizes it, generates a weld detection model, and sends the weld detection model to the path fitting module and the integrated application module;

[0043] The path fitting module uses the weld detection model to output a weld segmentation map, performs edge detection on the weld segmentation map, extracts the weld edge contour, obtains the best fitting path of the weld through a spline curve fitting algorithm, and sends the best fitting path to the integrated application module;

[0044] The defect detection module introduces a pre-trained YOLOv8n target detection model, simplifies the YOLOv8n target detection model, and adds a coordinate attention mechanism to obtain a defect detection model, and sends the defect detection model to the integrated application module;

[0045] The integrated application module integrates the weld detection model and the defect detection model into the industrial computer, connects the industrial camera and the electric welder, sets the user interface to display the weld segmentation map, the best fit path and other welding-related parameters, and tracks the welding path in real time when the electric welder performs the welding operation. If an offset and a defective area are detected in the welding path, an alarm is automatically issued and abnormal data is recorded.

[0046] Example 2, see Figure 1 and Figure 2 ,This embodiment is based on the above embodiment, and the model building module includes: a model import unit, a NAM attention mechanism unit, a dynamic convolution replacement unit, and a lightweight unit;

[0047] The model importing unit imports a pre-trained YOLOv8n-seg segmentation model as a basic segmentation model, obtains pre-trained weights, initializes a trunk structure, a neck structure, and a head structure of the basic segmentation model, and the YOLOv8n-seg segmentation model uses a convolution layer to perform a convolution operation on the weld image, and generates a feature map for each output channel;

[0048] The NAM attention mechanism unit adds a NAM attention mechanism to the basic segmentation model, including channel attention and spatial attention: the channel attention is weighted on the feature map generated by each output channel of the basic segmentation model, and the spatial attention is weighted on the spatial dimension, and finally outputs the spatial weighted feature map;

[0049] The dynamic convolution replacement unit replaces the standard convolution in the neck structure of the basic segmentation model with a dynamic convolution, assigns dynamic weights to each input channel to process the spatial weighted feature map, and outputs a dynamic convolution feature map. The formula used is as follows:

[0050] ;

[0051] In the formula, represents the dynamic convolution feature map, represents the channel index, Indicates the number of channels, represents the spatial weighted feature map Channel values, Represents the weight of the dynamic convolution kernel;

[0052] The lightweight unit replaces the backbone structure of the basic segmentation model with the ShuffleNetv2 structure, decomposes the convolution into depth-separable convolution and channel shuffle operations, inputs a dynamic convolution feature map, and outputs a lightweight feature map. The formula used is as follows:

[0053] ;

[0054] In the formula, Represents the lightweight feature map of the final output, represents a depth-wise separable convolution operation, represents the point convolution operation, Represents a channel shuffle operation.

[0055] By performing the operations, the general detection model has poor adaptability, low accuracy, high complexity and computational cost, and is difficult to meet the requirements of real-time detection due to the high complexity and computational cost when the weld shape and size change greatly. This solution adds a NAM attention mechanism, uses channel attention to highlight weld features, uses spatial attention to focus the model's attention on the weld area, and dynamically adapts to changes in weld shape and size through convolution replacement operations to improve detection accuracy. The lightweight model can be improved while maintaining a high feature extraction capability, while greatly reducing the number of model parameters and computational complexity, and meeting the real-time weld detection requirements of embedded devices.

[0056] Example 3, see Figure 1 and Figure 2This embodiment is based on the above embodiment. Channel attention is added to the attention mechanism unit. Specifically, the feature map is input, and average pooling and maximum pooling operations are performed on each input channel to obtain two feature vectors. The feature vectors are input into the fully connected network for feature fusion to obtain channel attention weights. The channel attention weights are multiplied by the input feature map to highlight the weld features. The feature map after channel attention processing is output. The formula used is as follows:

[0057] ;

[0058] In the formula, represents the input feature map, Represents the feature map output after adding channel attention, represents the feature vector obtained after average pooling processing, represents the feature vector obtained after the maximum pooling process, represents a fully connected network, represents the channel attention weight.

[0059] Example 4, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. The attention mechanism unit adds spatial attention. Specifically, the feature map after channel attention processing is average pooled and max pooled to obtain two feature vectors. The feature vectors are fused through convolution operation to obtain spatial attention weights. The spatial attention weights are multiplied by the feature map to obtain a spatial weighted feature map. The formula used is as follows:

[0060] X '' =σ' Conv [AvgPool X ' ,MaxPool X ' ] · X ' ;

[0061] In the formula, represents the spatial weighted feature map, represents the convolution operation, represents the spatial attention weight.

[0062] Example 5, see Figure 1 This embodiment is based on the above embodiment. The model training module first locks the parameters of the backbone structure and only trains the added NAM attention mechanism and dynamic convolution; then unlocks all layers, trains and fine-tunes the overall model; after the training is completed, the basic segmentation model is tested on the test set, and the channels, spatial dimension weights and dynamic convolution kernel size of the NAM attention mechanism are adjusted to generate the optimal weld detection model.

[0063] Example 6, see Figure 1 and Figure 3, this embodiment is based on the above embodiment, the path fitting module includes: an edge detection unit, a contour extraction unit, a path fitting unit, and a path generation unit;

[0064] The edge detection unit generates a weld segmentation map using a weld detection model, extracts the outer edge of the weld in the weld segmentation map using a Sobel edge detection operator, and generates an edge map. The formula used is as follows:

[0065] ;

[0066] In the formula, represents the edge graph, Represents the weld segmentation diagram, and Respectively represent the gradients of weld segmentation in the x and y directions;

[0067] The contour extraction unit extracts continuous weld edge contour points in the edge map to obtain a contour point set;

[0068] The path fitting unit uses a spline curve fitting algorithm to insert curve segments between contour points based on the contour point set to generate a fitting curve. The boundary condition of the spline fitting selects the natural boundary condition, and the formula used is as follows:

[0069] ;

[0070] In the formula, represents the fitting curve, represents the horizontal coordinate of the calculated contour point, Indicates The starting point of the segment spline, Indicates The function value of the segment spline at the starting point, , , represents the spline coefficient;

[0071] The path generation unit samples the fitting curve into a set of ordered path points, performs path smoothing operations and continuous operations on the path points, and generates and outputs the best fitting path.

[0072] By performing the operations, in view of the problems of insufficient edge detection accuracy, discontinuous contour extraction, fixed path fitting parameter selection, inability to adapt to slight changes in the weld edge, and risk of overfitting in general weld detection methods, this solution uses edge detection to find the outer edge contour of the weld area, and uses a path fitting algorithm to ensure a smooth transition of the path, balance fitting accuracy and path smoothness, make the welding path fit the weld edge as closely as possible, avoid overfitting, generate a continuous and smooth welding path, and ensure that the electric welder moves along the fitting path points to complete precise welding operations.

[0073] Embodiment 7, see Figure 1 and Figure 4 ,This embodiment is based on the above embodiment, and the defect detection module includes a layer replacement unit, a coordinate attention mechanism unit, and a defect location unit;

[0074] The layer replacement unit replaces the SPPF layer of the YOLOv8n target detection model with the SimSPPF layer, deletes the P5 feature layer, and uses the ReLU activation function instead of the SiLU activation function;

[0075] The coordinate attention mechanism unit adds a coordinate attention mechanism to the neck structure of the model and obtains channel and defect location information through average pooling operation. The formula used is as follows:

[0076] ;

[0077] ;

[0078] In the formula, and denote the row and column vectors of coordinates respectively, and Represents the position index, and Respectively represent the number of rows and columns of the feature map, and Respectively represent the row and column of the current index, Represents a feature map;

[0079] The defect localization unit uses the WIoUv3 loss function to replace the IoU loss function, trains and tests the YOLOv8n target detection model, adjusts the model parameters, and obtains the optimal defect detection model to extract the coordinate points of the defect area.

[0080] By performing the above operations, the general defect detection method performs poorly in detecting small defects and areas with irregular boundaries, and has the problem of inaccurate positioning of defective areas. This solution simplifies the defect detection model, focuses on higher-resolution feature extraction, more effectively captures subtle defect information, greatly improves detection accuracy, adds a coordinate attention mechanism, and more accurately extracts row and column position information of defective areas, thereby improving the model's sensitivity to defects and positioning accuracy.

[0081] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0082] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0083] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A CV-based automatic weld detection system for electric welders, characterized in that: It includes image acquisition module, model building module, model training module, path fitting module, defect detection module and integrated application module; The image acquisition module acquires a weld image, marks the weld edge and defect area in the weld image, processes the weld image using an image enhancement method, integrates the processed weld image into a data set, and sends the data set to a model training module; The model building module introduces a pre-trained basic segmentation model, uses the NAM attention mechanism, dynamic convolution replacement operation, and lightweight processing to improve the basic segmentation model, and sends the improved basic segmentation model to the model training module; The model training module divides the data set into a training set and a test set, uses the improved basic segmentation model for training on the training set, tests the accuracy of the basic segmentation model on the test set and further optimizes it, generates a weld detection model, and sends the weld detection model to the path fitting module and the integrated application module; The path fitting module uses the weld detection model to output a weld segmentation map, performs edge detection on the weld segmentation map, extracts the weld edge contour, obtains the best fitting path of the weld through a fitting algorithm, and sends the best fitting path to the integrated application module; The defect detection module introduces a pre-trained target detection model, simplifies the target detection model, and adds a coordinate attention mechanism to obtain a defect detection model, and sends the defect detection model to the integrated application module; The integrated application module integrates the weld detection model and the defect detection model into the industrial computer, connects the industrial camera and the electric welder, sets the user interface to display the weld segmentation map, the best fit path and other welding-related parameters, and tracks the welding path in real time when the electric welder performs the welding operation. If an offset and a defective area are detected in the welding path, an alarm is automatically issued and abnormal data is recorded.

2. The CV-based automatic weld detection system for electric welders according to claim 1, characterized in that: The model building module includes: a model import unit, a NAM attention mechanism unit, a dynamic convolution replacement unit, and a lightweight unit; The model importing unit imports a pre-trained basic segmentation model, obtains pre-trained weights, and initializes a trunk structure, a neck structure, and a head structure of the basic segmentation model. The basic segmentation model uses a convolution layer to perform a convolution operation on the weld image, and each output channel generates a feature map. The NAM attention mechanism unit adds a NAM attention mechanism to the basic segmentation model, including channel attention and spatial attention: the channel attention is weighted on the feature map generated by each output channel of the basic segmentation model, and the spatial attention is weighted on the spatial dimension, and finally outputs the spatial weighted feature map; The dynamic convolution replacement unit replaces the standard convolution in the neck structure of the basic segmentation model with a dynamic convolution, assigns dynamic weights to each input channel to process the spatial weighted feature map, and outputs a dynamic convolution feature map. The formula used is as follows: ； In the formula, represents the dynamic convolution feature map, represents the channel index, Indicates the number of channels, Represents the spatial weighted feature map Channel values, Represents the weight of the dynamic convolution kernel; The lightweight unit replaces the backbone structure of the basic segmentation model with a lightweight feature extraction structure, decomposes the convolution into depth-separable convolution and channel shuffle operations, inputs a dynamic convolution feature map, and outputs a lightweight feature map. The formula used is as follows: ； In the formula, Represents the lightweight feature map of the final output, represents a depth-wise separable convolution operation, represents the point convolution operation, Represents a channel shuffle operation.

3. The CV-based automatic weld detection system for electric welders according to claim 2, characterized in that: The path fitting module includes: an edge detection unit, a contour extraction unit, a path fitting unit, and a path generation unit; The edge detection unit generates a weld segmentation map using a weld detection model, and extracts the outer edge of the weld in the weld segmentation map using an edge detection operator to generate an edge map; The contour extraction unit extracts continuous weld edge contour points in the edge map to obtain a contour point set; The path fitting unit generates a fitting curve using a fitting algorithm based on the contour point set; The path generation unit samples the fitting curve into a set of ordered path points, performs path smoothing operations and continuous operations on the path points, and generates and outputs the best fitting path.

4. The CV-based automatic weld seam detection system for electric welders according to claim 3 is characterized in that: The defect detection module includes a layer replacement unit, a coordinate attention mechanism unit, and a defect location unit; The hierarchical replacement unit simplifies the pooling layer of the target detection model, deletes the low-resolution feature layer, and adds a ReLU activation function; The coordinate attention mechanism unit adds a coordinate attention mechanism to the neck structure of the model and obtains channel and defect position information through average pooling operation; The defect location unit adds a loss function, trains and tests the target detection model, adjusts the model parameters, and obtains the optimal defect detection model to extract the defect area coordinate points.

Images