Weld defect visual detection and tracking method based on improved Faster-R-CNN network

Abstract

The invention discloses a welding seam surface defect detection method based on machine vision. The welding seam surface defect detection method comprises the following steps: collecting a welding seam image; constructing sample data; converting the acquired image into a grey-scale map for subsequent processing; carrying out filtering and noise reduction on the converted grey-scale map; enhancing the image; and finally, performing normalization processing on the image to enable the image to be suitable for a data processing process of a detection model. A detection model is constructed, the detection model is based on a Faster-R-CNN network, a deformable convolution DCNv3 module and a convolution attention mechanism EMA module are introduced into a backbone network of the Faster-R-CNN network, and EIoULoss is used as a loss function of the model; carrying out defect detection by utilizing the constructed model; and finally, the welding seam is tracked. According to the invention, the accuracy and speed of weld defect detection are improved.

Description

Technical Field

[0001] This invention belongs to the field of vision-based defect detection technology, specifically a visual detection and tracking method for weld defects based on an improved Faster-R-CNN network. Background Technology

[0002] Welding, as a core manufacturing process, is a key technology in the manufacture of large equipment such as wind turbine towers, compressed air energy storage pipelines, and ships. The quality of the welds directly determines the reliability and safety of the equipment. During the welding process, defects such as weld beads, porosity, undercut, and cracks are prone to appear on the weld surface due to the influence of welding parameter adjustments, process fluctuations, and complex operating environments. These defects can lead to equipment downtime for maintenance or, in severe cases, catastrophic accidents such as pipeline leaks and structural fractures. Therefore, accurate inspection of weld surface quality is crucial. However, traditional weld defect inspection mainly relies on manual visual inspection or simple tools. Due to the subjective experience of the inspectors, visual fatigue, and limitations of operating space, it suffers from low inspection efficiency, high missed detection rates, and frequent false detections, making it difficult to meet the requirements of modern industrial automation and high-precision inspection.

[0003] With the development of computer vision technology, machine vision-based inspection methods are gradually replacing manual inspection. Traditional vision algorithms extract defect features through edge detection and threshold segmentation, but they require parameter optimization for specific defect types and scenarios, have weak noise resistance and poor versatility, and are difficult to handle complex textures and uneven lighting on weld surfaces. In recent years, deep learning algorithms, represented by convolutional neural networks (CNNs), have shown significant advantages in complex environments by automatically learning defect features through end-to-end training without the need for manual rule design.

[0004] However, common object detection algorithms in the field are one-stage and two-stage algorithms. One-stage object detection algorithms omit the candidate region generation stage, performing detection directly on the output layer based on the input image; the YOLO series is a representative example. However, these algorithms are prone to missed detections and have poor accuracy in target localization, resulting in low detection precision. In weld inspection, detection accuracy is paramount to ensuring the safe and stable operation of the equipment. Therefore, a two-stage object detection algorithm is adopted to ensure the accuracy of the detection results. The Faster-R-CNN network eliminates the feature extraction step for candidate regions, reducing computational complexity and significantly improving inference speed. Furthermore, deformable convolution and attention mechanisms are added to the network, and the loss function is optimized to improve both detection accuracy and detection speed. Summary of the Invention

[0005] The purpose of this invention is to provide a visual detection and tracking method for weld defects based on an improved Faster-R-CNN network for the weld defect detection process. Wiener filtering is used for image restoration, adaptive mean filtering is used for image denoising, and Retinex method is used for image enhancement. The aim is to improve the detection accuracy of the model, enhance the model's ability to detect multi-scale target defects, and improve the weld tracking ability.

[0006] This invention is achieved using the following technical solution:

[0007] A visual detection and tracking method for weld defects based on an improved Faster-R-CNN network includes the following steps:

[0008] Step S1, Image Acquisition: Acquire image data for multiple scenes and defect types;

[0009] Step S2: Construct the sample dataset: Divide the obtained data into training / test / validation ratios of 8:1:1;

[0010] Step S3, Image Conversion: Convert to grayscale image to eliminate color interference;

[0011] Step S4, Image Filtering and Denoising: Wiener filtering is used for restoration and adaptive mean filtering for denoising;

[0012] Step S5: Image Enhancement: The Retinex enhancement method is used to enhance the image and highlight defects;

[0013] Step S6, Image Normalization: Improves the model's generalization ability;

[0014] Step S7: Construct the detection model: Based on Fast-R-CNN, add a deformable convolutional DCNv3 module to the backbone network so that the model can adapt to the shape and scale of the defect; add a convolutional attention mechanism EMA module to the backbone network to focus on the ROI region and suppress irrelevant interference; use EIoULoss as the loss function of the model to optimize the bounding box.

[0015] Step S8, Defect Detection: Use the trained detection model to identify and detect defects in the weld, and output the weld identification results;

[0016] Step S9, Weld tracking: The detection model is used to identify the edge of the weld and output the position information of the weld center line to guide the robot's movement.

[0017] In a further preferred embodiment, the method for constructing the sample dataset in step S2 is as follows:

[0018] S2.1 The image data comes from two sources: one is publicly available weld defect images collected online, and the other is relevant images collected independently according to actual needs. This is used to construct the original image dataset to ensure the diversity and representativeness of the data.

[0019] S2.2. Divide the constructed sample database into three subsets in an 8:1:1 ratio: Training set (train): 80%, used to train the model, enabling it to learn defect features and classification rules. Test set (test): 10%, used to evaluate the model's generalization ability and detection accuracy after training. Validation set (val): 10%, used to verify model performance during training, adjust hyperparameters, and prevent overfitting.

[0020] S2.3. Use the LabelImg tool to annotate the defect locations in each weld defect image. After annotation, a corresponding txt file is generated for each image. Each file records two types of key information about the defect: location attribute: the center coordinates of the annotation box.<x_center,y_center> and relative width and height<width,height> Determine the specific location of the defect in the image; Category attributes: Digitally label the defect type, with weld bead marked as 0, porosity marked as 1, undercut marked as 2, and crack marked as 3;

[0021] S2.4. Form an image database (images, storing original images) and a label database (labels, storing annotation information).

[0022] Further preferably, in step S7, the specific method for improving the Faster-R-CNN network is as follows: The deformable convolutional DCVv3 module replaces the fixed convolutions in the residual blocks of the ResNet backbone network of the Faster-R-CNN model. An attention mechanism EMA module is added after the residual blocks of the backbone network and before the global average pooling layer. The regression loss function, SmoothL1 Loss, of the Region Generation Network (RPN) in the Faster-R-CNN network model is optimized to an EIoU loss function.

[0023] In a further preferred embodiment, the specific implementation method of weld seam tracking in step S9 is as follows:

[0024] S9.1 After converting, filtering, denoising, and enhancing the acquired weld seam image, calculate the average grayscale value of each row to lay the foundation for finding the center region of the weld seam in the future.

[0025] S9.2. In the [δ, L2-δ]th row interval of the image (L2 is the total number of rows of the image, and δ is used to avoid interference from image edges), traverse and find the minimum value C1 of the average gray value, and record the corresponding row x1.

[0026] S9.3, in the image of the first and Within the row interval (L0 is the number of pixel rows occupied by the weld width), find the minimum value C2 of the grayscale average again and record the corresponding row x2;

[0027] S9.4 Calculate the row containing the center line At the same time, determine the behavior of the upper and lower edges of the weld [x0-L0,x0+L0];

[0028] S9.5. Use Hough line detection to identify all straight lines in the weld image, remove straight lines with a length lower than the preset value, calculate the average slope K of all remaining straight lines, and calculate the corresponding tilt angle θ based on the average slope K.

[0029] S9.6 Rotate and correct the weld image according to the tilt angle θ.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] First, this invention improves the accuracy and speed of weld defect detection by improving the Faster-R-CNN network model, adding a deformable convolutional DCNv3 module and an attention mechanism module, and optimizing the loss function.

[0032] Second, this invention is based on the Faster-R-CNN network model. A deformable convolutional DCNv3 module is added to the backbone network ResNet to adapt to the shape and scale of defects. An attention mechanism EMA module is added before the global average pooling layer of the backbone network to analyze the important feature information in the image and eliminate the interference of irrelevant information. The loss function in the Region Generation Network (RPN) is optimized to EIoU, which reduces the computational complexity when training the model. In weld tracking, the gray-scale average method is selected to locate the weld centerline, and the Hough transform is used to realize the mapping from image space to parameter space.

[0033] This invention is rationally designed and improves the accuracy and speed of weld defect detection, thus having significant practical application value. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating the visual detection and tracking method for weld defects based on an improved Faster-R-CNN network in an embodiment of the present invention.

[0035] Figure 2 This represents the original Faster-R-CNN network model diagram.

[0036] Figure 3This represents the ResNet network structure, which forms the backbone of the Faster-R-CNN network model.

[0037] Figure 4 This represents the structure diagram of the Region Generation Network (RPN).

[0038] Figure 5 This diagram illustrates the improved Faster-R-CNN network model in this embodiment of the invention.

[0039] Figure 6 This diagram illustrates the operations of traditional convolution and deformable convolution (DCN).

[0040] Figure 7 This diagram illustrates the implementation principle of deformable convolution DCN.

[0041] Figure 8 This diagram illustrates the principle of the attention mechanism EMA. Detailed Implementation

[0042] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0043] like Figure 1 As shown, this invention provides a visual detection method for weld defects based on an improved Faster-R-CNN network, comprising the following steps:

[0044] Step S1: Image Acquisition: Acquire image data for multiple scenes and defect types.

[0045] Step S2: Construct a sample dataset: Divide the obtained data into a training / test / validation ratio of 8:1:1.

[0046] The sample dataset is constructed as follows:

[0047] S2.1 The image data comes from two sources: one is publicly available weld defect images collected online, and the other is relevant images collected independently according to actual needs. This is used to construct the original image dataset to ensure the diversity and representativeness of the data.

[0048] S2.2. Divide the constructed sample database into three subsets in an 8:1:1 ratio: Training set (train): 80%, used to train the model, enabling it to learn defect features and classification rules. Test set (test): 10%, used to evaluate the model's generalization ability and detection accuracy after training. Validation set (val): 10%, used to verify model performance during training, adjust hyperparameters, and prevent overfitting.

[0049] S2.3. Use the LabelImg tool to annotate the defect locations in each weld defect image. After annotation, a corresponding txt file is generated for each image. Each file records two types of key information about the defect: location attribute: the center coordinates of the annotation box.<x_center,y_center> and relative width and height<width,height> Determine the specific location of the defect in the image; Category attributes: digitally label the defect type, with weld beads labeled as 0, porosity as 1, undercut as 2, and cracks as 3.

[0050] In this embodiment, the training set, as the core sample set for training model parameters, contains 752 images and their corresponding label files, mainly used to drive the network to learn the mapping relationship between features and parameters; the test set consists of 94 labeled images, which are used to verify the performance of the trained model and evaluate its generalization ability on unknown data; the validation set also contains 94 labeled images, used to objectively quantify network performance indicators (such as accuracy, loss value, etc.), providing a neutral evaluation basis for model tuning.

[0051] S2.4. Form an image database (images, storing original images) and a label database (labels, storing annotation information).

[0052] Step S3, Image Conversion: Convert to grayscale image to eliminate color interference;

[0053] The conversion method for grayscale images is as follows: One common method for converting a color image to a grayscale image is the weighted average method. This method involves averaging the RGB values ​​of each pixel in the color image according to certain weights, thus obtaining the corresponding grayscale value. The weights are usually determined based on surveys of human eye sensitivity to different colors. Since the human eye is most sensitive to green, followed by red, and least sensitive to blue, the resulting grayscale value Y = 0.299R + 0.587G + 0.114B.

[0054] Step S4, Image Filtering and Denoising: Wiener filtering is used for restoration and adaptive mean filtering for denoising;

[0055] The Wiener filtering restoration method is as follows: Image restoration is the process of reconstructing a degraded image to restore the original appearance of the scene to the greatest extent possible. Wiener filtering is an optimal linear restoration algorithm. Its core is to suppress the interference of noise on the restoration by balancing the power spectra of the signal and noise in the frequency domain. The goal is to minimize the mean square error, i.e., to find the estimated image... The mean square error between the original image f(x,y) and the estimated image is such that... Minimum. The image is converted to the frequency domain, and for each frequency component, a "weight" is calculated: if there is more noise at that frequency, the restoration weight is decreased; if there is more signal, the weight is increased. The adjusted weights are used to restore the signal, and then the image is converted back to the spatial domain to obtain a clear image. The simplified Wiener filter formula is as follows: in, G(u,v) is the Fourier transform of the restored image, and H(u,v) is the Fourier transform of the degraded image. The value of k is usually adjusted according to the different experimental noise levels, and k is usually less than 1.

[0056] The adaptive mean filtering denoising method is as follows: Traditional mean filtering uses a fixed-size window to average the pixel neighborhood, which can cause blurring in edge and detail areas (such as areas where noise differs greatly from neighboring pixels). Adaptive mean filtering, on the other hand, dynamically adjusts the filtering window or weights based on the local statistical characteristics of the pixel neighborhood, achieving edge preservation and noise suppression. The adaptive mean filtering denoising method is as follows: Let the image be f(x,y) (containing noise), the filtered image be g(x,y), and the neighborhood window of the center pixel (x,y) be S. xy (e.g., 3×3, 5×5, etc.), then the statistics within the window include: neighborhood mean m xy The average value of all pixels within the window, reflecting the overall brightness of the area. Where S xy This represents the total number of pixels within the window. Neighborhood variance. The degree to which pixels within a window deviate from the mean reflects the complexity of the region. noise variance σ 2 The noise intensity needs to be reflected by calculating the variance of the smoothed region of the image. This is based on the neighborhood variance. With noise variance σ 2 The formula for calculating the output pixel g(x,y) is as follows: when The neighborhood variance is much larger than the noise variance, indicating that the region contains edges / textures (detail-driven). Output g(x,y)≈f(x,y), preserving the original details. When If the neighborhood variance is close to the noise variance, it indicates that the region is a smooth region (noise-dominated). Output g(x,y)≈m xy Smooth the noise using the mean. When Statistical bias may occur due to an excessively small window; in this case, noise should be considered the primary factor, and mean filtering should still be the main approach. Repeat the above steps for all pixels to obtain the complete filtered image.

[0057] Step S5: Image Enhancement: The Retinex enhancement method is used to enhance the image and highlight defects;

[0058] The Retinex image enhancement method works as follows: An object's color is determined by its ability to reflect red, green, and blue light, not by the absolute value of the reflected light intensity. This color perception consistency, unaffected by lighting conditions, is called color constancy. The Retinex model utilizes this color constancy to achieve a balance between dynamic range compression, edge enhancement, and color constancy, thus enabling adaptive enhancement of various image types. An image f(x,y) is represented as the product of the illumination component l(x,y) and the reflection component r(x,y): f(x,y) = l(x,y)r(x,y), where the illumination component l(x,y) describes the lighting distribution in the scene (low-frequency components, slowly changing), and the reflection component r(x,y) describes the reflective properties of the object's surface (high-frequency components, including texture, edges, and other details). By separating and adjusting the illumination component l(x,y), the reflection component r(x,y) is made clearer, thereby improving image contrast.

[0059] Step S6, Image Normalization: Improves the model's generalization ability.

[0060] Step S7: Construct the detection model: Based on Fast-R-CNN, a deformable convolutional DCNv3 module is added to the backbone network, allowing the model to adapt to the shape and scale of defects. A convolutional attention mechanism EMA module is added to the backbone network to focus on the ROI region and suppress irrelevant interference. EIoULoss is used as the model's loss function to optimize the bounding boxes.

[0061] The original Fast-R-CNN network model architecture is as follows: Figure 2 As shown. The deformable convolution DCVv3 module replaces the fixed convolutions in the residual blocks of the Faster-R-CNN backbone network ResNet. The original Fast-R-CNN backbone network ResNet architecture is as follows. Figure 3As shown, the backbone network ResNet is mainly used for extracting image features. ResNet has various structures with different depths, such as ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152. Based on the block type, they can be divided into two categories: shallow networks based on BasicBlocks: ResNet18 and ResNet34 are built using BasicBlocks. BasicBlocks mainly consist of two 3×3 convolutional layers. Each layer consists of several BasicBlocks; for example, layer 1 of ResNet18 consists of two identical BasicBlocks. Deep networks based on Bottlenecks: ResNet50, ResNet101, and ResNet152 are built using Bottlenecks. Bottleneck adds a convolutional layer between the input and output, and its conv3 expands the number of input channels to four times the original. An attention mechanism EMA module is added after the residual blocks of the backbone network and before the global average pooling layer. The regression loss function, Smooth L1 Loss, of the Region Generation Network (RPN) in the Faster-R-CNN network model is optimized to the EIoU loss function. The original Fast-R-CNN RPN architecture is shown below. Figure 4 As shown, the feature map extracted from the backbone network is first convolved to make the extracted features more robust, resulting in a new feature map. The feature map is then classified to determine whether it is foreground or background, outputting the probability of each feature being foreground or background. The offset of the foreground position relative to the ground truth bounding box is calculated, i.e., bounding box regression is performed to obtain a more precise location. The results of the classification and regression branches are combined, and non-maximum suppression (NMS) is used to remove candidate boxes with excessive overlap, as well as boxes that are too small or exceed the boundaries, ultimately outputting a series of candidate regions. The improved Fast-R-CNN network model architecture is as follows. Figure 5 As shown in the figure. The mechanism of action of each improvement is analyzed below.

[0062] The DCNv3 (Deformable Convolution) module borrows the design concept of separable convolution, achieving effective optimization compared to conventional convolution when handling long-term dependencies and adaptive spatial aggregation. This allows the DCNv3 convolution operator to better adapt to diverse and multi-scale defect detection tasks. Specifically, DCNv3 splits the weights of traditional convolution into depth and point components. While preserving the inductive bias characteristics of convolution, it enables information sharing and interaction between neurons in the convolutional neural network. This design further improves the efficiency of the model training process. Figure 6 The diagram shows the operations of traditional convolution and deformable convolution. Figure 7This demonstrates the implementation principle of DCNv3. One of the key challenges in weld surface defect detection lies in how to adapt to the scale differences, regularity changes, viewpoint shifts, and dynamic changes in geometric shape of the target object. However, the Faster-R-CNN model uses a large number of fixed-size convolutional kernels to perform convolution operations. This approach not only struggles to meet the aforementioned complex and ever-changing requirements but also significantly increases the number of model parameters. Deformable Convolutional Networks (DCNs) possess unique advantages, as they can flexibly adjust their receptive field based on the irregular shape and size of the target features. By replacing the residual block with DCNv3 in the Faster-R-CNN backbone network, the model's ability to analyze multi-scale target features can be enhanced, while expanding the algorithm's perceptual range and effectively reducing model complexity, achieving a dual optimization of performance and efficiency.

[0063] The Attention Mechanism (EMA) module is an attention mechanism designed to improve the feature representation of computer vision tasks. Its core lies in its innovative structural design, which enables efficient integration of multi-scale and cross-spatial information, thereby enhancing the model's ability to capture complex visual patterns.

[0064] like Figure 8 As shown, the process of the EMA module collecting multi-scale spatial information is as follows: for the input tensor x∈R C×H×W (C represents the number of channels, H and W represent spatial dimensions), it is divided into G sub-features, then reshaped and replaced, defining the new shape as 1×1 branch: Encodes channels along the x and y spatial directions using two one-dimensional global average pooling operations. The encoded features are concatenated along the image height direction, sharing a 1×1 convolution. After the output is decomposed into two vectors, a two-dimensional binomial distribution is fitted using the sigmoid activation function after linear convolution, and then the intelligent attention maps of the two channels within the group are aggregated through simple multiplication. 3×3 branch: Stacks 3×3 convolution kernels to capture multi-scale feature representations and captures local cross-channel interactions through 3×3 convolution, expanding the feature space.

[0065] like Figure 8 As shown, the cross-spatial learning process of the EMA module is as follows: For the 1×1 branch output, two-dimensional average pooling is used to encode the global spatial information, which is then converted into the corresponding dimension shape before joint activation of channel features. A linear transformation is fitted using the softmax function of a two-dimensional Gaussian mapping. The parallel output is multiplied by a matrix dot product to generate the first attention space map. For the 3×3 branch, global spatial information is encoded using two-dimensional global average pooling. For the 1×1 branch, the corresponding dimension is transformed before joint activation of channel features. Export a second spatial attention map that retains accurate spatial location information.

[0066] like Figure 8As shown, the final processing flow of the EMA module is as follows: EMA maps the output features of the 1×1 branch and the 3×3 branch, calculates the two sets of spatial attention weight values ​​generated, captures the pixel-level pairwise relationship through the sigmoid function, and completes the final operation.

[0067] One of the key challenges in weld surface defect detection lies in effectively handling the multi-scale variations, complex spatial distribution, and dynamic interactions between features of the target object. However, many traditional models rely on fixed patterns to process spatial information and feature aggregation, making it difficult to adapt to these complex and ever-changing requirements. The EMA module can efficiently collect multi-scale spatial information and flexibly capture pixel-level correlations through a cross-spatial learning mechanism, strengthening long-range dependencies and embedding location information. Introducing the EMA module into the backbone network, replacing part of the traditional structure, can enhance the model's understanding of multi-scale defect features and complex spatial layouts while optimizing the computational process, reducing unnecessary model complexity, and achieving a dual breakthrough in detection performance and computational efficiency, providing a superior solution for weld surface defect detection.

[0068] The EIoU loss function minimizes the difference in width and height between the predicted bounding box and the ground truth bounding box, resulting in faster convergence and better localization. The specific principle is as follows: Intersection over Union (IoU) is an algorithm that calculates the proportion of overlap between different images. Its value ranges from 0 to 1, where 0 indicates that the predicted information is unrelated to the actual information, and 1 indicates that the predicted information is completely related to the actual information.

[0069] In regression tasks, while the SmoothL1 Loss function of the Region Generation Network (RPN) in the Faster R-CNN model optimizes the regression problem to some extent, it still has the following shortcomings. Firstly, the core parameter β (controlling the smoothing region) of SmoothL1 Loss is usually a fixed value, making it difficult to adapt to the regression requirements of targets at different scales. This manifests as decreased regression accuracy for small targets and insufficient optimization for large targets. Secondly, this loss function only focuses on the coordinate error between the predicted and ground truth boxes, without directly incorporating geometric constraints such as Intersection over Union (IoU) and aspect ratio. When the coordinate error between the two boxes is small, but the orientation difference is large, SmoothL1 Loss cannot effectively distinguish them, affecting the quality of the detected boxes.

[0070] The EIoU loss function calculates the length and width of the predicted and ground truth boxes respectively, and adds FocalLoss to focus on high-quality anchor boxes. The specific formula is: Among them, 1-IOU is the traditional IOU term, which measures the proportion of the overlapping area between the predicted box and the ground truth box. The smaller the overlap, the greater the loss. Calculate the distance ρ(b,b) between the center points of the predicted bounding box and the ground truth bounding box. gtThe normalized value of ), where c is the diagonal length of the minimum bounding rectangle of the two boxes, to ensure the center position of the model optimization box; Normalize the absolute difference between width w and height h directly, c w c is the width of the minimum bounding rectangle. h ρ represents the height, and ρ represents the Euclidean distance.

[0071] Table 1 shows the parameters of each layer in the DEE-Faster-R-CNN algorithm model for detecting surface defects in welds.

[0072] Table 1 Parameters of the Improved Network Structure

[0073]

[0074] Step S8, Defect Detection: Use the trained detection model to identify and detect defects in the weld and output the weld identification results.

[0075] Step S9, Weld tracking: The detection model is used to identify the edge of the weld and output the position information of the weld center line to guide the robot's movement.

[0076] The method for weld seam tracking is as follows:

[0077] S9.1 After converting, filtering, denoising, and enhancing the acquired weld seam image, calculate the average grayscale value of each row to lay the foundation for finding the center region of the weld seam in the future.

[0078] S9.2. In the [δ, L2-δ]th row interval of the image (L2 is the total number of rows of the image, and δ is used to avoid interference from image edges), traverse and find the minimum value C1 of the average gray value, and record the corresponding row x1.

[0079] S9.3, in the image of the first and Within the row interval (L0 is the number of pixel rows occupied by the weld width), find the minimum value C2 of the grayscale average again and record the corresponding row x2;

[0080] S9.4 Calculate the row containing the center line At the same time, determine the behavior of the upper and lower edges of the weld [x0-L0,x0+L0];

[0081] S9.5. Use Hough line detection to identify all straight lines in the weld image, remove straight lines with a length lower than the preset value, calculate the average slope K of all remaining straight lines, and calculate the corresponding tilt angle θ based on the average slope K.

[0082] S9.6 Rotate and correct the weld image according to the tilt angle θ.

[0083] Compared to other algorithms, the improved weld defect detection model of this invention, DEE-Faster-R-CNN, achieves a 2.1% improvement in accuracy and a 2.0% improvement in mean accuracy (mAP@0.5). The computational complexity is lower than the original Faster-R-CNN, and although the number of parameters increases, it remains within a reasonable range. Experimental data demonstrate that high detection accuracy and precision for weld defects are achieved with relatively low computational cost.

[0084] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical features of the present invention do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visual detection and tracking method for weld defects based on an improved Faster-R-CNN network, characterized in that: Includes the following steps: Step S1, Image Acquisition: Acquire image data for multiple scenes and defect types; Step S2: Construct the sample dataset: Divide the obtained data into training / test / validation ratios of 8:1:1; Step S3, Image Conversion: Convert to grayscale image to eliminate color interference; Step S4, Image Filtering and Denoising: Wiener filtering is used for restoration and adaptive mean filtering for denoising; Step S5: Image Enhancement: The Retinex enhancement method is used to enhance the image and highlight defects; Step S6, Image Normalization: Improves the model's generalization ability; Step S7: Construct the detection model: Based on Fast-R-CNN, add a deformable convolutional DCNv3 module to the backbone network so that the model can adapt to the shape and scale of the defect; add a convolutional attention mechanism EMA module to the backbone network to focus on the ROI region and suppress irrelevant interference; use EIoULoss as the loss function of the model to optimize the bounding box. Step S8, Defect Detection: Use the trained detection model to identify and detect defects in the weld, and output the weld identification results; Step S9, Weld tracking: The detection model is used to identify the edge of the weld and output the position information of the weld center line to guide the robot's movement.

2. The method for visual detection and tracking of weld defects based on an improved Faster-R-CNN network according to claim 1, characterized in that: In step S2, the sample dataset is constructed as follows: S2.1 The image data comes from two sources: one is to collect publicly available images of weld defects, and the other is to collect relevant images according to actual needs, thereby constructing the original image dataset to ensure the diversity and representativeness of the data. S2.2 Divide the constructed sample database into three subsets in an 8:1:1 ratio: 80% for training the model to learn defect features and classification rules; 10% for testing the model to evaluate its generalization ability and detection accuracy after training; and 10% for validation to verify model performance, adjust hyperparameters, and prevent overfitting during training. S2.

3. Use the LabelImg tool to annotate the defect locations in each weld defect image; after annotation, generate a txt file corresponding to the image, with each file recording two types of key information about the defect: location attribute: the center coordinates of the annotation box.<x_center,y_center> and relative width and height<width,height> Determine the specific location of the defect in the image; Category attributes: Digitally label the defect type, with weld bead marked as 0, porosity marked as 1, undercut marked as 2, and crack marked as 3; S2.

4. Form an image database (images) to store the original images, and a label database (labels) to store the annotation information.

3. The method for visual detection and tracking of weld defects based on an improved Faster-R-CNN network according to claim 1, characterized in that: In step S3, the grayscale conversion method is as follows: the RGB values ​​of each pixel in the color image are weighted and averaged according to a certain weight to obtain the corresponding grayscale value. The weight is determined based on the survey results of human eyes' sensitivity to different colors. Since human eyes are most sensitive to green, followed by red, and least sensitive to blue, the obtained grayscale value Y = 0.299R + 0.587G + 0.114B.

4. The method for visual detection and tracking of weld defects based on an improved Faster-R-CNN network according to claim 1, characterized in that: In step S4, the Wiener filter restoration method is as follows: the objective is to minimize the mean square error, i.e., to find the estimated image. The mean square error between the original image f(x,y) and the estimated image is such that... Minimum; convert the image to the frequency domain, and calculate a weight for each frequency component: if there is a lot of noise at that frequency, reduce the repair weight; If there are many signals, increase the weights, use the adjusted weights to repair the signals, and then convert back to the spatial domain to obtain a clear image; The Wiener filter formula is as follows: in, G(u,v) is the Fourier transform of the restored image, H(u,v) is the Fourier transform of the degraded image, and k is the Fourier transform of the point spread function, where k takes a value less than 1.

5. The method for visual detection and tracking of weld defects based on an improved Faster-R-CNN network according to claim 1, characterized in that: In step S4, the adaptive mean filtering denoising method is as follows: Let the image be f(x,y), the filtered image be g(x,y), and the neighborhood window of the center pixel (x,y) be S. xy The statistics within the window include: neighborhood mean m xy Neighborhood variance noise variance σ 2 ; Where S xy This represents the total number of pixels within the window. Based on neighborhood variance With noise variance σ 2 The formula for calculating the output pixel g(x,y) is as follows: when The area in question contains edges / textures. Output g(x,y)≈f(x,y), preserving the original details; when The region in question is a smooth region. Output g(x,y)≈m xy Smooth noise using the mean; when At this point, noise can be considered the dominant factor, and mean filtering should still be the primary method. Repeat the above steps for all pixels to obtain a complete filtered image.

6. The method for visual detection and tracking of weld defects based on an improved Faster-R-CNN network according to claim 1, characterized in that: In step S5, the Retinex image enhancement method is as follows: the image f(x,y) is represented as the product of the illumination component l(x,y) and the reflection component r(x,y): f(x,y) = l(x,y)·r(x,y), where the illumination component l(x,y) describes the illumination distribution in the scene, and the reflection component r(x,y) describes the reflection characteristics of the object surface. By separating and adjusting the illumination component l(x,y), the reflection component r(x,y) is made clearer, thereby improving the image contrast.

7. The method for visual detection and tracking of weld defects based on an improved Faster-R-CNN network according to claim 1, characterized in that: In step S7, the specific improvements to the Faster-R-CNN network are as follows: the deformable convolution DCVv3 module replaces the fixed convolution in the residual block of the ResNet network, the backbone network of the Faster-R-CNN model; an attention mechanism EMA module is added after the residual block of the backbone network and before the global average pooling layer; the regression loss function of the region generation network RPN in the Faster-R-CNN network model is optimized from smooth L1 loss to EIoU loss function.

8. The method for visual detection and tracking of weld defects based on an improved Faster-R-CNN network according to claim 1, characterized in that: In step S9, the specific implementation method of weld seam tracking is as follows: S9.1 After converting, filtering, denoising, and enhancing the acquired weld seam image, calculate the average grayscale value of each row. S9.

2. Within the [δ, L2-δ]th row interval of the image, iterate through and find the minimum value C1 of the average gray value, and record the corresponding row x1; where L2 is the total number of rows in the image, and δ is used to avoid image edge interference; S9.3, in the image of the first and Within the row interval, find the minimum value C2 of the average grayscale value again and record the corresponding row x2; where L0 is the number of pixel rows occupied by the weld width; S9.4 Calculate the row containing the center line At the same time, determine the behavior of the upper and lower edges of the weld [x0-L0,x0+L0]; S9.

5. Use Hough line detection to identify all straight lines in the weld image, remove straight lines with a length lower than the preset value, calculate the average slope K of all remaining straight lines, and calculate the corresponding tilt angle θ based on the average slope K. S9.6 Rotate and correct the weld image according to the tilt angle θ.

Images