A multi-stage weld defect detection method and system fusing improved residual network
By improving the multi-stage weld defect detection method using YOLOv11 and dual-branch ResNet networks, the problem of distinguishing between defects and pseudo-defects in weld detection was solved, achieving efficient and accurate weld defect detection and quantification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI UNIV OF ENG SCI
- Filing Date
- 2026-03-03
- Publication Date
- 2026-06-02
AI Technical Summary
Existing weld inspection methods struggle to accurately distinguish between weld defects and pseudo-defects in complex environments, resulting in low inspection accuracy. This is especially true when multiple defects coexist, making it difficult to balance high recall and high precision.
An improved YOLOv11 network was used for initial localization and coarse classification, and a dual-branch ResNet network was used for pixel-level segmentation and fine classification. Multimodal feature fusion was used to distinguish defects, especially incomplete penetration and incomplete fusion.
It achieves efficient and accurate weld defect detection, improves recall rate by 3-5 times, and increases the accuracy of distinguishing between incomplete penetration and incomplete fusion from 70% to 90%, with pixel-level defect quantification capability.
Smart Images
Figure CN122134696A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of welding inspection technology, and in particular to a multi-stage weld defect detection method and system that integrates an improved residual network. Background Technology
[0002] Industrial X-ray weld inspection is a core means of ensuring the welding quality of critical equipment such as pressure vessels, pipelines, ships, and aerospace equipment, directly affecting the operational safety and service life of the equipment. On the production site, weld images often contain crucial information that determines product quality. If defects are missed or misjudged, subsequent rework costs are high, and it may even lead to serious safety accidents. Therefore, achieving accurate and reliable automated defect detection has extremely high engineering value.
[0003] Current weld defect detection methods generally face the challenge of unstable recognition performance in real-world industrial scenarios. Weld images have complex backgrounds and contain numerous false defects, such as spatter, undercut, surface scratches, and weld textures. These interferences are highly similar to real defects in grayscale distribution and local morphology, making it difficult for detection systems to effectively distinguish them. Furthermore, various defect types, such as porosity, cracks, lack of fusion, and slag inclusions, exhibit minimal differences in shape, size, and grayscale contrast. Especially when the defect size is small, these features are extremely subtle and easily masked by background noise, resulting in a large number of missed detections.
[0004] More importantly, the visual similarity between these defects and between defects and the background leads to extremely serious category confusion problems. For example, a thin, elongated crack and a thin, elongated lack of fusion may be almost indistinguishable in their overall outline, while a small, round pore and a circular artifact that occasionally appears in the weld are also very similar in grayscale statistical characteristics. When the detection system attempts to handle both localization and classification tasks simultaneously, this similarity directly leads to frequent errors in the classification stage of accurately located candidate regions, resulting in high confusion phenomena such as classifying cracks as lack of fusion and pores as slag inclusions. This confusion not only reduces the overall detection accuracy but also makes it difficult for the system to simultaneously achieve high recall and high accuracy when faced with complex weld images in real production where multiple defects coexist. This forces companies to repeatedly weigh the risks of missed detections against the false alarm rate in practical applications.
[0005] Therefore, how to accurately locate, clearly distinguish, and reliably classify weld defects under conditions of severe interference from complex backgrounds, high visual similarity between defects and pseudo-defects, and high similarity in visual features between different real defects has become a key problem that must be solved in intelligent industrial X-ray weld defect detection. Summary of the Invention
[0006] The technical problem to be solved by the present invention is to provide a multi-stage weld defect detection method that integrates an improved residual network. The first stage uses an improved YOLOv11 to perform coarse defect localization with high efficiency and high recall. The second stage uses a ResNet-based encoder-decoder network to perform pixel-level segmentation of candidate regions and extract fine features for secondary classification, which in particular improves the ability to distinguish between similar defects such as incomplete penetration and lack of fusion.
[0007] The technical solution of this invention is: a multi-stage weld defect detection method integrating an improved residual network, the innovation of which lies in: specifically including the following steps: S1: Image preprocessing; S2: Use the YOLOv11 network with integrated attention mechanism for preliminary defect localization and coarse classification; S3: Crop the ROI region and input it into the dual-branch ResNet network; S4: One branch performs pixel-level segmentation, while the other branch fuses depth features, segmented geometric features, and grayscale statistical features for fine classification. S5: Results fusion and output.
[0008] Furthermore, the S1 image preprocessing specifically involves: using adaptive histogram equalization to enhance contrast, and combining wavelet transform for image noise reduction while preserving defect edge information.
[0009] Furthermore, S2: Preliminary defect localization and coarse classification are performed using a YOLOv11 network with an integrated attention mechanism; specifically: Attention mechanism integration: A coordinate attention module is introduced into the Neck layer of YOLOv11, which makes the network pay more attention to the detailed features of the weld area and defects, and suppresses background interference; Small target detection layer: High-resolution shallow feature maps are preserved and enhanced in the feature pyramid network, specifically for detecting small targets such as pores; Output: Generate a series of bounding boxes containing suspected defects, along with preliminary classification confidence, covering the categories of: porosity, incomplete penetration, lack of fusion, cracks, seizures, and concavity; the goal of this stage is to improve recall.
[0010] Furthermore, the S3-trimmed ROI region is input into the dual-branch ResNet network; specifically: Input: Expand each candidate box region output by S2 proportionally, crop the region of interest (ROI) from the original image, and normalize it to a fixed size; Core network: Construct a ResNet-U-Net hybrid network with a dual-branch shared encoder.
[0011] Furthermore, the ResNet-U-Net hybrid network is specifically as follows: Basic skeleton: Based on ResNet-50 or ResNet-101, retain the output feature maps of the first four stages (Stage 1-4) (denoted as L1, L2, L3, L4), with the spatial resolution decreasing step by step (1 / 4, 1 / 8, 1 / 16, 1 / 32 of the original image respectively), and the semantic information being enhanced step by step; Cross-stage feature refinement module: inserted after the residual block of each stage; includes a parallel channel attention and spatial attention submodule, and performs feature fusion and dimensionality reduction through a 1x1 convolution to enhance the encoder's ability to capture weld defects, especially the subtle textures of weak and blurry linear defects; allows the network to suppress background noise such as weld ripples in the early stages and emphasize the edge and internal texture changes of suspected defect areas. Spatial Context Pyramid: Employs a variant of Spatial Pyramid Pooling (ASPP); uses 3x3 dilated convolutions with different dilation rates (e.g., rates=1, 6, 12, 18) to process L4 features in parallel, then upsamples and concatenates them with features from a global average pooling branch, and finally integrates them through 1x1 convolutions; introduces multi-scale contextual information at the bottleneck of the encoder (after Stage 4 output), significantly expanding the receptive field without sacrificing resolution, so that each feature point output by the encoder incorporates information from local details to global context, which helps to understand the morphology of the entire defect.
[0012] Furthermore, in step S4, one branch performs pixel-level segmentation, while the other branch fuses depth features, segmented geometric features, and grayscale statistical features for fine-grained classification; specifically: Branch 1: Using a U-Net structure with ResNet as the encoder, pixel-level semantic segmentation of the ROI is performed to accurately delineate the outline and internal structure of the defect; the segmentation results help to quantify geometric features such as defect area and length. Branch 2: Shares the deep features extracted by the ResNet encoder with the segmentation branch, and is followed by a multi-head attention mechanism and a fully connected layer.
[0013] Furthermore, the input in the second classification includes shared depth features and defect shape descriptors from the segmentation branches, including compactness, elongation, Hu moment, and gray-scale statistical features. The gray-scale statistical features include the gray-scale mean and variance comparison between the segmented region and the background region within the ROI.
[0014] Furthermore, S5: Result fusion and output; specifically: Map the precise bounding box obtained from S4 (the minimum bounding rectangle generated based on the segmentation mask) and the classification results back to the original image coordinates; Non-maximum suppression (NMS) is used to eliminate overlapping boxes; The final output is a structured report containing the defect type, precise location, confidence level, pixel-level profile, and calculated defect dimensions (such as pore diameter and crack length).
[0015] An innovative multi-stage weld defect detection system integrating an improved residual network is characterized by the following features: a preprocessing module, a preliminary detection module based on an improved YOLOv11, a fine analysis module based on a dual-branch ResNet network, and a post-processing and output module. The preprocessing module employs adaptive histogram equalization to enhance contrast and combines wavelet transform for image noise reduction while preserving defect edge information. The preliminary detection module based on the improved YOLOv11 is used to generate a series of bounding boxes containing suspected defects, along with preliminary classification confidence, covering categories such as porosity, incomplete penetration, lack of fusion, cracks, seizures, and concavity, thereby improving the recall rate. The fine analysis module based on the dual-branch ResNet network is used to enable the network to suppress background noise (such as weld ripples) in the early stages, emphasize the edge and internal texture changes of suspected defect areas, and significantly expand the receptive field without losing resolution, so that each feature point output by the encoder integrates information from local details to global context, which helps to understand the morphology of the entire defect. The post-processing and output module outputs the final result: a structured report containing the defect type, precise location, confidence level, pixel-level contour, and calculated defect dimensions (such as pore diameter and crack length).
[0016] The advantages of this invention are: 1) Cascaded architecture design: Adopting the strategy of "detection first, segmentation and classification later", it first ensures that no defects are missed, and then performs fine analysis on candidate areas, which balances speed and accuracy.
[0017] 2) Targeted improvements: The coordinate attention mechanism is integrated into YOLOv11 and the small target detection layer is enhanced, which significantly improves the ability to locate small and blurred targets in weld DR images.
[0018] 3) Multimodal feature fusion classification: It is proposed to use ResNet encoder shared features and combine the geometric features of defects and local gray-scale distribution features for comprehensive judgment, which provides an effective technical path to solve the industry segmentation problem of incomplete penetration and incomplete fusion.
[0019] 4) Dual-task collaboration: The segmentation and classification tasks share the same encoder. Segmentation results guide classification, and classification needs optimize segmentation. The two tasks promote each other during training, improving the overall performance of the model.
[0020] 5) End-to-end system: Provides a complete automated solution from raw DR image input to defect report output with accurate quantification information, with high practicality; inspection efficiency is improved by 3-5 times, the overall mAP reaches 97.6%, the accuracy of distinguishing between incomplete penetration and non-fusion is improved from 70% of the single model to 90%, and it has pixel-level defect quantification capability. Attached Figure Description
[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0022] Figure 1 This is a flowchart of a multi-stage weld defect detection method that integrates an improved residual network according to the present invention.
[0023] Figure 2 This is a schematic diagram of the improved YOLOv11 network structure for a multi-stage weld defect detection method that integrates an improved residual network according to the present invention.
[0024] Figure 3 This is a schematic diagram of the dual-branch ResNet (ResNet-U-Net hybrid) network structure of a multi-stage weld defect detection method that integrates improved residual networks according to the present invention. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0026] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0027] like Figures 1 to 3 This invention presents a multi-stage weld defect detection method and system that integrates an improved residual network. The aim of this invention is to provide a two-stage, multi-task fusion detection method to address the aforementioned problems. The first stage utilizes an improved YOLOv11 for efficient and high-recall coarse defect localization. The second stage uses a ResNet-based encoder-decoder network to perform pixel-level segmentation of candidate regions and extracts fine features for secondary classification, particularly improving the ability to distinguish between similar defects such as incomplete penetration and lack of fusion.
[0028] The core of this invention is a cascaded processing flow of "coarse positioning → fine segmentation → precise classification".
[0029] Specifically as follows: Input and preprocessing: Input: Digital radiographic (DR) image of the weld seam.
[0030] Preprocessing: Adaptive histogram equalization (CLAHE) is used to enhance contrast, and wavelet transform is combined to reduce image noise while preserving defect edge information.
[0031] Phase 1: Coarse localization and initial screening of defect regions based on improved YOLOv11 Model: YOLOv11 is used as the basic detection framework.
[0032] Areas for improvement: Attention mechanism integration: A coordinate attention (CA) module is introduced into the Neck layer of YOLOv11, which makes the network pay more attention to the detailed features of the weld area and defects, and suppresses background interference.
[0033] Small target detection layer: High-resolution shallow feature maps are preserved and enhanced in the Feature Pyramid Network (FPN) specifically for detecting small targets such as pores.
[0034] Output: Generate a series of bounding boxes containing suspected defects, along with preliminary classification confidence scores (covering coarse categories such as porosity, linear defects (incomplete penetration / lack fusion / cracks), and root defects (bite / concavity)). The goal at this stage is high recall; it is better to have false positives than false negatives.
[0035] Phase 2: Fine-grained defect segmentation and classification based on dual-branch ResNet Input: Expand each candidate box region output from the first stage by a certain ratio (e.g., 1.2 times), crop the region of interest (ROI) from the original image, and normalize it to a fixed size.
[0036] Core network: Construct a ResNet-U-Net hybrid network with a dual-branch shared encoder.
[0037] Branch 1 (Segmentation Branch): Employing a U-Net structure with ResNet as the encoder, pixel-level semantic segmentation of the ROI is performed to accurately delineate the contours and internal structures of defects. The segmentation results help quantify geometric features such as defect area and length.
[0038] Branch 2 (Classification Branch): Shares the deep features extracted by the ResNet encoder with the segmentation branch, and is followed by a multi-head attention mechanism and a fully connected layer.
[0039] The input to the classification branch not only includes shared depth features, but also incorporates defect shape descriptors (such as compactness, elongation, and Hu moment) and grayscale statistical features (comparison of grayscale mean and variance between the segmented region and the background region within the ROI) from the segmentation branch. This multimodal fusion of "visual features + geometric features + grayscale statistical features" is key to distinguishing between incomplete penetration and lack of fusion.
[0040] 2.1 Improved ResNet Encoder Basic skeleton: Based on ResNet-50 or ResNet-101, retain the output feature maps of the first four stages (Stage 1-4) (denoted as L1, L2, L3, L4), with the spatial resolution decreasing step by step (1 / 4, 1 / 8, 1 / 16, 1 / 32 of the original image respectively), and the semantic information being enhanced step by step.
[0041] Improvement 1: Cross-Stage Feature Refinement (CSFR) module Objective: To enhance the encoder's ability to capture localized fine textures of weld defects (especially weak, blurry linear defects).
[0042] Structure: Inserted after the residual block of each stage. Contains a parallel channel attention and spatial attention submodule, and performs feature fusion and dimensionality reduction through a 1x1 convolution.
[0043] Function: To enable the network to suppress background noise (such as weld ripples) in the early stages of the film and to highlight the edge and internal texture variations of suspected defect areas.
[0044] Improvement 2: Spatial Context Pyramid (SCP) Objective: To introduce multi-scale contextual information at the bottleneck of the encoder (after the Stage 4 output), which is crucial for identifying defects of different sizes (e.g., circular and strip defects).
[0045] Structure: A variant of Spatial Pyramid Pooling with Dilation (ASPP) is employed. L4 features are processed in parallel using 3x3 dilated convolutions with different dilation rates (e.g., rates=1, 6, 12, 18), then upsampled and concatenated with features from a global average pooling branch, and finally integrated using 1x1 convolutions.
[0046] Function: Without sacrificing resolution, it significantly expands the receptive field, so that each feature point output by the encoder incorporates information from local details to global context, which helps to understand the overall morphology of the defect.
[0047] 2.2 Dual-branch interaction design Shared and branching points: Features enhanced by the SCP module (referred to as "rich semantic features F") serve as the common input source for both branches.
[0048] (1) Segmentation Branch Decoder structure: It adopts a progressive upsampling structure similar to FPN / U-Net.
[0049] Feature fusion: After each upsampling, the features are concatenated with the corresponding encoder features (L1, L2, L3). The innovation lies in the fact that before concatenation, the encoder features are first passed through a lightweight feature selection gate—consisting of two convolutional layers and a sigmoid activation function—to learn and weight the more important channel features.
[0050] Final output: Generate a binary segmentation mask with the same resolution as the input ROI through a 1x1 convolution + Sigmoid.
[0051] (2) Classification Branch Feature input: Rich semantic features F: Derived from the SCP module, containing high-level semantics and global context.
[0052] Geometric features G: Calculated in real-time from the prediction mask of the segmentation branch (during training and inference). These include: area of the defect region, aspect ratio, density, and the first three invariant moments of the Hu moment. These features are encoded into feature vectors through a small fully connected network (FC layer).
[0053] Gray-scale statistical feature I: In real time, from the original ROI image, the region is divided into foreground (defect) and background according to the predicted mask, and the difference between the mean gray-scale values and the gray-scale variance ratio between the two are calculated. This is also encoded through an FC layer.
[0054] (3) Feature fusion and classification: Step 1: Deep Feature Processing. The semantically rich features F are flattened into vectors by passing them through a Global Average Pooling (GAP) layer.
[0055] Step 2: Multimodal feature fusion. The three feature vectors (from F, G, I) are concatenated together.
[0056] Step 3: Adaptive Feature Calibration (AFC). Design a small network (such as two FC layers with ReLU and Dropout in between) to perform nonlinear transformation and recalibration on the concatenated fused features, automatically learning the weights of different feature dimensions for the classification of the current sample.
[0057] Step 4: Classification. The calibrated features are fed into the final fully connected layer + Softmax, outputting the probability distribution of 6 types of defects.
[0058] Output: Precise defect binary segmentation mask.
[0059] Precise defect categories (porosity, incomplete penetration, lack of fusion, cracks, root engagement, root concavity) and their confidence levels.
[0060] Post-processing and result output: The accurate bounding box (the smallest bounding rectangle generated based on the segmentation mask) obtained in the second stage and the classification results are mapped back to the coordinates of the original image.
[0061] Non-maximum suppression (NMS) is used to eliminate overlapping boxes.
[0062] The final output is a structured report containing the defect type, precise location, confidence level, pixel-level profile, and calculated defect dimensions (such as pore diameter and crack length).
[0063] Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.
Claims
1. A multi-stage weld defect detection method integrating an improved residual network, characterized in that: Specifically, the following steps are included: S1: Image preprocessing; S2: Use the YOLOv11 network with integrated attention mechanism for preliminary defect localization and coarse classification; S3: Crop the ROI region and input it into the dual-branch ResNet network; S4: One branch performs pixel-level segmentation, while the other branch fuses depth features, segmented geometric features, and grayscale statistical features for fine classification. S5: Results fusion and output.
2. The multi-stage weld defect detection method integrating an improved residual network according to claim 1, characterized in that: The S1 image preprocessing specifically involves: using adaptive histogram equalization to enhance contrast, and combining wavelet transform for image noise reduction while preserving defect edge information.
3. The multi-stage weld defect detection method integrating an improved residual network according to claim 1, characterized in that: S2: Preliminary defect localization and coarse classification are performed using a YOLOv11 network with an integrated attention mechanism; specifically: Attention mechanism integration: A coordinate attention module is introduced into the Neck layer of YOLOv11, which makes the network pay more attention to the detailed features of the weld area and defects, and suppresses background interference; Small target detection layer: High-resolution shallow feature maps are preserved and enhanced in the feature pyramid network, specifically for detecting small targets such as pores; Output: Generates a series of bounding boxes containing suspected defects, along with preliminary classification confidence scores, covering the categories of: porosity, incomplete penetration, lack of fusion, cracks, seizures, and concavity; The goal at this stage is to improve the recall rate.
4. The multi-stage weld defect detection method integrating an improved residual network according to claim 1, characterized in that: The S3-trimmed ROI region is input into the dual-branch ResNet network; specifically: Input: Expand each candidate box region output by S2 proportionally, crop the region of interest (ROI) from the original image, and normalize it to a fixed size; Core network: Construct a ResNet-U-Net hybrid network with a dual-branch shared encoder.
5. The multi-stage weld defect detection method integrating an improved residual network according to claim 4, characterized in that: The ResNet-U-Net hybrid network is specifically as follows: Basic skeleton: Based on ResNet-50 or ResNet-101, retain the output feature maps of the first four stages (Stage 1-4) (denoted as L1, L2, L3, L4), with the spatial resolution decreasing step by step (1 / 4, 1 / 8, 1 / 16, 1 / 32 of the original image respectively), and the semantic information being enhanced step by step. Cross-stage feature refinement module: inserted after the residual block of each stage; includes a parallel channel attention and spatial attention submodule, and performs feature fusion and dimensionality reduction through a 1x1 convolution to enhance the encoder's ability to capture weld defects, especially the subtle textures of weak and blurry linear defects; allows the network to suppress background noise such as weld ripples in the early stages and emphasize the edge and internal texture changes of suspected defect areas. Spatial Context Pyramid: Employs a variant of Spatial Pyramid Pooling (ASPP); uses 3x3 dilated convolutions with different dilation rates (e.g., rates=1, 6, 12, 18) to process L4 features in parallel, then upsamples and concatenates them with features from a global average pooling branch, and finally integrates them through 1x1 convolutions; introduces multi-scale contextual information at the bottleneck of the encoder (after Stage 4 output), significantly expanding the receptive field without sacrificing resolution, so that each feature point output by the encoder incorporates information from local details to global context, which helps to understand the morphology of the entire defect.
6. The multi-stage weld defect detection method integrating an improved residual network according to claim 1, characterized in that: S4 has one branch performing pixel-level segmentation, and another branch fusing depth features, segmented geometric features, and grayscale statistical features for fine-grained classification; specifically: Branch 1: Using a U-Net structure with ResNet as the encoder, pixel-level semantic segmentation of the ROI is performed to accurately delineate the outline and internal structure of the defect; the segmentation results help to quantify geometric features such as defect area and length. Branch 2: Shares the deep features extracted by the ResNet encoder with the segmentation branch, and is followed by a multi-head attention mechanism and a fully connected layer.
7. The multi-stage weld defect detection method integrating an improved residual network according to claim 6, characterized in that: The input in the second category includes shared depth features and defect shape descriptors from the segmentation branches, including compactness, elongation, Hu moment, and gray-scale statistical features. The gray-scale statistical features include the gray-scale mean and variance comparison between the segmented region and the background region within the ROI.
8. The multi-stage weld defect detection method integrating an improved residual network according to claim 1, characterized in that: S5: Result fusion and output; specifically: Map the precise bounding box obtained from S4 (the minimum bounding rectangle generated based on the segmentation mask) and the classification results back to the original image coordinates; Non-maximum suppression (NMS) is used to eliminate overlapping boxes; The final output is a structured report containing the defect type, precise location, confidence level, pixel-level profile, and calculated defect dimensions (such as pore diameter and crack length).
9. A multi-stage weld defect detection system integrating an improved residual network, characterized in that: It includes a preprocessing module, a preliminary detection module based on improved YOLOv11, a fine analysis module based on a dual-branch ResNet network, and a post-processing and output module; The preprocessing module employs adaptive histogram equalization to enhance contrast and combines wavelet transform for image noise reduction while preserving defect edge information. The preliminary detection module based on the improved YOLOv11 is used to generate a series of bounding boxes containing suspected defects, along with preliminary classification confidence, covering categories such as porosity, incomplete penetration, lack of fusion, cracks, seizures, and concavity, thereby improving the recall rate. The fine analysis module based on the dual-branch ResNet network is used to enable the network to suppress background noise (such as weld ripples) in the early stages, emphasize the edge and internal texture changes of suspected defect areas, and significantly expand the receptive field without losing resolution, so that each feature point output by the encoder integrates information from local details to global context, which helps to understand the morphology of the entire defect. The post-processing and output module outputs the final result: a structured report containing the defect type, precise location, confidence level, pixel-level contour, and calculated defect dimensions (such as pore diameter and crack length).